funkwhale – Cubicgarden.com…

Adding activitypub or switching to peertube?

Mixcloud warning: You've reached your limit for published shows

In my latest mix the interdimensional transmission mix, I went to mixcloud to find, I’ve reached my limit for published shows. So been thinking about what mixcloud does do for me. Mainly distribution and a bit of charting.

On the distribution side, it seems like the fediverse could easily do a better job. Originally I would love to use funkwhale but from previous experience it doesn’t really support mixes well compared to single tunes. Which is why I setup my own webmixgarden.

Currently I’m using a static site generator (pubili) which is great because I could add webmonitisation. Although I’m wondering if I should switch to one which supports activitypub? However I feel there is a way which combines both and its likely another platform on the fediverse? I can’t be the only one who is putting out mixes right?

I tooted and had some suggestions. But decided to look at peertube to see if that could work. After finding a instance, which had rules which worked for what I was doing. I starter experimenting here but i’m waiting for my mix to be moderated (its a one man instance, so expect it might take some time). If things work out I might just install peertube on my server.

Update

My mix is now up on peertube via rankett.net, thats the big news.
However I wanted to talk through what happened.

William who runs the rankett instance of peertube messaged me via email asking about copyright details of the mix. This was a surprise because we are all use to faceless entities moderating via machine learning. We had a little chat over email and I explained what I was doing and shared this blog post. Happy with the explanation, my mix was unblacklisted and its now available for all to play.

I’m currently looking at ways to add chapters markers to the mix, by slightly abusing the captions feature. I can’t really see another way to do it.

Another update

LPS messaged me on mastodon and pointing out a undocumented feature of peertube.

I can easily add time points in the description and with the correct formatting will automatically turn into jump points into the mix.

We also talked about setting up peertube using yunohost, which I’m already using. So theres little excuse now for me to just set it up one day soon.

Replacing Mixcloud with Funkwhale

For a long while I have been threatening to leave Mixcloud in favour of hosting my own mixes. I looked around and thought funkwhale looks great as its a federated network for music. With some help from JonT, I started to scrape the metadata I stupidly forgot to keep for myself.

I installed Funkwhale on the Yunoserver but spent too much time trying to work out how to mount my NAS on the Yunoserver. I gave Navidrome but decided it wasn’t right as there was no way to listen without logging in, so went back to Funkwhale and got things up and running.

Seems Funkwhale might not be the best solution for the mixes, plus the developer is looking for new maintainers recently. Its really setup for single tracks not mixes. I could upload mixes but my plan to use cue files, won’t work. The only place to put playlists is in the comments. I also need to do more digging as I can’t change the year of the mixes. More importantly, the public sharing is a bit broken for me. You should be able to listen to the radio but its not working for me

As I get my head around it all but you can subscribe via RSS and if you’re using Mastodon or other fediverse applications, you can subscribe to this account: @digitalitalicmixes@mixes.cubicgarden.info.

Enjoy, I’m one step closer to self hosting my mixes.

Its been a while… the mayday rave up in your flat live mix

Its been a while since I uploaded a mix to mixcloud. I have plans to self host my own mixes using funkwhale or airsonic. But its not quite happened yet. So in the meanwhile I have done a few mixes and I’ll do the posse thing and upload it to mixcloud with a smaller amount of metadata.

Of course I’ll be keep a copy of my metadata for when I get my own system up and running. In the meanwhile the mixes will form not travels with pacemaker but locked in mixing out.

I present the mayday rave up in your flat live mix. Recorded live while mixing live on R&Dio the other day. The volume levels were a bit high sorry but its not horribly distorted

Here’s the tracklist… (at about 133bpm) as always enjoy!

Gouryella (fall from heaven remix) – Gouryella
The Desert – protoculture
Arise – victor ruiz & d nox
Switch (oliver klien & peter jurgens remix) – beckers
Special K (timo maas remix) – Placebo
Prosac (original mix) – Tomcraft
Floyd (extended remix) – Jerome isma-ae & alastor
Soundbar (extended mix) – Giuseppe Ottaviani
Cuckoo – Jon Mangan
Breathe (Blake Jarrell remix) – Anna nalick
Cut and run feat Emma lock – Beat Service
Freedom (extended mix) – Arty vs Muvy
Tears (protoculture remix) – Dakota
Megalodon – MaRLo
Grotesque – RAM & Alex M.O.R.P.H
The Legacy 2.0 (alphazone remix – Funabashi pres Saltwater
Anahera (extended mix) – Ferry Corsten pres Gouryella
Echoes (extended mix) – Henrik B, Niklas Gustavsson, Peter Johansson
Higher State of Conscientiousness – Josh wink

The mixcloud metadata take down

I mentioned a while ago how I was slowly migrating away from Mixcloud as their business model is starting to impinge on people listening to my mixes and I’m not so keen on that. I already mentioned trying to get Funkwhale working and using cue files.

While looking through my mixes I noticed I really didn’t do a great job with some of the metadata. While I had spent most of my time adding to the metadata to Mixcloud (not ideal)

In Mixcloud (once logged in) there are some human friendly urls which I was able to grab images from. The key one being the upload edit page – https://www.mixcloud.com/upload/{username}/{mixname}/edit/ for example https://www.mixcloud.com/upload/cubicgarden/follow-me-into-the-fading-moonlight/edit/

Follow me into the moonlight edit page

My plan was to manually copy the times into my newly written cue files but while talking to Jon about it, he said give him 5mins and he could knock up a script to pull the values out of the HTML page. I thought about it before but using XSLT, however noticed there is a lot javascript rendering making things difficult.

Jon’s quick script written was just what I needed.

#!/usr/bin/env python3

import csv
import sys
from collections import namedtuple
from typing import List

import bs4
from bs4 import Tag

SongInfo = namedtuple('SongInfo', ['number', 'artist', 'title', 'time'])

def load_html(filename: str):
    with open(filename, 'r', encoding='utf-8') as fo:
        return ''.join(fo.readlines())

def extract_song_info(song: Tag):
    try:
        number = song.find(class_='section-number').text
        artist = song.find(class_='section-artist').text
        title = song.find(class_='section-title').text
        time = song.find(class_='section-time')['value']
        result = SongInfo(number, artist, title, time)
        print(f'Extracted {result}')
        return result
    except AttributeError:
        print(f'Error with item {song}')
        return None

def parse_table(input_html: str):
    soup = bs4.BeautifulSoup(input_html, features="html5lib")
    songs = [row for row in soup.find_all(class_="section-row")]
    return [x for x in [extract_song_info(song) for song in songs] if x is not None]


def save_to_csv(file_name: str, songs: List[SongInfo]):
    with open(file_name, 'w', encoding='utf-8') as fo:
        writer = csv.writer(fo)
        for song in songs:
            writer.writerow(song)


if __name__=="__main__":
    if len(sys.argv) != 3:
        print('Usage: extractor.py [input_html_file] [output_csv_file]')
    html = load_html(sys.argv[1])
    songs = parse_table(html)
    save_to_csv(sys.argv[2], songs)
    print(f'Saved to {sys.argv[2]} successfully - Done!')

With it and the HTML pages, which I almost got with Chromedriver, again thanks to Jon, but I couldn’t be bothered to sort out the cookies, etc. I quickly wrote a quick /dirty bash script and fired up a terminal.

#!/bin/bash
./extractor.py $1.html $1.csv
# Verify
echo Details for $1

I thought about modifying Jon’s script to generate the cue files directly bypassing the csv file but decided I should just get them all done. Because I still need to get funkwhale going.

I did notice the edit page doesn’t include genre or the year of the mix, but I can live with that, for now… Scraping web pages is certainly a throw back but its better solution that what I originally was thinking.

This will teach me to sort out my own house of data!

A better way to listen to mixes online?

For a while I have been thinking about leaving Mixcloud. Its nothing personal, I think its a great service for djs but its clear their business model is starting to interfere with the listening experience.

I had a thought about what Mixcloud do for me and decided these are the key things.

Hosting the full mix with limited rights problems (unless you do something like play a track from the same author twice)
The community of people and djs in one place
Ability to see the actual tracks within the mix

With this in mind, I thought I’d see what else was out there as I’m lucky enough to be sitting on a large enough internet connection to host my own mixes and become a node on a larger decentralised network. If it was build like the fediverse, that could solve the community side too? After looking around for alternatives I found a new upstart called funkwhale. You may have seen I mention it recently in a previous blog.

Maybe 1 and 2 could be solved but what about 3?

The last part of the puzzle seemed to be the track problem, as you want seamless playback but get an idea of what you are listening to. Mixcloud does this via metadata, which you can create via a slider over the audio waveform. Some DJ systems create this for you like the Pacemaker for example but that metadata is lost in translation I found. If only there was a standard way to define areas of a mix without slicing the audio mix up?

For example, here is a ambient mix I liked recently.

Its by Tonepoet and even they have gone through the effort of adding this metadata to their personal site complete with timings. I do a similar thing but without the timings (which I really should have added since I had them all and entered them into Mixcloud manually.

I looked a number of things including a bunch of playlist formats including pls, m3u and xspf. Even looked at smil and asx to see if they would help, but their problem was player support. The issue seemed to be they all treated their smallest objects as physical files rather than subsets of files. I did buy into xspf thought it was close with this extension.

The extension element allows non-XSPF XML to be included in XSPF documents. The purpose is to allow nested XML, which the meta and link elements do not. xspf:playlist elements MAY contain zero or more extension elements.
<playlist version="1" xmlns="http://xspf.org/ns/0/" xmlns:cl="http://example.com">
  <extension application="http://example.com">
    <cl:clip start="25000" end="34500"/>
  </extension>
  <trackList />
</playlist>

Close but not quite right and player support for extensions was going to be low. This is when I rethought the problem with something like .nfo files and found .cue files. Here is an example…

REM GENRE Electronica
REM DATE 1998
PERFORMER "Faithless"
TITLE "Live in Berlin"
FILE "Faithless - Live in Berlin.mp3" MP3
  TRACK 01 AUDIO
    TITLE "Reverence"
    PERFORMER "Faithless"
    INDEX 01 00:00:00
  TRACK 02 AUDIO
    TITLE "She's My Baby"
    PERFORMER "Faithless"
    INDEX 01 06:42:00
  TRACK 03 AUDIO
    TITLE "Take the Long Way Home"
    PERFORMER "Faithless"
    INDEX 01 10:54:00
  TRACK 04 AUDIO
    TITLE "Insomnia"
    PERFORMER "Faithless"
    INDEX 01 17:04:00
  TRACK 05 AUDIO
    TITLE "Bring the Family Back"
    PERFORMER "Faithless"
    INDEX 01 25:44:00
  TRACK 06 AUDIO
    TITLE "Salva Mea"
    PERFORMER "Faithless"
    INDEX 01 30:50:00
  TRACK 07 AUDIO
    TITLE "Dirty Old Man"
    PERFORMER "Faithless"
    INDEX 01 38:24:00
  TRACK 08 AUDIO
    TITLE "God Is a DJ"
    PERFORMER "Faithless"
    INDEX 01 42:35:00

Perfect, so I took one of mixes, Quiver in the underground and turned the .nfo file into a .cue file.

REM GENRE Tech Trance
REM DATE 2019
PERFORMER "Digital Italic"
TITLE "Quiver in the underground mix"
FILE "Quiver in the underground mix.mp3" MP3
TRACK 01 AUDIO
TITLE "Activator, I know you can (That kid chris mix)"
PERFORMER "Whatever girl"
INDEX 01 00:00:00
TRACK 02 AUDIO
TITLE "Air traffic (Erik De Koning remix)"
PERFORMER "Three drives"
INDEX 01 02:07:00
TRACK 03 AUDIO
TITLE "Chinook"
PERFORMER "Markus Schulz pres. Dakota"
INDEX 01 07:25:00
TRACK 04 AUDIO
TITLE "Opium (Quivver remix)"
PERFORMER "Jerome Isma-Ae & Alastor"
INDEX 01 10:32:00
TRACK 05 AUDIO
TITLE "Surveillance"
PERFORMER "Jordon Suckley & Kutski"
INDEX 01 15:24:00
TRACK 06 AUDIO
TITLE "Nitric (Division one remix)"
PERFORMER "Hybrid system"
INDEX 01 19:02:00
TRACK 07 AUDIO
TITLE "Circa-Forever (Galen Behr & Organ Nilsen remix)"
PERFORMER "Rapid eye"
INDEX 01 23:55:00
TRACK 08 AUDIO
TITLE "Opulence"
PERFORMER "Simon Patterson"
INDEX 01 30:05:00
TRACK 09 AUDIO
TITLE "J'ai envie de toi (Protoculture remix)"
PERFORMER "Armin Van Buuren presents Gaia"
INDEX 01 35:10:00
TRACK 10 AUDIO
TITLE "Z.I.T.A (M.I.K.E's progressiva mix)"
PERFORMER "Hiver & Hammer with Funabashi"
INDEX 01 39:30:00
TRACK 11 AUDIO
TITLE "Kubrick (Extended mix)"
PERFORMER "Jerome Isma-Ae & Alastor"
INDEX 01 42:01:00

Tried it out on a number of media player and they all worked except Plex.

There is a very good chance if I was to run my own funkwhale node/server I could set it to index .cue files and playback the mix in a seamless way like Mixcloud? Theres only one way to find out really… Get Funkwhale installed!

On a side note I am slightly kicking myself now because I entered all that metadata into mixcloud but never stored it myself. I’m going to need to go through 90 of my own mixes and convert my .nfo files into .cue files.

I have the SQLite Database for my pacemaker device with the actual real timings but I need to identify which mix is which one (another thing I should have done, as changed the names of the mix depending on many things). I also hoped mixcloud’s developers guide would come to the rescue but it looks like maybe a GDPR request is my only option if I want the metadata for my mixes?

Getting on the self-hosted train again

A long time ago, accessing cubicgarden.com meant accessing my direct server sitting in my home. I use to run Blojsom on top of Resin server. I was self hosting from my 512k ADSL line with 256k up (remember how fast that use to be to!?)

There were a lot of problems I grant you that but it mainly worked ok, although I didn’t like the sysadmin side of it all, as I was using Windows 2000 as the operating system. At some point I decided to switch to wordpress only because PHP hosting was cheaper than Java, although I got some incredible breaks during my time. In 2014 I moved my blog to WPengine thanks to dotBen

That was a while ago and since then I have massively upgraded my connection speed to 1gigabit up and down thanks to Hyperoptic and upgraded my server quite a bit (6 core AMD with 16 gig of memory). The first thing I did was installed Plex server.

Since then I have been slowly adding more services to my server. I guess the most noteworthy ones being tiny tiny rss, icecast2, plex and zerotier vpn (which I’m considering changing to wireguard with the recent announcements). Tiny tiny RSS is useful as I don’t like what feedly and others are doing with my data. Zerotier VPN is very cool and very much like the old and forgotten Hamachi. Because it uses internal ip addresses (non-addressable?) any device I have it connected with can access those addresses like they are on a internal network. This ultimately means I can access all my services including tiny tiny rss without opening up ports on my firewall and exposing it to the internet.

Anyway I’ve been thinking about adding more services to my server including Wekan (alternative to trello), Pixelfed (feiverse instagram), wisemapping (web based mindmapping tool), wallabag (alternative to instapaper), standardnotes server, mastodon (fediverse twitter), funkwhale (fediverse spoitfy), language tool (alternative to grammerly) and matrix (powerful alternative to slack).

Doing it under Ubuntu isn’t a problem as theres lots of tutorials and theres plenty which use Docker to manage everything.

But there is issue it seems when installing multiple services on top of each other. Most of the tutorials require a Apache or Ngnix then some SQL database. The tutorials are written like you are running just one service alone and things become more tricky when you have services using certain ports, etc. Trying to move the ports, database tables is sometimes tricky to follow.

Right now, I’m focused on doing one service at a time or really getting to grips with Docker which was meant to make this easier to deal with???

Ok so why selfhosting (and there is a lot of self-hosting services as I found here) and all the hassle?

I found something which sums it up nicely from a different but connected context.

Decentralized, peer-to-peer networks are evolutionarily superior to the bastardized corporate ‘sharing economy’ platforms like Uber and Lyft. Their billion-dollar budgets won’t save them from the inevitability of the blockchain-based peer-to-peer economy.

The decentralization revolution is here.