xml and web 2.0 – Page 3 – Cubicgarden.com…

It started with a subscription and a email…

I started subscribing to the Dyslexic Advantage, as I have gotten much benefit from the book and decided it would be good to digest much more.

After looking at their premium section, which has a lot of media I started thinking there is maybe too much and they are adding more to it all the time. I started thinking if they have RSS, I could subscribe and get updated media without having to go to the site to check.

Looking at their RSS it was the generic one for the blog no matter which page I went to. There was a note at the end of the podcast section saying if you have troubles or would like it another way ask.

So I did and got into a discussion with Dr. Fernette Eide and Dr. Brock Eide the researchers and writers of the dyslexic advantage. I talked about the advantage of RSS and explained you can have http authentication on RSS to keep their premium content secure.

They were using some other system which was costing them a bit and there was extra step of uploading content to the other system from their dropbox drive, which they nicely shared with me.

Dropbox drive I thought… sure I saw a service which will take a directory of files and generate a RSS feed? I remembered it was called Justcast.com

I set up a account and tried out Justcast for myself and was impressed with how easy it was to get up and running. The one thing which seemed to be missing was authentication on RSS feeds. So I ping them a support request.

Josh from Justcast wrote back pretty quickly… They were on it!

Thanks so much for your suggestion and interest on JustCast, and you know what, we are actually going to work on implementing this Authentication to the feed feature in July. I will definitely keep you in the loop on our development progress.

Following that email we went back and forth and he showed me what it would look like. Then a day ago (29th June) a email

Adding authentication to the feed feature went live. You will able to find the config under the Settings > Advanced. Please give it a try.

I did and it worked exactly how I specified previously. https://user:password@www.justcast.com/mypodcast/blah/index.rss

Perfect… and the Justcast team have so many features, check out their blog. If I was creating podcasts not on archive.org. This would be my number one option now.

In the meanwhile I was equally impressed to see the dyslexic advantage team had taken my advice, converted their whole premium content to Justcast and were asking me to test the RSS feed.

It worked perfectly, no need to have access to the dropbox anymore. I was able to subscribe to the RSS feed (theres a button called subscribe which gives you the full RSS feed link). I was able to add it the feed to my complex setup.

I was impressed with both sides and everything seems so much easier for all now. It reminded me how important it is to take advantage of those opportunities.

Justcast got a new client, dyslexic advantage cut their costs and time to upload and share new premium content. I got my RSS feed(s) with a automated drip of new content as they come.

Another nice unexpected thing came out of the whole thing. The dyslexic advantage team wanted to know my story and may turn it into a recording…!

Clearview AI GDPR request submitted

There is so much to say about Clearview AI. If you never heard of them, well put it this way…

They have amassed a database of peoples faces by illegally scraping the likes of facebook, twitter, instagram, youtube, flickr, etc, etc… All the companies have sent legal cease and desists but Clearview don’t seem to really care too much. Recently they were hacked allowing exposing all those pictures and training data to attackers.

Because of this and my experience with the IBM Dif project, I wanted to know if I’m in the database and the best way to find out is to send a GDPR request. This all follows my GDPR request from Houseparty just recently,

I think they have gotten serious about the EU and the UK because I didn’t need to send my usual email. I filled in the form using my junk mail and used my Estonian digital ID for verification.

Look forward to seeing what comes back. I’m expecting quite a lot.

Of course IBM, Microsoft and Amazon have backed away (for now) from their facial recognition systems because the huge amount of bias of the datasets have against black people. We will see how long they will keep this line over the year and next year?

Update
In my inbox from for the two requests…

EU/UK/Switzerland Data Access Form Request
EU/UK/Switzerland Data Objection Form Request

This e-mail is to confirm that we have received your EU/UK/Switzerland Data (Access/Objection) Request. We will get back to you as soon as possible.

Sincerely,

Team Clearview

My Houseparty GDPR data dump

During the start of the Covid19 lockdown, I was convinced by friends to try houseparty and decided it was pretty crappy so stopped using it as mentioned in a previous blog.

After many emails I finally got my personal GDPR data copy. From Hotel Charlie!

I can’t tell you how much hassle its been… even when they sent me a horribly long link (we are talking about 300 characters long) to the zip file, it would expire a few hours later on their Amazon S3 bucket.

<Error>
<Code>ExpiredToken</Code>
<Message>The provided token has expired.</Message>
<Token-...{very large token}...</Token>
</Error>

Finally once I got it… It was a zip file with a index.html and 5 different folders.

room_visits / room_visits.html
profiles /profile.html
photos / 216A9AAE57194410901F8BA7981E63AB (a png file)
interactions / interactions.html
friends / friends.html

All the .html files are horrible tables for example here is interactions.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Room Visits</title>
</head>
<style>
table {
    border-collapse: collapse;
}
table, th, td {
    border: 1px solid black;
}
</style>
<body>
<table border="1">
    <tr>
        <td>Room ID</td>
        <td>Room Visit Start Date</td>
        <td>Room Visit End Date</td>
        <td>Users</td>
    </tr>
    
    <tr>
        <td>e021116-bae-44d6-cc17-9121fbeaccc13</td>
        <td>
    2020-06-45T21:23:11Z
</td>
        <td>
    2020-06-11T21:16:41Z
</td>
        <td>
            <ul>
            
            </ul>
        </td>
    </tr>
    
</table>

</body>
</html>

The data isn’t that interesting but I think thats because I wasn’t using the app just my Chromium browser. I also only friended one person, so its all pretty slim on data.

Not that interesting but I’m very sure theres lots they have on me, however I requested my account is deleted. There is no way to delete your account if you are using the browser and the Android app from within the system. You have to request deletion!

My next GDPR request is for Clearview AI!

The mixcloud metadata take down

I mentioned a while ago how I was slowly migrating away from Mixcloud as their business model is starting to impinge on people listening to my mixes and I’m not so keen on that. I already mentioned trying to get Funkwhale working and using cue files.

While looking through my mixes I noticed I really didn’t do a great job with some of the metadata. While I had spent most of my time adding to the metadata to Mixcloud (not ideal)

In Mixcloud (once logged in) there are some human friendly urls which I was able to grab images from. The key one being the upload edit page – https://www.mixcloud.com/upload/{username}/{mixname}/edit/ for example https://www.mixcloud.com/upload/cubicgarden/follow-me-into-the-fading-moonlight/edit/

Follow me into the moonlight edit page

My plan was to manually copy the times into my newly written cue files but while talking to Jon about it, he said give him 5mins and he could knock up a script to pull the values out of the HTML page. I thought about it before but using XSLT, however noticed there is a lot javascript rendering making things difficult.

Jon’s quick script written was just what I needed.

#!/usr/bin/env python3

import csv
import sys
from collections import namedtuple
from typing import List

import bs4
from bs4 import Tag

SongInfo = namedtuple('SongInfo', ['number', 'artist', 'title', 'time'])

def load_html(filename: str):
    with open(filename, 'r', encoding='utf-8') as fo:
        return ''.join(fo.readlines())

def extract_song_info(song: Tag):
    try:
        number = song.find(class_='section-number').text
        artist = song.find(class_='section-artist').text
        title = song.find(class_='section-title').text
        time = song.find(class_='section-time')['value']
        result = SongInfo(number, artist, title, time)
        print(f'Extracted {result}')
        return result
    except AttributeError:
        print(f'Error with item {song}')
        return None

def parse_table(input_html: str):
    soup = bs4.BeautifulSoup(input_html, features="html5lib")
    songs = [row for row in soup.find_all(class_="section-row")]
    return [x for x in [extract_song_info(song) for song in songs] if x is not None]


def save_to_csv(file_name: str, songs: List[SongInfo]):
    with open(file_name, 'w', encoding='utf-8') as fo:
        writer = csv.writer(fo)
        for song in songs:
            writer.writerow(song)


if __name__=="__main__":
    if len(sys.argv) != 3:
        print('Usage: extractor.py [input_html_file] [output_csv_file]')
    html = load_html(sys.argv[1])
    songs = parse_table(html)
    save_to_csv(sys.argv[2], songs)
    print(f'Saved to {sys.argv[2]} successfully - Done!')

With it and the HTML pages, which I almost got with Chromedriver, again thanks to Jon, but I couldn’t be bothered to sort out the cookies, etc. I quickly wrote a quick /dirty bash script and fired up a terminal.

#!/bin/bash
./extractor.py $1.html $1.csv
# Verify
echo Details for $1

I thought about modifying Jon’s script to generate the cue files directly bypassing the csv file but decided I should just get them all done. Because I still need to get funkwhale going.

I did notice the edit page doesn’t include genre or the year of the mix, but I can live with that, for now… Scraping web pages is certainly a throw back but its better solution that what I originally was thinking.

This will teach me to sort out my own house of data!

Every once in a while its a win win for all, except the algorithms

Every once in a while I like messing with the algorithms which rule our world. As Cory says in this critical piece, found via Ade,

Machine learning is fundamentally conservative, and it hates change. If you start a text message to your partner with “Hey darling,” the next time you start typing a message to them, “Hey” will beget an autosuggestion of “darling” as the next word, even if this time you are announcing a break-up.

This isn’t a new thing and I have to thank Miles who gave me the idea a long time ago to mess with the algorithms every once in a while.

Every once in a while, when I feel the recommendations are getting pretty good I buy something completely different. For example with Google I’ve done some very strange things, but the impact isn’t so clearly felt as with shopping algorithms.

Very cool to see free sanitary products in the @BBC #MediaCityUK toilets (seen in the disabled toilet).
Really helpful for all women including guests and visitors not just staff. @BBCNorthPR https://t.co/H0nfP9kMHc pic.twitter.com/IX11Hhvptw

— Ian Forrester | @cubicgarden (@cubicgarden) November 5, 2019

Recently I bought tampons which were 2 for the price of 1 on Tesco online. I bought them because I wanted to screw up the algorithm but more importantly I wanted to support my female colleagues (extra special shout out to Jasmine) who have been fighting the good fight to provide women & girls with free sanitary products in BBC buildings. As they really should have!

Donated sanitary products to @bbc staff in @mediacityuk recently. Bought for a good cause – https://t.co/yLrAWNTjMi but also messed up @tesco's algorithm a little bit.
I was in the disabled loos btw, just in case anybody was wondering! pic.twitter.com/diaz7n4bYn

— Ian Forrester | @cubicgarden (@cubicgarden) January 8, 2020

Maybe this is a triple win, one for my colleagues, two for messing up Tesco’s recommendations and three for my pocket? What ever it is, I noticed Tesco recommendation now includes pointers to shampoo products which I certainly don’t need but makes me laugh the algorithm is so easily manipulated.

Already planning similar on Amazon and Ebay…

Decentralisation an important step forward

Its easy to think decentralisation is a new fanged thing the savvy technorati talk about while drinking their double macha latte. But the importance of decentralised networks is made very clear in this VOX piece and the video.

The secret ecosystem of my personal data is being prepared

Recently in the last public service internet note, I posted…

The secret ecosystem of personal data is being unfolded

Ian thinks: People are having fun with this right now, wonder how many people will actually request their data? I put my request in a few days ago, will you?

I sent my requests off a few days following my GDPR dating data template. I’ve had quite a few replies from Sift in the last few weeks.

Starting with this one a day after my formal GDPR request

Thank you for reaching out to Sift. Due to recent press coverage, we are experiencing a high volume of data access requests. We are scaling our operations to accommodate all requests and appreciate your patience. Please expect a followup email to help us verify your identity so that your data does not fall into the wrong hands. Separately, we’ve answered a few commonly asked questions below.

What does Sift do?
Sift provides fraud prevention services to online businesses, e.g. e-commerce. Our goal is to make the internet a safer place so that businesses and their users (like you) don’t have to worry about fraud. You can learn more about our mission here.

We only use your data to provide fraud prevention services to our customers – we do not sell, share, or use your data for any other purpose. For more details about how our service works and what types of data we process, please see our Service Privacy Notice.

How may I access the data that Sift processes about me?
In order to process your data access request, we need to verify your identity to ensure that we are sharing your data with you and not a fraudster impersonating you. As unlikely as that sounds, it happens more often than you’d expect. Please expect a followup email with instructions on how to verify your identity.

How soon will Sift process my request?
Once we verify your identity, we will honor all requests under the European Union (EU) General Data Protection Regulation (GDPR) within ninety (90) days per Article 12(3) of the GDPR for verified EU citizens only. Please note we are extending this period by sixty (60) days due to the high volume of requests.

All other requests, including those from the United States, will take more time. We thank you for your patience as we must give priority to those requests for which our timely response is legally required.

Can you provide my score?
If you have requested your “Sift Score” or other type of consumer score, we’d like to clarify that Sift does not have a “Sift Score” for you (or any user) because we don’t score users; we score user interactions on a specific website for a specific type of fraud. We calculate the likelihood of whether actions you have taken on a Sift customer site are associated with specific types of fraud. The actions we analyze depends on the particular Sift product our customer uses.

However, these interactions do not add up into a single Sift Score about you. A single score is not an effective way of assessing fraud. Instead, the best way to predict fraud and provide users like you the best possible experience is to analyze each specific interaction. For more information on scores, please read our blog post here.

And then a few days later…

Thank you for contacting Sift support! We received your email and typically respond within one business day for questions related to Sift’s suite of products. In the meantime, you can browse our Help Center for answers to some common questions: https://support.sift.com/hc/en-us

Best,

The Sift Support Team

Finally I got the email to verify my identity, which needed to be done within 14 days of the email with a unique link. Which I needed to type in my phone number for the service to then send another unique link to my phone.

Verification is done via a 3rd party service called Berbix inc, and required me to scan my driving license or passport then a selfie and the site tells you to strike a pose and take a selfie (prove its not just a photo). Its all done on the phone using chrome browser rather than an app (thankfully). I had a read of their privacy policy of course and Sift’s.

Now I’m looking forward to seeing what they send me back…

Public Service Internet monthly newsletter (Nov 2019)

The first Velvet Underground album only sold 10,000 copies, but everyone who bought it formed a band - Brian Eno — The first Velvet Underground album only sold 10,000 copies, but everyone who bought it formed a band

We live in incredible times with such possibilities that is clear. Although its easily dismissed by looking down at our feet or at the endless attempts to regain our trust from the big corps.

To quote Buckminster Fuller “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.”

You are seeing aspects of this happening with hard work going into building an open hardware ebook reader.

A framework for human values

Ian thinks: This work is so essential for all public service, non-profits and government organisations. Starting to chip away at what value means beyond the attention economy.

Yancey co-founder of Kickstarter talks about a new framework called bentosim (full episode)

Ian thinks: Yancey talks a good game about going beyond financial maximization and society changes but I’m not convinced about bentoism.

Another attempt at the decentralized file-storage system

Ian thinks: Its another attempt, good idea combining projects but wondering about the applications of use?

China’s free market system grab on other economies

Ian thinks: Maybe Jamies conspiracy is a little heavy but a good thoughtful podcast

Introducing the Dweb

Ian thinks: good introduction by ex Mozillan written a few years ago but parts later are up to date

Panel about sex-tech from Techcrunch (NSFW)

Ian thinks: Sex tech grows its own infrastructure to over come the adolescent thoughts of the tech industry

He used the tech and wasn’t used by the tech

Ian thinks: Vinnie and Douglas talk about the importance of the human element in music and everything.

Why you shouldn’t go to Harvard?

Ian thinks: Got to love Malcolm Gladwell’s analysis of the university system, although maybe not quite right. He’s funny and rolls the research into a great story.

The secret ecosystem of personal data is being unfolded

Ian thinks: People are having fun with this right now, wonder how many people will actually request their data? I put my request in a few days ago, will you?

Public Service Internet monthly newsletter (Oct 2019)

Carole Cadwalladr & Paul-Olivier Dehaye's deep dive into the great hack

We live in incredible times with such possibilities that is clear. Although its easily dismissed by looking down at our feet or at the endless twitter fighting.

To quote Buckminster Fuller “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.”

You are seeing aspects of this happening with Matt Mullenweg’s comments about a open and diverse web after buying tumblr.

Don’t forget if you find this useful, you will find “Public Spaces, Private Data: can we build a better internet?” at the RSA London on 21st October 2019, right up your street.

Watching the labrats scurrying away

Ian thinks: Recently read Labrats book after seeing Dan Lyons at Thinking Digital. Its quite a raw insider view on silicon valley culture, the laughable and the horrific sides in equal lashings.

The Great Hack Workshop from Mydata 2019

Ian thinks: This was one of the highlights of Mydata 2019. Carole Cadwalladr & Paul-Olivier Dehaye’s deep dive into the build up to the great hack was fascinating. Lots of useful resources were revealed.

Are Boris Johnson’s PR People Manipulating Google Search?

Ian thinks: True or not, our dependence on a single search engine/service makes any potential manipulating even more impactful.

Ted Nelson on Hypertext, Douglas Englebart and Xanadu

Ian thinks: Its always amazing to see pioneers who narrowly missed out pushing concepts which were too early, but could come back.

Look out here comes the hyperledgers

Ian thinks: More ledger/blockchain projects to power your projects than you can shake a stick at. Very happy at least some are open-source.

ReasonTV’s look at the Decentralised web

Ian thinks: I was expecting something light touch but having Cory Doctorow mainly interviewed means its got some depth.

Etiquette and privacy in the age of IoT

Ian thinks: Etiquette tends to be forgotten in the advancement of technology. I don’t consider it rude to shut off a Alexa, I’m sure others will disagree.

Tipping etiquette set by user interface

Ian thinks: Talking about etiquette, very interesting to see norms set by user interface design decisions. Obviously set to benefit the company but its stuck now.

Exploiting technology or exploited by technology?

Ian thinks: Curious tale, but it does raise a question about digital access and backups. Least we forget about power and when things go technically wrong.

Public Service Internet monthly newsletter (Sept 2019)

We live in incredible times with such possibilities that is clear. Although its easily dismissed by looking down at our feet or watch how democracy is being gamed and broken. To quote Buckminster Fuller “You never change things by fighting the existing reality. To change something, build a new model that makes the existing model obsolete.”

With a focus on new models in business, technology, society, policy, processes, etc. I present my public service internet newsletter.

You are seeing aspects of this happening as people rethink public transport.