Built in Filter and Algorthm failure

I enjoyed Jon Udell’s thoughts on Filter Failure.

The problem isn’t information overload, Clay Shirky famously said, it’s filter failure. Lately, though, I’m more worried about filter success. Increasingly my filters are being defined for me by systems that watch my behavior and suggest More Like This. More things to read, people to follow, songs to hear. These filters do a great job of hiding things that are dissimilar and surprising. But that’s the very definition of information! Formally it’s the one thing that’s not like the others, the one that surprises you.

One of the questions people have when they think about Perceptive Media is the Filter bubble.

filter bubble is a result state in which a website algorithm selectively guesses what information a user would like to see based on information about the user (such as location, past click behaviour and search history) and, as a result, users become separated from information that disagrees with their viewpoints, effectively isolating them in their own cultural or ideological bubbles. Prime examples are Google‘s personalised search results and Facebook‘s personalised news stream. The term was coined by internet activist Eli Pariser in his book by the same name; according to Pariser, users get less exposure to conflicting viewpoints and are isolated intellectually in their own informational bubble.

The filter bubble is still being heavily debated to if its real or not but the idea of filters which get things wrong to add a level of serendipity sounds good. But I do wonder if people will be happy with a level of fuzziness in the algorithms they become dependable on?

I’m always on the lookout for ways to defeat the filters and see things through lenses other than my own. On Facebook, for example, I stay connected to people with whom I profoundly disagree. As a tourist of other people’s echo chambers I gain perspective on my native echo chamber. Facebook doesn’t discourage this tourism, but it doesn’t actively encourage it either.

The way Jon Udell is defeating the filters, he retains some kind of control. Its a nice way to get a balance, but as someone who only follows 200ish people on Twitter and don’t look at Facebook much, I actively like to remove the noise from my bubble.

As I think back on the evolution of social media I recall a few moments when my filters did “fail” in ways that delivered the kinds of surprises I value. Napster was the first. When you found a tune on Napster you could also explore the library of the person who shared that tune. That person had no idea who I was or what I’d like. By way of a tune we randomly shared in common I found many delightful surprises. I don’t have that experience on Pandora today.

Likewise the early blogosophere. I built my echo chamber there by following people whose lenses on the world complemented mine. For us the common thread was Net tech. But anything could and did appear in the feeds we shared directly with one another. Again there were many delightful surprises.

Oh yes I remember spending hours in Easy Everything internet cafes after work or going out checking out users library’s, not really recognizing the name and listening to see if I liked it. Jon may not admit it but I found the dark net provides some very interesting parallels with this. Looking through what else someone shared can be a real delight when you strike upon something unheard of.

And likewise the blogosphere can lead you down some interesting paths. Take my blog for example, some people read it because of my interest in Technology, but the next post may be something to do with dating or life experience.

I do want some filter failure but I want to be in control of when really… And I think thats the point Jon is getting at…

want my filters to fail, and I want dials that control the degrees and kinds of failures.

Where that statement leaves the concept of pure Perceptive Media, who knows…? But its certainly something I’ve been considering for a long while.

Reminds me of that old saying… Its not a bug, its a feature

Tweet digging the rules

Tweetdig

The concept is simple… Imagine if your Twitter client was crossed with your mail client?

Tweetdig is exactly that.

Currently in private beta although it feels slightly more like a alpha. I was lucky to gain a activation code when they previewed it at BarCampBlackpool last weekend.

So how does it work? well it works very well. Like most twitter clients you have the tweets going up the screen but theres a few options to do a bunch of things including create a filter based on the tweet you’ve selected. You can just write a filter but the best way is to start with something.

Tweetdig

So in the screenshot above I have a folder called BBC discussions which usually come from my boss Adew, Si_lumb, or a number of other people. I actually added a bunch more people from the BBC, so now any tweets from them containing one of the others is classes as a conversation which I might want to be aware of.

The filters are pretty much what you’d expect but there a great start. I’d really like to see a better way to group people, so instead of listing everyone on each line, I could say if anyone in my BBC list says something to 2 or more BBC people, drop a copy of the message in a folder. In actual fact I’m not seeing much in the way of the twitter lists being used in tweetdig. Most likely because they haven’t got around to them yet or maybe because lists are not used much?

I’m sure the team will be all over lists at some point, they may even make it transparent, so when you create a grouping in tweetdig, it actually creates a list automatically?

To be honest, it really needs a lot stabilising, and I know there all working on that right now. But they certainly have plans to make a mobile client and maybe a desktop client. Once you start using it, its hard to go back and even more frustrating to use a mobile client without the rules and folders. In fact it would be great if you could assign clients to a folder. For example my mobile client would only show certain folders instead of everything and me having to cascade through it.

I did have a word with them on the side at the barcampblackpool and asked if they knew of any clients which support filtering or rules? Got thinking it would be good if you could simply export your rules out of the site and into your choice of client instead of waiting for them to create a client for your platform.

I’ve already created the dream filter which removes all tweets with the hashtag to the bin so I don’t have to hear how much fun people are having in Austin next year. I’ve also setup the same for #iphone5 #iphone4s and #ipad3. Yes it may sound a bit crude but what I really need a little more structure so I could say, ignore all those unless @bbcnews tweet something.

As usual I’d also like to see a more Xpath type logic and the ability to do conditional things like, if @tdobson tweets a link to a video and its retweeted by at least 2 of my followers tell me about it (usually I can never trust what @tdobson links to, as its usually balls or great). Also like to see stuff like the ability to mute someone if they tweet more than x times in a set period of time. Automatically send anyone who tweets the same thing over x times in a set period of time to the junk folder.

Lastly I’d really like to be able to feed Facebook/Buzz/Google+/Idents status/updates/messages into the same thing. Maybe these guys are sitting on the perfect idea of the social operating system (stowe boyd?)

You get the general idea…

Tweetdig

The humour of the startup is fun and reflects the people behind it. Its great to see a original idea being pushed forward by these guys who are regular barcampers and such friendly people.

I wish them lots of luck and I think they got something that in the end may be copyable by others but at least they can say, we had the original idea and followed through with a decent product. I can already imagine Tweetdeck or Seesmic with rules which span across not just twitter but also Facebook and Buzz.

So impressed with this service, I’ve closed down Tweetdeck (for now) and made this my number 2 in Top10 interesting tech startups. Novel service with a good solid concept, although I do feel the email methodology will trap them in the end. Its all about the rules 🙂