Tag: data

Better to know or to live in blissful ignorance?

23andme box

Following my brush with death I have taken a much deeper look at my family genes.

But one of the ultimate ways to know more is by signing up to 23 and me.

Although the idea freaks most people out, when you’ve been so close to death theres really little which can freak you out. So I have been considering the process quite a lot and looking through there material and some of the backstory of 23 and me.

Besides my thoughts of a world like Gattaca, I’m wondering if its better to know or live in a world of blissful ignorance? Although I do have to say blissful ignorance is part of the reason why I ended up in hospital. I do however wonder about 23 and me’s terms of conditions. There privacy statement is interesting too…

You may learn information about yourself that you do not anticipate. This information may evoke strong emotions and has the potential to alter your life and worldview. You may discover things about yourself that trouble you and that you may not have the ability to control or change (e.g., your father is not genetically your father, surprising facts related to your ancestry, or that someone with your genotype may have a higher than average chance of developing a specific condition or disease). These outcomes could have social, legal, or economic implications.

Genetic Information that you choose to share with your physician or other health care provider may become part of your medical record and through that route be accessible to other health care providers and/or insurance companies in the future. Genetic Information that you share with family, friends or employers may be used against your interests. Even if you share Genetic Information that has no or limited meaning today, that information could have greater meaning in the future as new discoveries are made. If you are asked by an insurance company whether you have learned Genetic Information about health conditions and you do not disclose this to them, this may be considered to be fraud.

We do not sell, lease, or rent your individual-level Personal Information without your explicit consent. As a general rule, 23andMe will not disclose your individual-level Personal Information to any third party, except under the following circumstances:

  • Partners or service providers (e.g. credit card processors or our contracted genotyping laboratory) process and/or store the information in order to help us provide, understand or improve 23andMe’s Services.
  • If you have consented for research, research contractors may access your individual-level Genetic and Self-Reported Information onsite at 23andMe’s offices for the purpose of scientific research, provided that all such research contractors will be supervised by 23andMe and subject to 23andMe’s access rules and guidelines.
  • If you have consented to use of your individual-level data in the Research Portal feature, qualified researchers (who must comply with certain requirements) may access your individual-level Genetic and/or Self-Reported Information for the purpose of scientific research, which could lead to commercial use.
  • We are required to do so by law or we do so in coordination with regulatory authorities (see the section below titled “Information Disclosure Required By Law”).
  • You have provided explicit consent for us to do so.

On a whole I don’t quite feel at ease with there terms of conditions or privacy statements. 23andme is a company and that worries me. If they were a charity or something like that, I would feel a slight bit better. How long will it be till they sell out on there users? You only have to look at what’s happened with Flickr to see the bizarre things that can happen.

So its not the information I’m worried about, its the way its stored and disclosed, now and into the future…

 

 

Future Everything 2013

I had the pleasure of attending Future Everything again this year. Manchester’s answer to SXSW in my own eyes. Now in its 18th year (I believe Drew said to me) its decided to move from the already packed May month to the earlier month of March. As usual theres a conference line up somewhere in the mists of the busy festival of events.

The themes this year are

These are my highlights from the ones I attended…

Future Cities…

Dave Carter

The never conventional Dave Carter is a real asset to Manchester, I can’t give the man enough credit for what he says and what he goes and does… It was great to hear his version of ask for forgiveness not permission.

Martijn de Waal did a talk titled A tale of 3 cities… social cities of tomorrow. In the talk about 3 cities in South Korea, Songdo, Homdu and Seoul City. Songdo was the perfectly designed city of the future, clean, designed and all that. Homdu is organic in its design and gives rise to some strange human made constructions. Seoul City is a responsive city with lots of systems which allow feedback and change. Its almost responsive in nature.

Rest of the talk was about the differences and how the platform of the city can best help the citizens within it. Which kind of city would people like to live in kept coming up, and generally a balance of all three seemed to be the general view.

I could hear the sharp intake of breath when Scott Cain of the TSB (Tech strategy board) made a comment about something being in London because that makes the most sense. But no one picked him up on it which seemed a missed opportunity.

Redsigning the Future

The redesigning the future talk was interesting but bugged me…  I think it bugged me for being very vague and not revealing a lot. I certainly got a lot more out of the talk with Magnus at Thinking Digital 2012. There were some stuff which was thrown out including the notion of “Super density” which I gather is the opposite of unevenly distribution. A day made of glass was mentioned a few times along with the science fiction condition and internet fridge too.

Which leads me nicely on to the after event called ideas are theft.

It sold its self outspoken, fun, spiky and dangerous but it turned into one of the biggest let downs in Future Everything history. What got me was there was some great panellists including Dave Mee, Usman Haque and Natalie Jeremijenko. All would be fun and could talk about stuff in a spiky dangerous way if the moderator would shut up, questions were any good and made sense. The 2nd half was better but to be honest the damage was done, people started talking within themselves and the guests looked pissed off. I know it was meant to be funny but it felt very amateur which isn’t what I associate with Future Everything.

On the Data Society front…

The super smart Mel Woods seems to be the person behind the interesting project I experienced called Chattr. The premise is simply to wear a microphone and have your conversation turned into a twitter transcript. You can see the transcripts if you look at the twitter bot ChattrLeaks or hashtag. There was a delay as everything was recorded then on handing the recorder back its send to the 3rd floor to be transcribed and tweeted. For me it was the balance of privacy which was super interesting. For example a conversation later with a freelancer had to be deleted because I didn’t feel comfortable with it being tweeted even though I was very careful not to repeat anything she said.

Of course when I first got the mic, I couldn’t help but spill lots of pearls of wisdom to the world…

“I would never invite someone over to my house on a first date” #chattr

— Chattr Leaks (@ChattrLeaks) March 22, 2013

The point of the project is to feel the tension between public and private. For someone like me to feel that tension, it certainly did the job well. Really got me thinking Mel, well played!

Farida Vis and Usman Haque had a session I wish I had attended from the very start. Living in an age of Algorithmic Culture is something I’m very interested in, specially in regards to big data. They digged into the idea of algorithms and are they useful to us? Farida joined the algorithm with the health of a company. Which got me thinking about something I saw where the company banned certain users from inputing more data because it was unbalancing the algorithm and causing excess processing time. Could it be possible to starve or bloat an algorithm (ultimately hack it) to slow down the processing? Farida and Usman did agree, that most startups use external processing power and yes that could if left unchecked cause excess processing and therefore money.

I’d love to dig into love in algorithms with these guys one day, but thats another blog post and maybe more soon.

API Economy

On the Creative Code front I saw a number of mini-hack events and also a good discussion about the Politics of Open Data and API Economy. Some good thinking about moving away from the big players such as Facebook and Twitter. Also talking about not just simply running to the next big player, so no running to Google plus (specially with whats happening with Google reader!)

There was a thought that the only way to run a API was to charge for it which had me reaching for the sky but there was so many questions I missed my chance. There were a number of artistic talks but none really stuck in my head or had me typing on my tablet. Bringing the archive to life with BBC’s Tony Ageh was interesting to hear where we are years later. Tony even suggested a date of finishing, which if I remember correctly was 2017? Awesome work… Except I have no idea why there was a makeie doll on the panel? Maybe only Bill Thompson knows…?

Makie

The Future Everything Summit was a good one, the venue in Piccadilly place is a lot better than MOSI and I liked the little touches like the honestly payment system for lunch and the like. I do agree with Imran that the layout and signage could do with a designers eye because it didn’t make total sense. I did like the fact hacks and bof/unconference events were happening in the spare spaces, this felt closer than years previously. I gather there was a lot of speakers who dropped out at the last moment but it all worked and it felt like a good event. You could hardly go wrong for less than 100 pounds.

Good job Future Everything, I look forward to other summits through out the year?

Why I shut down BBC backstage

BBC Backstage Meets the NW communities networking bash

George sent me a tweet saying how much Elizabeth Murdoch loved BBC Backstage, as she mentioned it in her speech to the Edinburgh TV Festival last year.

The BBC has been the market leader for building new relationships and services with creative’s from every sector. Be it the early ground breaking Backstage initiative for technology engineers.

Shes right and it does beg the question, why is there no more BBC Backstage?

I thought this was covered in the BBC Backstage ebook which was put together by the lovely Suw. But it looks like I may have been slightly mistaken. On top of this, I keep making reference to this blog post which I never seem to quite finish. So enough, its finished and out there for all to read…

First misconception: The BBC never shutdown BBC Backstage

Actually I did… When I first mentioned the possibility of closing down BBC Backstage to Adrian (my manager) he thought I had totally lost it. I remember a meeting with Adrian and Matthew (head of R&D) where I talked about shutting it down and I gave my reasoning which made soften the blow a little. I had thought long and hard about leaving BBC Backstage and passing it on to someone else younger and full of energy (I even had a number of names put forward to consider). But it didn’t make sense.

The problems with Backstage were not about who was running it but more about what was happening around it (as we will see in number 4)

Second misconception: The BBC sits on a ton of data.

The core of BBC Backstage was the backstage license which is founded on non-commercial reuse of data. This gave backstage the license to go around the BBC educating/persuading/convincing stakeholders about the benefits of open data at a time when data wasn’t a big thing. The problem is the data wasn’t ours. For example the Met Office would make the weather data available to the BBC under strict licensing. Deals were done for non-commercial use and it was always neigh impossible to reverse a deal without effecting the production side of the things.

Lots of people imagine most of Backstage was hacks. In actual fact lots of it was people experimenting.

Third misconception: Developers found new business models

This backs off the non-commercial problem. Because everything was under the non-commercial license, when things like the Apple App Store came along and offered developers clear ways to make money from their work. We had to shut down a lot of prototypes and tell people not to use BBC backstage data in there apps.

This was actually a issue from early on when Google Adsense, offered developers a nice way to make a small amount of money based on numbers of people who came to the site. It was argued that if developers made enough money to just cover the hosting of the prototype, we could turn a blind eye to. This wasn’t sustainable as it kept coming back to bite every once in a while. But it wasn’t till the App stores when the number of prototypes and services wanting to go commercial blew up.

Once developers learned it was actually against the terms and conditions, they naturally moved on to other platforms.  We did talk to BBC Worldwide many times about working together but it just wasn’t to be.

Forth misconception: The Open Data Revolution passed it by

Backstage had a hand in getting this revolution going in the UK and beyond. 7 years later, we had influenced everyone from other companies to the government. We were there right at the start of this revolution and fundamentally changed the BBC’s thinking about data. However it was clear this was just the start and as a part of BBC R&D, it was right to move on and have the same impact in another emerging area. The developer network part of Backstage was tricky to balance with the push to drive forward.

We did think about splitting it off and working in partnership with others who were later to the scene but it just didn’t quite happen and in the era of cost cutting and doing the things which really have an impact for our audiences it was harder to justify.

Fifth misconception: It was all about DRM and the BBC wanted rid

Looking at the mailing list, its easy to imagine it being all about DRM and not a lot else. But in actual fact while the DRM debates rages on, there were lots of people creating and making lots of prototypes. Lots of them were documented on the website but there were some which were so illegal there was no way I could put them anywhere public. Those were more of a look what we could do…

Even though they were much more black/grey around the licensing terms, they drove the imagination and clearly got a number of us thinking what if…? One such example is the widely talked about blast from the past called Panadora PVR (now called Promise.TV) which lead to Tom Loosemore’s talk at Etech 2007, the Edinburgh TV unfestival and the building of the infamous BBC Redux.

The BBC gained a lot from having the debate and being rather open about it all.

Sixth misconception: There was no money or love for BBC Backstage

This is somewhat true and false. Yes it became more difficult to justify and we had gone through quite a difficult patch, while losing some key people to project. On top of that we had a new head of Future Media (Erik Huggers), moved into BBC R&D and was shifting the project up to the north of England to fit in with BBC’s increasing push to solve the London and South East bias.

Everything was changing and everytime we took BBC Backstage in a different direction, there was push back from the dedicated community. To me this is the way of the world (forever changing) but it certainly makes funding such projects difficult when you want a 3-5 year plan.

There was much love for BBC Backstage from Future Media and other departments in the BBC, there was lots of talk about setting up other Backstages in different areas as a outreach project alone it hit audiences the BBC was not so good at having conversations with. The formula was repeatable but should it be? We could have done Mashed all over the UK but was that a good idea? I certainly didn’t think so and ultimately my thoughts about driving forward were correct.

Seventh misconception: We ran out of steam

Ok this might be true to a certain extent. But not from the lack of trying… You only have to look at the new things I’ve been working on since, including Channelography, Perceptive Media, etc. There is still fire in myself and I still have a lot to give… During that time, I will admit I was well over worked and I was being contacted by many people on the off chance just because I was out in the open. This certainly slowed down daily looking through BBC emails. Hence why I now have a another BBC email.

Ultimately I want to thank everyone who has been involved in BBC Backstage in the past (too many to name). The decision was made under a ton of stress on my part but I felt I was making the correct decision for everyone including the founders, the BBC and the community. Then and even now. I mean can you imagine BBC Backstage in 2013!?

Things need to end (such as BBC Backstage, Innovation Labs, etc) for others to spark, grow and mature like BBC Connected Studio.

 

Dream journal metadata

I said a while ago how I found a site which is much closer to mydreamscape than anything else I’ve found to date.

One of the key parts was the dream journaling and its interesting to see the amount of metadata lucidpedia requires.

Title, date, dream… is fine and they actually have a level of privacy

If you like, you can annote dreamsigns by enclosing words or sentences with [ds] … [/ds] tags.
Parts of your dream description that you rather prefer to be private when you decide to share this dream with others,
use the [private] … [/private]

Cool but, how about,

  • Sleep time – First state the time when you went into bed and then input the time when you woke up in the morning
  • Image and Video – Add your own Image or YouTube movie to this dream!
  • Rating out of 5
  • Lucidity – Were you not lucid, half lucid or fully lucid anytime during this dream?
  • Characters – Which people that you know from real-life did you meet in your dream?
  • Permissions which are…

Share with Lucidipedia.com users only
Only people with a Lucidipedia account are able to read, rate and comment on this dream record by using Dream Journal.

Make this dream readable for everyone
Share also with anyone outside of Lucidipedia.com, Anyone you provide with the URL of this dream record would be able to read it. Great for sharing on Facebook or Twitter with friends!

Thats one heck of a load of metadata about a dream…! By the time you filled all that in I’m sure you would have forgotten most of the facts… I’m fascinated by the idea of sticking a video or image into the dream but I’m sure it would be used in the right way.

Why no twitter import?

I said on Techgrumps 66 recently that I found it very strange that there was no Twitter dump import tools/services?

Now I have my twitter data/dump, I want to host it myself which is pretty easy because there is JSON and HTML (already thought about transforming it into XML for easier transformation in the future) but I don’t understand why Tent.io, Status.net, Facebook or heck Google don’t have an option to import your twitter data?

When Twitter made the data available to you, it was clearly yours. Aka you own the rights to all of it. So you can do what you like with it now. Including handing it over to someone else to mine and use if you so wish.

It was something we talked about again while at the dataportability group, data providers who would handle, mine and make sense of your data on your behalf.

I would consider allowing Google mine the data, if it improved my already great Google Now experience.

I would also like to see my Tent.io allow me to upload all my tweets, therefore making it a heck lot more interesting and useful.

Update… Imran reminds me Timehop supports your twitter data, actually I’ve been looking at singly.com recently…

Reconnecting with half the memories

hajimemashite watashin wa. Ian desu. dozo yoroshiku

Its great looking through my Twitter archive/dump but its frustrating that I can only read half of the conversation.

Funny looking back at this tweet as it was less than a week before I had my brush with death…

6 May 10
These early morning meetings are killing me, wondering what people would say if I started setting meetings for 6pm and 7pm?

Yes just imagine… Anyway I can clearly see the gap in my tweeting From 7th May 2010 to June 9th 2010. Not a single tweet…! I also noticed theres no Direct messages in the archive so I can see… Which is a good and bad thing I guess?

Anyway enough doom and gloom…

Here’s is that classic moment which I’ve used to talk about how great open sharing can be/or how twitter is better than facebook.

From 4th January 2009…

Unfortunately you will need to read it from the bottom upwards

Although you know what happens, so it doesn’t really matter 🙂

“thank you but…” passing the card back.”I have a boyfriend, he’s picking me up from the station” so no joy but thanks twitterverse View on Twitter

so I gave her the card and smiled. she looked at it read it with richards jp on one side and my contacts on the other and said…. View on Twitter

Ok headphones are off and shes packing up. Here comes my moment. Geez  View on Twitter

@richardsproject: Ok Richard wrote it, the card is ready. I just need to know what it says View on Twitter

At Stockport View on Twitter

So i was planning to offer her one of my kitkats but i like the card thing. if Richard tells me what it says  View on Twitter

@richardsproject: Tell me what it says and i’ll do it View on Twitter

@sheilaellen: Nope only 10mins left View on Twitter

@JeniT: Ummm like what? View on Twitter

@billt: Ok now i’m laughing out loud, thanks Bill. Think she is running Windows too actually. But shes watching a film View on Twitter

Just passed Mansfield so not long till we get to Manchester View on Twitter

@cisnky: Now that would be good. But how do I start that conversation 🙂 Geez I wonder what @tommorris makes of all of this View on Twitter

@davemee: The twitterverse is my strategy, come on people I got 30mins left to make a good impression and type up my blog entries View on Twitter

@sheilaellen: Shes got headphones on, so that won’t work View on Twitter

Sitting opposite a stunning Japanese lady with a oversized acer laptop. Playing footsie under the table but no joy… View on Twitter

Microblogging dataportability at last?

Twitter data dump

Finally got the ability to download my tweets… Over 6 years of tweets in 6.8 meg of files.

It comes in a zip file not a tar file which is interesting because Facebook uses Tars for its data dumps. Structures interesting because its less of a dump and more a formal backup of your data complete with HTML file bring it all together. Theres a README.txt file which reads…

# How to use your Twitter archive data
The simplest way to use your Twitter archive data is through the archive browser interface provided in this file. Just double-click `index.html` from the root folder and you can browse your entire history of Tweets from inside your browser.
—
In the `data` folder, your Twitter archive is present in two formats: JSON and CSV exports by month and year.

  • CSV is a generic format that can be imported into many data tools, spreadsheet applications, or consumed simply using a programming language.
  • ## JSON for Developers
  • The JSON export contains a full representation of your Tweets as returned by v1.1 of the Twitter API. See https://dev.twitter.com/docs/api/1.1 for more information.
  • The JSON export is also used to power the archive browser interface (index.html).
  • To consume the export in a generic JSON parser in any language, strip the first and last lines of each file.

To provide feedback, ask questions, or share ideas with other Twitter developers, join the discussion forums on https://dev.twitter.com.

Most of the data is JSON which bugs me a little only because I would personally have to transform it all to XML but alas I’m sure everyone loves it. The CSV spreadsheets are odd and could do with being XML instead of CSV but once again sure its useful to someone out there. The nice thing is there is tons of meta around each microblog/tweet including the geo-location, time and device/client. Even the URLs have some interesting things around it, because I was wondering how they were going to deal with shorten urls, retweets and mentions…

 “urls” : [ {
“indices” : [ 69, 89 ],
“url” : “http://t.co/GSzy55vc”,
“expanded_url” : “http://epicwerewolf.eventbrite.com/”,
“display_url” : “epicwerewolf.eventbrite.com”
} ]

Doesn’t always work… specially when using urls shortener which don’t keep the url after a certain time period. Interesting internally twitter always uses its own t.co for everything…

Right now I’m just interested in the period around my brush with death… Real shame theres no references to mentions you’ve had, as I would have loved to have seen some of those. Guess Twitter were not going to delve into that can of worms…

I want to know why theres no status.net inporter?

Cnet have a overview of how and what to do with the archive. Thanks Matt

Welcome to Love in the Time of Algorithms

Imran sent me a link to this book titled Love in the time of algorithms which instantly I instantly liked…

Love in the time of algorithms

The description is exactly what I would write if I was to publish my own thoughts instead of talking about it and doing it. Actually this post pretty much sums up what I think the book is going to cover

“If online dating can blunt the emotional pain of separation, if adults can afford to be increasingly demanding about what they want from a relationship, the effect of online dating seems positive. But what if it’s also the case that the prospect of finding an ever more compatible mate with the click of a mouse means a future of relationship instability, a paradox of choice that keeps us chasing the illusive bunny around the dating track?”
 
It’s the mother of all search problems: how to find a spouse, a mate, a date. The escalating marriage age and declin­ing marriage rate mean we’re spending a greater portion of our lives unattached, searching for love well into our thirties and forties.
It’s no wonder that a third of America’s 90 million singles are turning to dating Web sites. Once considered the realm of the lonely and desperate, sites like eHarmony, Match, OkCupid, and Plenty of Fish have been embraced by pretty much every demographic. Thanks to the increasingly efficient algorithms that power these sites, dating has been transformed from a daunting transaction based on scarcity to one in which the possibilities are almost endless. Now anyone—young, old, straight, gay, and even married—can search for exactly what they want, connect with more people, and get more information about those people than ever before.
As journalist Dan Slater shows, online dating is changing society in more profound ways than we imagine. He explores how these new technologies, by altering our perception of what’s possible, are reconditioning our feelings about commitment and challenging the traditional paradigm of adult life.
Like the sexual revolution of the 1960s and ’70s, the digital revolution is forcing us to ask new questions about what constitutes “normal”: Why should we settle for someone who falls short of our expectations if there are thousands of other options just a click away? Can commitment thrive in a world of unlimited choice? Can chemistry really be quantified by math geeks? As one of Slater’s subjects wonders, “What’s the etiquette here?”
Blending history, psychology, and interviews with site creators and users, Slater takes readers behind the scenes of a fascinating business. Dating sites capitalize on our quest for love, but how do their creators’ ideas about profits, morality, and the nature of desire shape the virtual worlds they’ve created for us? Should we trust an industry whose revenue model benefits from our avoiding monogamy?
Documenting the untold story of the online-dating industry’s rise from ignominy to ubiquity—beginning with its early days as “computer dating” at Harvard in 1965—Slater offers a lively, entertaining, and thought provoking account of how we have, for better and worse, embraced technology in the most intimate aspect of our lives.

Its not available till Aug 15th but is available to pre-order if you so wish…

I’ll be keeping an eye out for this one and hopefully if Dan does a book tour or something I can rope him into doing something in Manchester which has the 2nd biggest singles population in the UK behind London. Maybe it can be a special #smc_mcr event or maybe a return to prestonsocial with something more solid?

The obvious thing would be to do a relationships 2.0?

Its not the first time I’ve seen Dan’s name come up, he wrote this critical piece about dating algorithms. Which is one of the pieces,  which got me thinking about dating sites and are they actually doing what they claim to be doing? His articles reads similar to my own blog if you go by the titles alone. Just need Onlinedatingpost and Datinginsider for a full house? Anyone know how to contact any of these people?

Implicit data is the anti-matter of big data

Dylan [Two thumbs up for Photographers]

Almost everything we’ve focused on recently has been the explicit actions and feedback of people. But as pointed out in Perceptive Media, the rich stuff is the implicit actions and feedback. This is also the stuff which advertisers would cream in their pants for… And it sometimes feels too intimate for us to ever let it be collected… However that has never stopped anyone.

This obviously scares a lot of people including myself but I think the future is about the implicit.

I wrote a blog following a audio piece about how 2012 was the year of big data. But the fundamentally all that data is explicit data not implicit. Something I also made clear during a panel in London at last years Trans-media festival.

In a recently interview Valve’s Gabe Newell talked about the Steam Box’s future. Steam is a very interesting gaming ecosystem and recently Valve’s been moving to Linux after Microsoft said Windows 8 must work the way they said it does. Anyhow the important thing is Gabe’s discussion regarding implicit forms of data

Speaking of controllers, what kind of creative inputs are you working on?
Valve has already confessed its dissatisfaction with existing controllers and the kinds of inputs available. Kinect? Motion?

We’ve struggled for a long time to try to think of ways to use motion input and we really haven’t [found any]. Wii Sports is still kind of the pinnacle of that. We look at that, and for us at least, as a games developer, we can’t see how it makes games fundamentally better. On the controller side, the stuff we’re thinking of is kind of super boring stuff all around latency and precision. There’s no magic there, everybody understands when you say “I want something that’s more precise and is less laggy.” We think that, unlike motion input where we kind of struggled to come up with ideas, [there’s potential in] biometrics. We have lots of ideas.

I think you’ll see controllers coming from us that use a lot of biometric data. Maybe the motion stuff is just failure of imagination on our part, but we’re a lot more excited about biometrics as an input method. Motion just seems to be a way of [thinking] of your body as a set of communication channels. Your hands, and your wrist muscles, and your fingers are actually your highest bandwidth — so to trying to talk to a game with your arms is essentially saying “oh we’re going to stop using ethernet and go back to 300 baud dial-up.” Maybe there are other ways to think of that. There’s more engagement when you’re using larger skeletal muscles, but whenever we go down [that path] we sort of come away unconvinced. Biometrics on the other hand is essentially adding more communication bandwidth between the game and the person playing it, especially in ways the player isn’t necessarily conscious of. Biometrics gives us more visibility. Also, gaze tracking. We think gaze tracking is going to turn out to be super important.

I’ve recently upgraded my phone to run Google now and its so weird…

When talking about it, people say show me and I have nothing to show them except the weather and maybe a couple of calendar things like someone birthday or a appointment I have upcoming. But when waking up this morning, the phone had tons of information about getting to work. Every time I would look at the screen another option was available to me (as time passed). The lack of ability to dig up stuff and look back at stuff is really interesting, as google now is simply that… Now!

Interestingly like google now, I discovered when showing people the first perceptive media prototype, futurebroadcasts.com. I would need to use my own machine because it relies on your implicit data for parts of the play. Meaning I couldn’t just load it up on another persons machine (or at least reliably), and expect it to work the same way.

I already said its the difference which in the future will be more interesting than the similarities, and I stick to that.

I know how people love quotes… So here’s one… Implicit data is the anti-matter of big data…

The trends, forecasts, etc will all be displaced (change) once we know implicit data’s place in the over all sum. We’ll throw our hands in the air and shout, well of course! How silly of us to make judgements with an incomplete sum… The early adopters are already homing in on this fact.

Big Data should be the word of the year

bigdata_network

I heard Geoff Nunberg’s piece on NPR’s podcast and I got to say, although I’m pretty much big dated out from BBC Backstage (in a nice way) I’m in total agreement. Here’s a few key points… Well worth listening to in audio form…

Whether it’s explicitly mentioned or not, the Big Data phenomenon has been all over the news. It’s responsible for a lot of our anxieties about intrusions on our privacy, whether from the government’s anti-terrorist data sweeps or the ads that track us as we wander around the Web. It has even turned statistics into a sexy major. So if you haven’t heard the phrase yet, there’s still time — it will be around a lot longer than “gangnam style.”

What’s new is the way data is generated and processed. It’s like dust in that regard, too. We kick up clouds of it wherever we go. Cellphones and cable boxes; Google and Amazon, Facebook and Twitter; cable boxes and the cameras at stoplights; the bar codes on milk cartons; and the RFID chip that whips you through the toll plaza — each of them captures a sliver of what we’re doing, and nowadays they’re all calling home.

It’s only when all those little chunks are aggregated that they turn into Big Data; then the software called analytics can scour it for patterns. Epidemiologists watch for blips in Google queries to localize flu outbreaks; economists use them to spot shifts in consumer confidence. Police analytics comb over crime data looking for hot zones; security agencies comb over travel and credit card records looking for possible terrorists.

It’s the amalgamation of all that personal data that makes it possible for businesses to target their customers online and tailor their sales pitches to individual consumers. You idly click on an ad for a pair of red sneakers one morning, and they’ll stalk you to the end of your days. It makes me nostalgic for the age when cyberspace promised a liberating anonymity. I think of that famous 1993 New Yorker cartoon by Peter Steiner: “On the Internet, nobody knows you’re a dog.” Now it’s more like, “On the Internet, everybody knows what brand of dog food you buy.”

Decentralised networking is hard, no really?

Sydney, January 2009

Straight out of the “No Sh*t Sherlock…” book….

Although I think its amazing what developers do, I can imagine how hard it must be to write decent decentralised software. The Diaspora guys spell out how difficult it is… which Adwale likes to make sure I and others fully understand.

  • If you build a decentralized application, you actually need to ship software. You need to package, test, create installers, test on a variety of platforms, write defensive code to work around misconfigurations your customers are likely to create, etc. For a centralized website, you can often edit files in place on the production server.
    Result: decentralized is 10x harder at least.
  • Somebody somewhere will run every single version of your app that you ever shipped. It will be badly out of date, full of security holes (you fixed years ago), outmoded graphics etc. It will cost you additional support, and your brand will suffer. Almost nobody upgrades to the latest and greatest within a life time it seems.
    Result: decentralized is less functional, less pretty, and less secure.
  • Decentralized software is much harder to monetize. You can’t run ads on somebody else’s installation. You can’t data mine your users (because most of them aren’t in a place that you have access to, it’s somebody else’s installation). You can’t do cross-promotions and referrals etc. You can charge those people who install your software, but there’s a reason most websites are free: much better business.
    Result: decentralized produces less money for you, so you have less investment dollars at your disposal.
  • Database migrations and the like for decentralized apps have to be fully productized, because they will be run by somebody else who does not know what to do when something fails 15 minutes into an ALTER TABLE command.
    Result: decentralized is 10x harder at least.
  • Same thing for performance optimizations and the like: it’s much easier to optimize your code for your own server farm than trying to help Joe remotely whose installation and servers you don’t have access to.
    Result: decentralized is slower, more expensive, and harder.

Frankly although I take the points… If you want to stand out in a clearly over crowded field, and one which has a major elephant using up all the space. You need to think differently (to quote someone we all know too well).

This means doing the difficult things which no one understands and owning the platform!

Your business model should/could be charging other developers to build and be creative on top of your platform. App.net have got the right idea, charge the developers who then create the experiences. Your focus should be on managing the platform and supporting their creativity. Anything else is greed and/or lack of focus.

What do I mean by creativity? Think about Tweetdeck…

Tweetdeck innovated on top of the Twitter platform and in the end the platform twitter bought them (stupid move). Tweetdeck for a lot of people made twitter usable at long last. The amount of news rooms I’ve been to and seen tweetdeck with a million panels open is untrue. The same isn’t true now… Tweetdeck guys innovated on top of Twitter and instead of sharing revenue with them or something. They bough them…!

A quote which comes to mind is something like…

The train company thought they were in the railroad business, what they didn’t get was that they were actually in the transportation business.

I really like twitter but frankly their control/greed/whatever is getting out of control. While on a panel yesterday at the London transmedia festival in Ravensbourne College. I was sat with Danielle from Tumblr, Bruce from Twitter, Cat from BBC and Doug Scott from Ogilvy. Although its tempting to make a few comments about there change in stance, I passed. Although I did notice say something which could be seen as slightly negative. Doug said how useful Twitter is for understanding users and I agreed but I said,

“Well its important to remember Twitter is only explicit data, implicit data is the stuff people really want to get there hands on…”

Anyway, the point stands and its hard to see how Twitter will get into the implicit data game at this point. If they acted like a platform, maybe someone else would do the innovation for them. But back to the main point why would you do it on someone closed system?

Decentralised network systems are harder but will drive much more interesting creativity… I can see how this might be at odds with setting up a business, startup and having investors etc… But I’m sure I could make a argument that its better in the long run…

The killer application for distributed social networking?

How do we make things move along quicker in the area of distributed/federated technology? Things are moving very slowly although it seems most of the components are in place.

When I wrote the blog about Rebel mouse, I found some interesting links to some distributed solutions which could see the end of the likes of twitter and facebook.

OStatus is an open standard for distributed status updates. The goal is to have a specification that allows different messaging hubs to route status updates between users in near-real-time. This spec took over from the OpenMicroBlogging spec of old.

I remember writing about wordpress’s distributed solution a while ago.

The weird thing is I logged into Diaspora again today and not only is it a ghost town (not like G+, but really like a ghost town) but it got me thinking whats different about Diaspora and G+? Now the hype died down, its time to see some very cool uses of Diaspora. What have they got to loose? Dare I say it, wheres the killer application? Wheres the thing which will make people sit up and take note once again? Heck whys no one doing cool stuff with the API?

So what is the killer application which will tip people over? I have some thoughts but what ever it is, please let it happen soon before we’re all forced to beg twitter, facebook, etc for our data back.

Love of the Self or Data sexuality?

Gary Wolf at Quantified Self 2011

When I read this article about the new breed of urban datasexuals I instantly thought maybe heck I might be one of them…?

The datasexual looks a lot like you and me, but what’s different is their preoccupation with personal data. They are relentlessly digital, they obsessively record everything about their personal lives, and they think that data is sexy. In fact, the bigger the data, the sexier it becomes. Their lives – from a data perspective, at least – are perfectly groomed.

Oh crap that sounds just like me… I find it very difficult to maintain things on paper and much prefer them in data because I can manage them much better. I assumed it was a dyslexia thing to be honest (it might still be).

The origin of the datasexual in all likelihood started with the humble infographic, which is a highly stylized and well-designed way to talk about all the data out there on Web. The infographic trend was followed by the data visualization trend, which made it even cooler to display data in innovative new ways. These data visualization tools eventually gave us cultural artifacts like Nicholas Felton’s annual Feltron Reports, which made the obsessive recording of everyday activities seem cool. From there, it was only a small evolutionary step to the whole quantified self (QS) movement, which promises “self knowledge through numbers.” QS proponents obsessively track every single bit of data about themselves throughout the day. The QS movement eventually led us to the embrace of data by consumer-facing companies Nike, who found a way for urban datasexuals to flaunt their obsessive data-grooming to the rest of us in a way that’s stylish and mod.

For me it stems back to my ideas of the information behind the graphics.

When I was in college, I got into XML because I loved the idea of creating graphics which are self describing and can alter depending on the data there based on.  Hence my love of Scalable Vector Graphics (w3c’s SVG specification). I was also a major pusher of SVG at the BBC for graphics and informational graphics but at the time browser technology was way behind.

Maybe this also reflects why my love of the idea of online dating via numbers, maths, science also really intrigues me so deeply?

The link up between the Quantified Self, personal data and sharing is so tight and also asks many questions. Questions which the early adopters and hackers are answering for themselves right now.

I remember my previous manager Miles Metcalfe, talking about the intangible of the semantic web… Somethings can not be quantified, at least in the ways were going about it right now. I would agree but we’ll have a good old go trying to do so. And from doing so we’ll have lots of fun, its when it stops becoming fun is the problem…

I’ll say it now… Data is Sexy no two ways about it… but the term data sexual does worry me a little along with quite a few of the commenters. Its Data Love but under the understanding that not everything can be defined or captured yet.

Updated

After the debate on Techgrumps 60 yesterday (listen to the last 10mins to understand) with Tom Morris who compares Data sexuality to something else which is certainly not pretty or nice. But the point is taken, what has this got to do with sexuality? Spicing it up with sexuality just confuses the whole thing and maybe makes those who love data into something their not. Data love is much better as a overall idea.

Is it possible to match people with science?

This has got to be one the eternal questions? Maths or science has solved so many of our questions but can it be used for working out compatibility of humans?

That was one of the things which really intrigued me about a year of making love. I assume you’ve seen how it turned in on its self since the production team totally screwed up the process and kept us all in the dark about it. And if you want further evidence do check out the tweets for #yearofmakinglove and #yoml…

However because of the total screwup most people are saying its a total failure (maybe very true) but also science or rather maths was never going to work… I can’t disagree specially after the experience we all had yesterday. However basing any judgments off the back of yesterdays experience would be a mistake.

So do I personally think maths/science can match humans? Maybe… (yes what a cope out) but to be honest no one knows for sure. And thats the point of the experiment.

At the very start of the day (ordeal) we were introduced to the professor who devised the test/questions and the matching algorithm. I remember tweeting this…

As Michael replied a far…

And he’s right…

In my own experience to date, the matching algorithm over at OkCupid.com has been pretty darn good (not perfect!) (OKCupid’s OK Trends are legendary – check out the biggest lies people tell each other on dating sites and How race effects the messages you get). But I had to train it to be good. I’ve to date answered about 700+ questions and there not just questions. There detailed, so you have to answer it, then specify how important this is to you and what answer your ideal match would pick. This makes for much more dimensions in the answer criteria and ultimately the algorithm. Aka the algorithm is only as good as the dataset its working on.

You got to put in the data/time, if you want it to be good… Otherwise your going to get crappy results.

This makes the 50 questions answered for the year of making love look like a pop quiz (hotshot), to be honest.

So back to the original question slightly modified, can a algorithm match people in the interest of love? I think so to a certain extent. But its not the complete picture. Chemistry is a big deal which is very difficult to understand. Its not found by answering questions but watching the interaction between people. Its a different type of algorithm… Situation can cause chemistry, aka the reason why everyone came together on the coaches home (or to the wrong city as some of them seemed to do) is because there was a social situation which we could all share/talk about. (cue talk about social objects/places) Chemistry was in full effect?

I hope people don’t give up on science as a way to find their ideal partner just because of the terrible experience they had at The year of making love… is I guess what I’m saying…

Public 2.0: The era of personal data openness

The era of personal data openness

I was in London Thursday for the Public 2.0 conference, which the guys behind the Data Art  project put together. It was a nicely put together conference with a mix of speakers and topics.
I kicked off the day with my presentation titled The era of personal data openness.
When I was approached about doing a presentation for the data art conference. I wasn’t sure which angle to take. After a few thoughts, I decided to contact the data art guys and see what they were exactly after. After a brief chat, I decided to take the more interesting path in this presentation
The premise of the presentation is Open data from organisations like the government, companies is interesting and the movement around this has finally sunk in. There wasn’t a single government proposed agenda last year which didn’t include something about releasing more open data. And every startup and online business is building APIs, so they can take advantage of the overwhelming power of the rich ecosystem of developers, hackers and early adopters. But I’ve noticed a increase in tools and systems to take advantage of our own data and the data we generate everyday.
I was tracking this very much from the sidelines and had not found a decent way of explaining the topic of self documentation. That was till I had lunch with Rain Ashford.
We talked through a bunch of stuff but got talking about my presentation which I was due to give next day. And after describing the premise like I am now. She said it sounds a lot like Quantified Self
Bingo! Having never heard of the movement, it instantly made sense and further research clarified everything.
Quantified Self is the Era of personal Data Openness….
Its also worth noting Walter De Brouwer’s presentation at Thinking Digital also had some influence but I forgot to mention it. Two links from that session http://curetogether.com & www.patientslikeme.com all fit perfectly…