Tristan Harris essay on attention hijacking and ever so dark patterns

Human attention is a scarce commodity

I heard about Tristan Harris through Time well spent which some people have been sharing a while ago. Kept meaning to read more about him and the essay he wrote. Its a excellent read and well worth reading. A few times while reading it, I wanted to annotate it some how. I know the w3C have finally sorted out the spec and I could do it via Diigo or even Wallabag if I wanted to; but sharing it seems to need more research on my part.

So instead I thought I’d half blog about it while copying the main points (once again you should read the whole thing yourself). Tristan has sectioned the points so I’ll copy that.

But I did want to say I find it interesting that Adrian Westaway from Special Projects and Tristan Harris are both magicians. The link between magic and design is a interesting one.

Hijack #1: If You Control the Menu, You Control the Choices

Western Culture is built around ideals of individual choice and freedom. Millions of us fiercely defend our right to make “free” choices, while we ignore how we’re manipulated upstream by limited menus we didn’t choose.

This is exactly what magicians do. They give people the illusion of free choice while architecting the menu so that they win, no matter what you choose. I can’t emphasize how deep this insight is.

When people are given a menu of choices, they rarely ask:

  • “what’s not on the menu?”
  • “why am I being given these options and not others?”
  • “do I know the menu provider’s goals?”
  • “is this menu empowering for my original need, or are the choices actually a distraction?” (e.g. an overwhelmingly array of toothpastes)

Absolutely, I do this a lot because I’m wondering how to break the system or hijack for my own needs. Usually when going to restaurants I need to hack it because I have so many allergies. If I didn’t hack it then I’d be pretty much dead.

I also find patterns quite interesting and can identify them quickly, so my tesco monthly shop will have every 2-3 months a deal on toilet rolls because I assume thats when they get the new stock in and need to shift some of the older ones. This funny example of understanding allows me to hack the system for my own needs.

I also tend to ignore all the recommendation stuff including the instant reply stuff I seen google has added to gmail. I also start to wonder more and more how this data is being mined to generate these results. Of course I got a big interest in big/linked data, data ethics and opinionated software.

Hijack #2: Put a Slot Machine In a Billion Pockets

One of the most tricky things I’ve seen many people try and deal with is not checking their phones and when they do, they do for what reason? To check out someone has liked something they have done. This comes straight out of the Sherry Turkle’s book Alone Together.

If you’re an app, how do you keep people hooked? Turn yourself into a slot machine.

But here’s the unfortunate truth — several billion people have a slot machine their pocket:

When we pull our phone out of our pocket, we’re playing a slot machineto see what notifications we got.

  • When we pull to refresh our email, we’re playing a slot machine to see what new email we got.
  • When we swipe down our finger to scroll the Instagram feed, we’replaying a slot machine to see what photo comes next.
  • When we swipe faces left/right on dating apps like Tinder, we’re playing a slot machine to see if we got a match.
  • When we tap the # of red notifications, we’re playing a slot machine to what’s underneath.

It takes some serious will to break away from the slot machines, especially when every once in a while it actually pays out (as such).

bThis is very much a dark pattern or dark art which drives a huge economy. Notifications like the breaking news banner on news sites tap right into the dopamine sender and the only way to break this is being more conscious. The truth is unsettling and we may not be able to easily change this without both sides being more aware/conscious of this all. Tristan points the finger at Google and Apple and yes they have responsibility but it can’t come from them alone.

Hijack #3: Fear of Missing Something Important (FOMSI)

Creating, inducing or manufacturing FOMO (fear of missing out) is pretty dark stuff.

Another way apps and websites hijack people’s minds is by inducing a “1% chance you could be missing something important.”

If I convince you that I’m a channel for important information, messages, friendships, or potential sexual opportunities — it will be hard for you to turn me off, unsubscribe, or remove your account — because (aha, I win) you might miss something important:

  • This keeps us subscribed to newsletters even after they haven’t delivered recent benefits (“what if I miss a future announcement?”)
  • This keeps us “friended” to people with whom we haven’t spoke in ages (“what if I miss something important from them?”)
  • This keeps us swiping faces on dating apps, even when we haven’t even met up with anyone in a while (“what if I miss that one hot match who likes me?”
  • This keeps us using social media (“what if I miss that important news story or fall behind what my friends are talking about?”)

I personally don’t subscribe to a lot of things because I’m wary of the effect of FOMO. I also don’t follow a lot people on Twitter because I don’t use twitter in that way much to the annoyance of some of my friends and followers. I do have a lot of friend connections on Facebook but also don’t read the timeline (its not a timeline, rather a curated feed for you based on algorithms and what FB thinks you want, remember point 1 about what the provider wants out of the deal?)

My friend Jon Rogers left twitter saying I was right about twitter (I can’t find any trace of him on twitter too). I wish I could find the conversation/blog (which seems to be down), but I partly blamed the fact he was using the official twitter client which would do things which were not to the benefit of him in anyway. Similarly Oli who left FB and then joined again after feeling FOMO.

Final example is why I left Bumble; I recognised the pattern of FOMSI being manufactured by Bumble and decided I wasn’t interested in being involved. Its a shame because I liked the concept but it was ruined for me by this forced FOSMI.

Hijack #4: Social Approval

We’re all vulnerable to social approval. The need to belong, to be approved or appreciated by our peers is among the highest human motivations. But now our social approval is in the hands of tech companies (like when we’re tagged in a photo).

Social approval is massive and drives us to do things which we wouldn’t normally do if we stopped and thought. I’d add this mixed with FOMO are a pretty lethal combination.

I wish I could filter out the likes on FB which clutter up my notifications, the little hit of dopamine just isn’t worth it. But then again I also like to click like to almost give my approval. Maybe I should stop doing this? This would also stop helping out the FB algorithm with positive reactions, now that can’t be a bad thing?

Of course social approval goes way beyond the likes and into the scoring stuff which I have talked about before.

Hijack #5: Social Reciprocity (Tit-for-tat)

Now this one really bugs me… I understand reciprocity theory and how it can be hijacked to con/cheat people out of something they wouldn’t normally give. Influence is a great book which I’d highly recommend to everyone.

We are vulnerableto needing to reciprocate others’ gestures. But as with Social Approval, tech companies now manipulate how often we experience it.

In some cases, it’s by accident. Email, texting and messaging apps are social reciprocity factories. But in other cases, companies exploit this vulnerability on purpose.

There was a period of time when the laws of social reciprocity seemed to dictate if you follow someone, you need to follow you back. This was rubbish of course, but pushed by twitters own system which encouraged you to follow back with one click. Twitter was a async follow but the service was changed to encourage something similar to a friend request later – most likely once the money became more important.

Of course Tristan is dead right about linkedin being a shocking example of this. I almost have to give them a award for their use of dark patterns to get you to do more within Linkedin.

orginal LinkedIn wants as many people creating social obligations for each other as possible, because each time they reciprocate (by accepting a connection, responding to a message, or endorsing someone back for a skill) they have to come back through linkedin.com where they can get people to spend more time.

Like Facebook, LinkedIn exploits an asymmetry in perception. When you receive an invitation from someone to connect, you imagine that person making a conscious choice to invite you, when in reality, they likely unconsciously responded to LinkedIn’s list of suggested contacts. In other words, LinkedIn turns your unconscious impulses (to “add” a person) into new social obligations that millions of people feel obligated to repay. All while they profit from the time people spend doing it.

Hijack #6: Bottomless bowls, Infinite Feeds, and Autoplay

Oh boy this winds me up big time, endless feeds. Its very similar to the all you can eat buffets. The quality of the things you are consuming are dubious at best and although you started out with something decent it suddenly drops in quality or go so far off the original purpose or reason.

Another way to hijack people is to keep them consuming things, even when they aren’t hungry anymore.

How? Easy. Take an experience that was bounded and finite, and turn it into a bottomless flowthat keeps going.

Cornell professor Brian Wansink demonstrated this in his study showing you can trick people into keep eating soup by giving them a bottomless bowl that automatically refills as they eat. With bottomless bowls, people eat 73% more calories than those with normal bowls and underestimate how many calories they ate by 140 calories.

Tech companies exploit the same principle. News feeds are purposely designed to auto-refill with reasons to keep you scrolling, and purposely eliminate any reason for you to pause, reconsider or leave.

This is partly why I prefer to read RSS than get the endless supply of stuff from Google, etc. At least there is a bottom and you can see a number of unread items. With these news feeds, its endless and the quality or value of the content is dependent on the agenda or services current goals (that can be as simple as this advertiser wants to pay us lots of money).

Endless also sucks you into the world that its only available now/its temporary and next time you look it will be gone or different. This is why I use services like wallabag, pocket or even youtube watch it later. If its worth saving its worth spending some time on and not being rushed to the next thing. Yes its hard and there is a social pressure to have watched or read it quickly (skimmed) to keep up with the conversation. In fact coming back to something in twitter usually causes confusion if you come back to a post a few days later. This is why I tend to just blog it to give it context and the effort once I read it fully.

Endless scroll is becoming a bit of thing now too, similar to the swipe forever stuff. Don’t get me started about auto play video, which I have seen cause much problems with presentations in conferences; as you can imagine

Hijack #7: Instant Interruption vs. “Respectful” Delivery

Companies know that messages that interrupt people immediately are more persuasive at getting people to respond than messages delivered asynchronously (like email or any deferred inbox).

Given the choice, Facebook Messenger (or WhatsApp, WeChat or SnapChat for that matter) would prefer to design their messaging system tointerrupt recipients immediately (and show a chat box) instead of helping users respect each other’s attention.

In other words, interruption is good for business.

It’s also in their interest to heighten the feeling of urgency and social reciprocity. For example, Facebook automatically tells the sender when you “saw” their message, instead of letting you avoid disclosing whether you read it(“now that you know I’ve seen the message, I feel even more obligated to respond.”) By contrast, Apple more respectfully lets users toggle “Read Receipts” on or off.

I do generally avoid a lot of these instant messaging systems but even those I use have included this way (Gtalk, Wire and even Signal). If I can turn it off I do but I have observed how Facebook now throws up notification as a window above other stuff like a instant message. Lets not forget those horrible chat heads too.

Respectful delivery is getting rare and even when they are, you need to work at it. I feel quite lucky that I’m running Ubuntu as my host operating system which gives me complete control over the notifications but this doesn’t help when looking at a browser tab like Facebook, which wants to dominate (trust me this is the right word) the view. This is also another reason why I don’t have Facebook on my phones/tablets and why I limit messengers permissions.

Hijack #8: Bundling Your Reasons with Their Reasons

In the physical world of grocery stories, the #1 and #2 most popular reasons to visit are pharmacy refills and buying milk. But grocery stores want to maximize how much people buy, so they put the pharmacy and the milk at the back of the store.

In other words, they make the thing customers want (milk, pharmacy) inseparable from what the business wants. If stores were truly organized to support people, they would put the most popular items in the front.

This is bloody annoying and one of the reasons why a lot of apps dont really care or advertise direct links into parts of there systems. This is why I have to keep FB in a tab otherwise everytime I login, I would need to go via the news feed each time, a total waste of my time.

The whole point of the web is not having to go on a journey each time. Remember when you saw VR shopping malls and thought wtf? Well thats pretty much the same coming back to haunt us all, for whose benefit? Certainly not yours!

Hijack #9: Inconvenient Choices

This is a recurring dark pattern, the roach motel.

We’re told that it’s enough for businesses to “make choices available.”

“If you don’t like it you can always use a different product.”
“If you don’t like it, you can always unsubscribe.”
“If you’re addicted to our app, you can always uninstall it from your phone.”

Businesses naturally want to make the choices they want you to make easier, and the choices they don’t want you to make harder. Magicians do the same thing. You make it easier for a spectator to pick the thing you want them to pick, and harder to pick the thing you don’t.

For example, NYTimes.com let’s you “make a free choice” to cancel your digital subscription. But instead of just doing it when you hit “Cancel Subscription,” they force you to call a phone number that’s only open at certain times.

Hijack #10: Forecasting Errors, “Foot in the Door” strategies

People don’t intuitively forecast the true cost of a click when it’s presented to them. Sales people use “foot in the door” techniques by asking for a small innocuous request to begin with (“just one click”), and escalating from there (“why don’t you stay awhile?”). Virtually all engagement websites use this trick. Imagine if web browsers and smartphones, the gateways through which people make these choices, were truly watching out for people and helped them forecast the consequences of clicks (based on real data about what it actually costs most people?). That’s why I add “Estimated reading time” to the top of my posts. When you put the “true cost” of a choice in front of people, you’re treating your users or audience with dignity and respect.
This is tied to so many of the things said previously. One of the useful things I found is the putting things into wallabag and pocket is I can manager my own time; and not be forced into making a poor decision under time pressure
The Hurrah – A sudden crisis or change of events forces the victim to act immediately.
 
Its clear most humans do not make good decisions under pressure and scammers, con-artists, the systems we use know this too well.

There is so much more to discuss including the how to fix this all… but thats for another blog post…

Sexortation the new catfish scam?

Black mirror s3 ep3: Shut up and Dance screenshot
After a virus infects his laptop, a teen faces a daunting choice: carry out orders delivered by text message, or risk having intimate secrets exposed.

Tom Morris wrote on his blog

For some, Black Mirror seems to be a HOWTO guide. I guess if you are going to use sexual shame to make money, Britain is the best place to target.

He’s not wrong, from the Telegraph paper link.

Organised criminal gangs are blackmailing growing numbers of young men after using social media to entice them into performing sex acts on screen.

Police have revealed an unprecedented rise in the new crime of webcam blackmail – known as ‘sextortion’ – with more than 900 cases reported so far this year.

That is already more than double the total for the whole of 2015.

But senior officers at the National Crime Agency fear the true scale of the problem is far bigger, with many victims too ashamed to report their involvement to police.

Among recent victims were four young men who became so desperate at the thought of being publicly humiliated that they took their own lives.

I mentioned this in my talk at TedXManchester 4. At the time people seemed a little baffled and you can see how its can be confused with Cat-fishing. This partly why I felt it deserved a entry in urban dictionary (even if I did spell it slightly wrong).

Watching Black Mirror Season 3 episode 3: Shut up and dance I was instantly thinking about this even if [Promise not to spoil it… but there is so much I could say] but there is a really nice breakdown at the psychtech podcast too.

If people think blocking your webcam will solve the problem, think again! This has just got started, I dare not think how low this scam will go.

Sexoration is now in the Urban Dictionary

test 2

I defined sexoration in the urban dictionary a while ago.

Sexoration

A type of dating scam which involves exchanging pictures and videos with a target. Then blackmailing them later in return for money or some other type of currency.
Works directly with Catfish, as the shared pictures are usually ripped from elsewhere. Also similar to Ransomware in impact. She contacted out the blue, it felt like it was for sexoration

I used it in my TEDxTalk: Dating against humanity

Is this a made up thing?

Although the video above isn’t strictly sexortation, you can see how blackmail crossed with catfishing can lead to a dangerious place. Its a very real thing, you only have look at the Skype support site. Its super destructive and one bad mistake can cause the endless worry and pain. I simply gave it a name which made sense from what I heard and seenUrban dictionary agreed.

Those bloody Facebook like scams

Virtual Bagel Facebook scam

Where ever there is people being social there’s room for social engineering? Welcome to the Facebook Like scams… A little old but still a pain!

As you may have noticed I’m not exactly a fan of facebook but seeing a few of those crappy “do this now” posts in my timeline irritates me even more. As you can imagine these are just social engineering scams. One of the most common is ‘Press Like if you hate cancer, ignore if you don’t’? but theres many more… Heck even the BBC did a scam to prove it can be done.

The ‘my sister Mallory’ scam
This post stated that someones ‘sister’ Mallory has down syndrome & doesn’t think she’s beautiful. It then asked for ‘likes’ it to show her she is. The REAL story about this little girl is something much different: Read about it here

As usual you can tell by the call to action and social pressure being layered.

So how does it work?

  1. A page is created.
  2. This page put out a constant stream of heart wrenching and/or mildly amusing images that are shared publicly with a call to action to click, share or comment.
  3. These posts are initially shared by a big group of people all in the same network who have all built up their edge rank over a period of time that then results in the posts eventually leaking into the newsfeeds of real-life accounts.
  4. These people share, like or comment which then spreads.
  5. Eventually a friend of yours hits that little thumbs up button
  6. It’s in your newsfeed.

And within 3 days a post like this one has 70,000 likes, and someone somewhere is about to make a nice little profit by selling the page to a business wanting some quick wins.

The buyer then changes some of the page details. BAM! Instant fanpage with a big following, lots of likes and an in depth edge rank, capable of pushing out content to a pre-built list of thousands of Facebook users. (edit: Page name cannot be altered after 200 likes, just details of the page).

And there you have it… Facebook scammed

Thanks to my friends who stupidly fell for the scams on multiple times. I’ll be sending them this blog post in future.

I am calling you from Windows… Oh really?

Ars Technica has a great piece about the scammy calls I sometimes get when I pick up my house phone (currently broke).

When the call came yesterday morning, I assumed at first I was being trolled—it was just too perfect to be true. My phone showed only “Private Caller” and, when I answered out of curiosity, I was connected to “John,” a young man with a clear Indian accent who said he was calling from “Windows Technical Support.” My computer, he told me, had alerted him that it was infested with viruses. He wanted to show me the problem—then charge me to fix it.

Thankfully somethings happening to stop this scam

This scam itself is a few years old now, but I had not personally received one of the calls until yesterday—the very day that the Federal Trade Commission (FTC) announced a major crackdown on such “boiler room” call center operations. The very day that six civil lawsuits were filed against the top practitioners

Its a shame Ars Technica didn’t record it, but of course I did, twice…!


The ammyy scam: the worst social engineering I’ve ever heard

Email Scam

For some stupid reason which I have no idea… I got 3 calls from a call centre while I was at home trying to work today.

It got to be a bit of joke by the second call because with the first call I got so peed off about what they were trying to tell me I just hung up after 30secs. When someone called again, claiming to be calling from Microsoft customer support, this time I playing along with this call just to waste there time and work out what they wanted me to do so i could warn other people not to follow the steps.

Caller: open Internet Explorer and type in ammyy.com.

Caller: click to download and install ammyy

Me: I can’t do that (lies of course)

Caller: Why not? click the link and choose install.

Anyway that went on and on for about 20mins, and so of course I hit Twitter with some funny bits I was hearing on the phone. By the time I finished… I was doing stuff like using the toilet and saying I was still in front of the windows XP machine (I would have thought the sound of me peeing would be a clear clue that I wasn’t really listening)

By the time it finished, Nic Ferrier suggested I should record them next time they call. So I did, but I didn’t catch the start of the conversation, so I started recording about 10-15mins in. Here’s the recording with a con-artist.

Recording-1 with a con-artist by cubicgarden

It is a scam (so popular its actually called the ammyy scam) as you can guess but weirdly it does actually catch people out… [1][2][3]

Hopefully the recording will help raise the profile of this scam and stop other people falling for this frankly terriable social engineering scam.

Wanna buy a laptop mate, yours for 300 pounds

So I was walking for the train today because my scooter has a flat tyre. Walking down briskly, listening to a podcast as usual when these guys roll up in a slightly beaten up old style Vauxhall Caliver. The passenger rolls down his window and says, You wanna buy a laptop mate? At first I couldn't fully hear him but once he opened the laptop , a Sony Viao. It all quickly clicked into place.

So the first thing which goes through my mind is not to get too close to the car just in case they happen to have some weapon. But I do stop and lift up my sunglasses. The guy with a very strong Irish accent, almost sounded put on. Says just got it from PC World brand new with recepts, yours for 300 pounds. I look at the driver who seems more concerned about driving off quickly that the deal which was happening. So anyway, being the haggler that I am, I said sounds expensive for a knocked off Laptop, hows 100 pounds? And you know what he seriously considers it. By which I'm already reaching for my sunglasses and walking. These guys follow me down the road, showing off all the features of a standard XP install. Till I say 100 pounds or nothing. They disappear shouting its too low for them and they will easily find someone else in this area (woolwich).

To be fair I shouldn't have said anything, but I wanted to see how desperate they were to get rid of the laptop. It was a nice Sony Viao with motion eye camera and looked to be about a 2.4ghz Pentium 4. But I was never going to buy it. I was also thinking about all the angles of there deal. For example I carry a bag which looks like a laptop bag, so I was wondering if they were really after that. I mean it would make sense why they would ask me rather than any of the other people walking down the crowded street. And there were lots of people walking fast to catch the train to London.

Just thought I'd share this, as I have got older my brush with the blackmarket (why do they call it that?) has become less and less which is a good thing. I've not had this kind of a deal since I was living in Thornton Heath (south central London) funny stuff, eh?

Comments [Comments]
Trackbacks [0]