Showing all posts tagged big-data:

Sowing Bitter Seeds

The Internet is outraged by… well, a whole lot of things, as usual, but in particular by Apple. For once, however, the issue is not phones that are both unexciting and unavailable, lacking innovation and wilfully discarding convention, and also both over- and under-priced. No, this time the issue is apps, and in particular VPN apps.

Authoritarian regimes around the world (Russia, "Saudi" Arabia, China, North Korea, etc) have long sought to control their populations' access to information in general, and to the Internet in particular. Of course anyone with a modicum of technical savvy - or a friend, relative, or passing acquaintance willing to do the simple setup - can keep unfettered access to the Internet by going through a Virtual Private Network, or VPN.

A VPN does what it says on the tin: it creates a virtual network that connects directly with an endpoint somewhere else; importantly, somewhere outside the authoritarian regime's control. As such, VPNs have always existed in something of a grey area, but now China (the People's Republic, not that other China) has gone ahead and formally banned their use.

In turn, Apple have responded by removing unregistered VPN apps (which in practical terms means all of them) from their App Store in China. In the face of the Internet's predictable outrage, Apple provided this bald statement (via TechChrunch):

Earlier this year China’s MIIT announced that all developers offering VPNs must obtain a license from the government. We have been required to remove some VPN apps in China that do not meet the new regulations. These apps remain available in all other markets where they do business.

Now Apple do have a point; the law is indeed the law, and because they operate in China, they need to enforce it, just as they would with laws in any other country.

Here's the rub, though. By the regionalised way they have set up their App Store service, they have made themselves unnecessarily vulnerable to this sort of arm-twisting by unfriendly governments. Last time I wrote about geo-fencing and its consequences, the cause of the day was Russia demanding removal of the LinkedIn app, and China (them again!) demanding removal of the New York Times app. As I wrote at the time, companies like Apple originally set up the infrastructure for these geographic restrictions to enable IP protection, but the same tools are being repurposed for censorship:

This sort of restriction used to be "just" hostile to consumers. Now, it is turning into a weapon that authoritarian regimes can wield against Apple, Google, and whoever else. Nobody would allow Russia to ban LinkedIn around the world, or China to remove the New York Times app everywhere - but because dedicated App Stores exist for .ru and .cn, they are able to demand these bans as local exceptions, and even defend them as respecting local laws and sensibilities. If there were one worldwide App Store, this gambit would not work.

The argument against the infrastructure of laws and regulations that was put in place to enable (ineffective) IP restrictions was always that it could be, and would be, repurposed to enable repression by authoritarian regimes. People scoffed at these privacy concerns, saying "if you have nothing to hide, you have nothing to fear". But what if your government is the next to decide that reading the NYT or having a LinkedIn profile is against the law? How scared should you be then?

If you are designing a social network or other system with the expectation of widespread adoption, these days this has to be part of your threat model. Otherwise, one day the government may come knocking, demanding your user database for any reason or no reason at all - and what seemed like a good idea at the time will end up messing up a lot of people's lives.

Product designers by and large do not think of such things, as we saw when Amazon decided that it would be perfectly reasonable to give everyone in your address book access to your Alexa device - and make it so users could not turn off this feature without a telephone call to Amazon support.

How well do you think that would go down if you were a dissident, or just in the social circle of one?

Our instinctive attitude to data is to hoard them, but this instinct is obsolete, forged in a time when data were hard to gather, store, and access. It took something on the scale of the Stasi to build and maintain profiles on even six million citizens (out of a population of sixteen million), and the effort and expense was part of what broke the East German regime in the end. These days, it's trivial to build and access such a profile for pretty much anyone, so we need to change our thinking about data - how we gather them, and how we treat them once we have them.

Personal data are more akin to toxic waste, generated as a byproduct of valuable activity and needing to be stored with extreme care because of the dire consequences of any leaks. Luckily, data are different from toxic waste in one key respect: they can be deleted, or better, never gathered in the first place. The same goes for many other choices, such as restricting users to one particular geographical App Store, or making it easy to share your entire contact list (including by mistake), but very difficult to take that decision back.

What other design decisions are being made today based on obsolete assumptions that will come back to bite users in the future?

UPDATE: And there we go, now Russia is following China’s example and banning VPNs as well. The idea of a technical fix to social and legal problems is always a short-term illusion.

Image by Sean DuBois via Unsplash

Wishing for a Wish List

Why does Apple hate wish lists so much?

The wish list is the main thing I miss since I fell out with Amazon and moved all of my media buying over to iTunes. Amazon not only has great management of its wish list, allowing you to sort it any way you like and highlighting deals, or sharing it with friends and family as suggestions; it also uses the contents of your wish list as inputs to its recommendation engine.

Over the decade or so that I used Amazon regularly, its recommendations grew to be uncannily accurate, alerting me to new books or albums that I might be interested in. The algorithm involved was clever enough to recommend not only new works by artists I had already bought from in the past, but also works by other artists I had not previously encountered. This was driven by their ability to identify that "other people who bought X also bought Y", based on their insight into all of our purchasing histories.

Of course this is a critical feature for Amazon, which explains why they spend so much time and effort on refining it. In fact, it was only when they messed with my wish list that I left in a huff.

I had continued to buy from Amazon’s UK site after leaving the UK, because with free shipping within the EU, it made no difference, while it allowed me to keep that all-important wish list history. A few years later, however, Amazon in their wisdom decided that many items would no longer be made available to ship outside the UK. Instead of simply tagging the items with a notice, they simply removed the items from users’ stored wish lists. In my case, this meant I lost nearly half of my wish list items.

I use wish lists as a way to spread out purchases or remind me of items that are due to come out in the future but that I am not committed enough to pre-order right away (or which may not yet be available to pre-order). Deleting half of my wish list in this high-handed way was enough for me to quit a triple-figure-per-month Amazon habit cold-turkey.

This coincided with the move to a new house, where even our existing media collections were overflowing the shelves once we had finished unpacking. The time was therefore ripe for a move to electronic content only, and given that I was cross with Amazon, Apple was the only real alternative.

It’s been a couple of years now, and I have not regretted it in any way. I adapted very quickly to reading on the iPad, and music and the occasional film are of course super-easy. There is only one glaring problem, and that is the utterly inconsistent handling of wish lists on the part of the Apple store apps.

iBooks app on iPhone - note lack of wish list button

The fact that it’s plural "apps" is a bit of a problem in its own right, actually. I have a Music app to listen to music, that I buy in the iTunes Store app. That is where I also buy videos, that I then watch in the Videos app. But if I want to buy books, I have to do that in a special tab of the iBooks app.

Historically this makes sense - iBooks came along much later than the rest of iTunes. But why the weird inconsistencies in when I can add something to my iTunes/iBooks wish lists? iBooks on iOS won’t allow this, but iBooks on the Mac will. On the other hand, iTunes on the Mac won’t let me add an album to my wish list, but the iTunes Store app on iOS will.

Same screen in iTunes Store app on iPhone - note "Add to Wish List" button

This is why I have a file in Notes with iTunes Store links to items that I wanted to add to my wish list, but couldn’t because I didn’t have access to the specific device that would let me do that at the time.


This is admittedly a pretty minor niggle in the grand scheme of things, but I think it’s philosophically important for Apple to fix this inconsistency. It lies right at the heart of the iTunes ecosystem, and creates an unexpected and annoying discrepancy between MacOS and iOS platforms, and even between different devices on iOS.

Big Brother, Big Data

Together with everybody else who has any interest in how we live today and how we can expect to live for the next few decades, I have been reading Douglas Coupland’s recent piece in FT Weekend magazine.

The topic is what he calls "Artificial Intuition" - basically the convergence of Big Data1 and all sorts of loyalty and activity tracking, in which algorithms will be able to correlate our data exhaust from all sorts of different sources, aggregate and correlate it, and use it to document and even predict our behaviour with a high degree of accuracy.

Many people find this scary or otherwise undesirable. Evgeny Morozov is something of a cheerleader for this rejectionist camp, calling the rise of data "the death of politics". The overall point of this reaction is that political change has always required a grey area where activities that might be illegal are not enforced. This is how homosexuality or racial equality could move from illegal, to tolerated, to embraced and legalised: because places and spaces existed where it was possible to practice illegal behaviours on a limited scale and in a group tolerant of those behaviours.

Coupland’s take is that these Temporary Autonomous Zones are being cleared up by the algorithmic approach to everything from shopping, to dating, to politics:

In fact, [Artificial Intuition] is accelerating at an astonishing clip, and it’s the true and definite and undeniable human future.

Later in the same piece:

The amount of internet freedom we have right now is the most we’re ever going to get.

This is probably correct - although I might remove the word "internet". Absent some sort of major crash of online surveillance mechanisms, it seems that we are heading to the point where this data-driven approach becomes irreversible. We are probably past the point where even full-blown societal rejection would work.


As ever, there’s a relevant Gibson quote:

Mona's life has left virtually no trace on the fabric of things, and represents, in Legba's system, the nearest thing to innocence.

From Mona Lisa Overdrive. Published in 1988, people!

My previous company had a deeply unsettling (to me) employee healthcare programme in the USA, where employees who did not participate in company-mandated health&fitness routines - and share data from those! - were penalised in their healthcare costs.

The black humour of the situation is that when Americans want to criticise European-style nationalised healthcare systems, they usually trot out arguments about smokers or overweight people being denied medical treatments because of their lifestyle choices. But here was a private company, under contract to another private company, enacting and enforcing something far more intrusive.

The usual argument here is that you should be staying healthy for yourself anyway, so this tracking should not matter to you. In fact, you should enjoy the discounts or loyalty points or whatever you get for meeting the targets of the health&fitness programme.2

Basically it’s a variation of the "if you have nothing to hide, you have nothing to fear" argument. The problem is that with Big Data, eventually everyone will have something to hide. Had an extra glass of wine with dinner? Drifted a little over the speed limit that one time? Inadvertently short-changed someone? Guess what - that’s tracked now. And what other behaviours, routine today, might be criminalised tomorrow? Overconsumption of sugar? Excessive screen time allowed to children?

This is not an idle question. Data don’t have a statute of limitations. Regardless of what you think of them personally, users of the hacked Ashley Madison cheaters’ site often paid specifically to delete their accounts. One reason someone might do this is that they had seen the error of their ways and resolved to be a better spouse - but now they risk being outed together with the biggest adulterers out there.

For once I’m going to quote Julian Assange, someone with whom I have all sorts of issues, but who has become something of a poster child for net privacy:

My version of that is to say, 'well, you're so boring then we shouldn't be talking to you, and neither should anyone else', but philosophically, the real answer is this: Mass surveillance is a mass structural change. When society goes bad, its going to take you with it, even if you are the blandest person on earth.

There’s a reason we have doors to our houses, and blinds and curtains to our windows - and it’s not so we can commit crimes in comfort, it’s simply so we can live our lives in private. Remember, the Panopticon was a prison. When secrets are outlawed, only outlaws will have secrets.

Image by Edgaras Maselskis via Unsplash

  1. shudder 

  2. The cynic in me suspects that those discounts and other benefits will last exactly as long as needed to get people to sign up, and then be withdrawn - so sorry! - in the next cost-cutting or restructuring exercise.