Showing all posts tagged amazon:

Which Algorithms Will Watch The Algorithms?

This week’s AI-powered scandal is the news that Amazon scrapped a "secret" AI recruiting tool that showed bias against women.

Amazon.com Inc’s machine-learning specialists uncovered a big problem: their new recruiting engine did not like women.

The AI, of course, has no opinion about one women one way or the other. Amazon HR's recruitment tool was not “biased" in the sense that a human recruiter might be; it was simply unearthing existing bias:

That is because Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period. Most came from men, a reflection of male dominance across the tech industry.

If you train your neural networks with biased data, you are just re-encoding and reinforcing the bias! This was not news even in 2015 when Amazon started this experiment. However, it illustrates a general problem with AI, namely its users’ naive tendency to take the algorithms’ output at face value. As with any automated system, human experts will be needed to make intelligent evaluations of the output of the system.

Train The Guardians

The need for secondary evaluation is only going to increase as these systems proliferate. For instance, Canada now plans to use AI to decide immigration cases. If your request for a visa is rejected by an expert system, what recourse do you have? What if the basis of rejection is simply that claims like yours were usually denied in the past?

These concerns will become more and more critical as AI tools continue to become more mainstream.

"The computer says no" has always been the jobsworth’s go-to excuse. But who programmed the computer to say "no"? The fact that the computers are now programming themselves to say "no" does not absolve organisations of responsibility. The neural networks are simply reflecting the inputs they are given. It is on us to give them good inputs.

After all, we are educating our own children.

Privacy Versus AI

There is a widespread assumption in tech circles that privacy and (useful) AI are mutually exclusive. Apple is assumed to be behind Amazon and Google in this race because of its choice to do most data processing locally on the phone, instead of uploading users’ private data in bulk to the cloud.

A recent example of this attitude comes courtesy of The Register:

Predicting an eventual upturn in the sagging smartphone market, [Gartner] research director Ranjit Atwal told The Reg that while artificial intelligence has proven key to making phones more useful by removing friction from transactions, AI required more permissive use of data to deliver. An example he cited was Uber "knowing" from your calendar that you needed a lift from the airport.

I really, really resent this assumption that connecting these services requires each and every one of them to have access to everything about me. I might not want information about my upcoming flight shared with Uber – where it can be accessed improperly, leading to someone knowing I am away from home and planning a burglary at my house. Instead, I want my phone to know that I have an upcoming flight, and offer to call me an Uber to the airport. At that point, of course I am sharing information with Uber, but I am also getting value out of it. Otherwise, the only one getting value is Uber. They get to see how many people in a particular geographical area received a suggestion to take an Uber and declined it, so they can then target those people with special offers or other marketing to persuade them to use Uber next time they have to get to the airport.

I might be happy sharing a monthly aggregate of my trips with the government – so many by car, so many on foot, or by bicycle, public transport, or ride sharing service – which they could use for better planning. I would absolutely not be okay with sharing details of every trip in real time, or giving every busybody the right to query my location in real time.

The fact that so much of the debate is taken up with unproductive discussions is what is preventing progress here. I have written about this concept of granular privacy controls before:

The government sets up an IDDB which has all of everyone's information in it; so far, so icky. But here's the thing: set it up so that individuals can grant access to specific data in that DB - such as the address. Instead of telling various credit card companies, utilities, magazine companies, Amazon, and everyone else my new address, I just update it in the IDDB, and bam, those companies' tokens automatically update too - assuming I don't revoke access in the mean time.

This could also be useful for all sorts of other things, like marital status, insurance, healthcare, and so on. Segregated, granular access to the information is the name of the game. Instead of letting government agencies and private companies read all the data, users each get access only to those data they need to do their jobs.

Unfortunately, we are stuck in an stale all-or-nothing discussion: either you surround yourself with always-on internet-connected microphones and cameras, or you might as well retreat to a shack in the woods. There is a middle ground, and I wish more people (besides Apple) recognised that.


Photo by Kyle Glenn on Unsplash

War of the World Views

There has been this interesting shift going on in coverage of Silicon Valley companies, with increasing scepticism informing what had previously been reliable hero-worshipping. Case in point: this fascinating polemic by John Battelle about the oft-ignored human externalities of “disruption" (scare quotes definitely intended).

Battelle starts from a critique of Amazon Go, the new cashier-less stores Amazon is trialling. I think it’s safe to say that he’s not a fan:

My first take on Amazon Go is this: F*cking A, do we really want eggplants and cuts of meat reduced to parameterized choices spit onto algorithmized shelves? Ick. I like the human confidence I get when a butcher considers a particular rib eye, then explains the best way to cook that one cut of meat. Sure, technology could probably deliver me a defensibly "better" steak, perhaps even one tailored to my preferences as expressed through reams of data collected through means I’ll probably never understand.
But come on.
Sometimes you just want to look a guy in the eye and sense, at that moment, that THIS rib eye is perfect for ME, because I trust that butcher across the counter. We don’t need meat informed by data and butchered by bloodless algorithms. We want our steak with a side of humanity. We lose that, we lose our own narrative.

Battelle then goes on to extrapolate that "ick" out to a critique of the whole Silicon Valley model:

It’s this question that dogs me as I think about how Facebook comports itself : We know what’s best for you, better than you do in fact, so trust us, we’ll roll the code, you consume what we put in front of you.
But… all interactions of humanity should not be seen as a decision tree waiting to be modeled, as data sets that can be scanned for patterns to inform algorithms.

Cut Down The Decision Tree For Firewood

I do think there is some merit to this critique. Charlie Stross has previously characterised corporations as immortal hive organisms which pursue the three corporate objectives of growth, profitability, and pain avoidance:

We are now living in a global state that has been structured for the benefit of non-human entities with non-human goals. They have enormous media reach, which they use to distract attention from threats to their own survival. They also have an enormous ability to support litigation against public participation, except in the very limited circumstances where such action is forbidden. Individual atomized humans are thus either co-opted by these entities (you can live very nicely as a CEO or a politician, as long as you don't bite the feeding hand) or steamrollered if they try to resist.
In short, we are living in the aftermath of an alien invasion.

These alien beings do not quite understand our human reactions and relations, and they try pin them down and quantify them in their models. Searching for understanding through modelling is value-neutral in general, but problems start to appear when the model is taken as authoritative, with any real-life deviation from the model considered as an error to be rectified – by correcting the real-life discrepancy.

Fred Turner describes the echo chamber these corporations inhabit, and the circular reasoning it leads to, in this interview:

About ten years back, I spent a lot of time inside Google. What I saw there was an interesting loop. It started with, "Don't be evil." So then the question became, "Okay, what's good?" Well, information is good. Information empowers people. So providing information is good. Okay, great. Who provides information? Oh, right: Google provides information. So you end up in this loop where what's good for people is what's good for Google, and vice versa. And that is a challenging space to live in.

We all live in Google’s space, and it can indeed be challenging, especially if you disagree with Google about how information should be gathered and disseminated. We are all grist for its mighty Algorithm.

This presumption of infallibility on the part of the Algorithm, and of the world view that it implements is dangerous, as I have written before. Machines simply do not see the world as we do. Building our entire financial and governance systems around them risks some very unwelcome consequences.

But What About The Supermarket?

Back to Battelle for a moment, zooming back in on Amazon and its supermarket efforts:

But as they pursue the crack cocaine of capitalism — unmitigated growth — are technology platforms pushing into markets where perhaps they simply don’t belong? When a tech startup called Bodega launched with a business plan nearly identical to Amazon’s, it was laughed off the pages of TechCrunch. Why do we accept the same idea from Amazon? Because Amazon can actually pull it off?

The simple answer is that Bodega falls into the uncanny valley of AI assistance, trying to mimic a human interaction instead of embracing its new medium. A smart vending machine that learns what to stock? That has value - for the sorts of products that people like to buy from vending machines.

This is Amazon’s home turf, where the Everything Store got its start, shipping the ultimate undifferentiated good. A book is a book is a book; it doesn’t really get any less fresh, at least not once it has undergone its metamorphosis from newborn hardback to long-lived paperback.

In this context, nappies/diapers1 or bottled water are a perfect fit, and something that Amazon Prime has already been selling for a long time, albeit at a larger remove. Witness those ridiculous Dash buttons, those single-purpose IoT devices that you can place around your home so that when you see you’re low on laundry powder or toilet paper you can press the button and the product will appear miraculously on your next Amazon order.

Steaks or fresh vegetables are a different story entirely. I have yet to see the combination of sensors and algorithms that can figure out that a) these avocados are close to over-ripe, but b) that’s okay because I need them for guacamole tonight, or c) these bananas are too green to eat any time soon, and d) that’s exactly what I need because they’re for the kids’ after-school snack all next week.

People Curate, Algorithms Deliver

Why get rid of the produce guy in the first place?

Why indeed? But why make me deal with a guy for my bottled water?2

I already do cashier-less shopping; I use a hand-held scanner, scan products as I go, and swipe my credit card (or these days, my phone) on my way out. The interaction with the cashier was not the valuable one. The valuable interaction was with the people behind the various counters - fish, meat, deli - who really were, and still are, giving me personalised service. If I want even more personalised service, I go to the actual greengrocer, where the staff all know me and my kids, and will actively recommend produce for us and our tastes.

All of that personalisation would be overkill, though, if all I needed were to stock up on kitchen rolls, bottled milk, and breakfast cereal. These are routine, undifferentiated transactions, and the more human effort we can remove from those, the better. Interactions with humans are costly activities, in time (that I spend dealing with a person instead of just taking a product off the shelf) and in money (someone has to pay that person’s salary, healthcare, taxes, and so on). They should be reserved for situations where there is a proportionate payoff: the assurance that my avos will be ripe, my cut of beef will be right for the dish I am making, and my kids’ bananas will not have gone off by the time they are ready to eat them.

We are cyborgs, every day a little bit more: humans augmented by machine intelligence, with new abilities that we are only just learning to deal with. The idea of a cashier-less supermarket does not worry me that much. In fact, I suspect that if anything, by taking the friction out of shopping for undifferentiated goods, we will actually create more demand for, and appreciation of, the sort of "curated" (sorry) experience that only human experts can provide.


Photos by Julian Hanslmaier and Anurag Arora on Unsplash


  1. Delete as appropriate, depending on which side of the Atlantic you learned your English. 

  2. I like my water carbonated, so sue me. I recycle the plastic bottles, if that helps. Sometimes I even refill them from the municipal carbonated-water taps. No, I’m not even kidding; those are a thing around here (link in Italian). 

Sowing Bitter Seeds

The Internet is outraged by… well, a whole lot of things, as usual, but in particular by Apple. For once, however, the issue is not phones that are both unexciting and unavailable, lacking innovation and wilfully discarding convention, and also both over- and under-priced. No, this time the issue is apps, and in particular VPN apps.

Authoritarian regimes around the world (Russia, "Saudi" Arabia, China, North Korea, etc) have long sought to control their populations' access to information in general, and to the Internet in particular. Of course anyone with a modicum of technical savvy - or a friend, relative, or passing acquaintance willing to do the simple setup - can keep unfettered access to the Internet by going through a Virtual Private Network, or VPN.

A VPN does what it says on the tin: it creates a virtual network that connects directly with an endpoint somewhere else; importantly, somewhere outside the authoritarian regime's control. As such, VPNs have always existed in something of a grey area, but now China (the People's Republic, not that other China) has gone ahead and formally banned their use.

In turn, Apple have responded by removing unregistered VPN apps (which in practical terms means all of them) from their App Store in China. In the face of the Internet's predictable outrage, Apple provided this bald statement (via TechChrunch):

Earlier this year China’s MIIT announced that all developers offering VPNs must obtain a license from the government. We have been required to remove some VPN apps in China that do not meet the new regulations. These apps remain available in all other markets where they do business.

Now Apple do have a point; the law is indeed the law, and because they operate in China, they need to enforce it, just as they would with laws in any other country.

Here's the rub, though. By the regionalised way they have set up their App Store service, they have made themselves unnecessarily vulnerable to this sort of arm-twisting by unfriendly governments. Last time I wrote about geo-fencing and its consequences, the cause of the day was Russia demanding removal of the LinkedIn app, and China (them again!) demanding removal of the New York Times app. As I wrote at the time, companies like Apple originally set up the infrastructure for these geographic restrictions to enable IP protection, but the same tools are being repurposed for censorship:

This sort of restriction used to be “just" hostile to consumers. Now, it is turning into a weapon that authoritarian regimes can wield against Apple, Google, and whoever else. Nobody would allow Russia to ban LinkedIn around the world, or China to remove the New York Times app everywhere - but because dedicated App Stores exist for .ru and .cn, they are able to demand these bans as local exceptions, and even defend them as respecting local laws and sensibilities. If there were one worldwide App Store, this gambit would not work.

The argument against the infrastructure of laws and regulations that was put in place to enable (ineffective) IP restrictions was always that it could be, and would be, repurposed to enable repression by authoritarian regimes. People scoffed at these privacy concerns, saying "if you have nothing to hide, you have nothing to fear". But what if your government is the next to decide that reading the NYT or having a LinkedIn profile is against the law? How scared should you be then?

If you are designing a social network or other system with the expectation of widespread adoption, these days this has to be part of your threat model. Otherwise, one day the government may come knocking, demanding your user database for any reason or no reason at all - and what seemed like a good idea at the time will end up messing up a lot of people's lives.

Product designers by and large do not think of such things, as we saw when Amazon decided that it would be perfectly reasonable to give everyone in your address book access to your Alexa device - and make it so users could not turn off this feature without a telephone call to Amazon support.

How well do you think that would go down if you were a dissident, or just in the social circle of one?

Our instinctive attitude to data is to hoard them, but this instinct is obsolete, forged in a time when data were hard to gather, store, and access. It took something on the scale of the Stasi to build and maintain profiles on even six million citizens (out of a population of sixteen million), and the effort and expense was part of what broke the East German regime in the end. These days, it's trivial to build and access such a profile for pretty much anyone, so we need to change our thinking about data - how we gather them, and how we treat them once we have them.

Personal data are more akin to toxic waste, generated as a byproduct of valuable activity and needing to be stored with extreme care because of the dire consequences of any leaks. Luckily, data are different from toxic waste in one key respect: they can be deleted, or better, never gathered in the first place. The same goes for many other choices, such as restricting users to one particular geographical App Store, or making it easy to share your entire contact list (including by mistake), but very difficult to take that decision back.

What other design decisions are being made today based on obsolete assumptions that will come back to bite users in the future?


UPDATE: And there we go, now Russia is following China’s example and banning VPNs as well. The idea of a technical fix to social and legal problems is always a short-term illusion.


Image by Sean DuBois via Unsplash

The Middle Cannot Hold

In light of Amazon's latest moves - first the acquisition of Whole Foods, and now a cooperation with Sears - the stock market is trying to work out what will be the next area of retail to be hit.

Here's a sample of the latest Finimize newsletter:

Investors should be asking, who’s next in Amazon’s line of fire?
One of the biggest threats is likely to come from Amazon making more of its own-label products. For example, why would people pay a premium for Colgate toothpaste or Dove soap if Amazon basics are cheaper and arrive at our front door on the same day? Procter & Gamble, Unilever and other consumer goods giants could experience serious upheaval to their business models. Similarly, apparel companies are threatened: Amazon’s own-label clothing sales grew 25% last year, far outstripping the industry’s 3% growth. Do you care if your downward dog is performed in Amazon yoga pants rather than Lululemon? Perhaps not.

This analysis is not wrong, but is incomplete. Amazon will indeed take over a large chunk of the bottom end of the market, which is almost undifferentiated. The top end will be largely unaffected, as it is driven by completely different mechanisms.

The vulnerable actors are the ones in the middle of the market, who offered neither exceptionally high quality, nor particularly low prices. Historically, these brands succeeded by controlling distribution, especially outside major population centres.

Once an alternative becomes available that offers lower prices for equivalent quality (or the perception thereof), those middle-of-the-road brands and distribution outlets get squeezed hard.

I have noticed this over the last few years on the ski slopes, where a number of relatively undifferentiated brands have disappeared under a rising tide of previously unkown logos. All of these - Wed'ze, Quechua, and the like - are the house brands of Decathlon, the French sports superstore.

This is the Amazon Basics formula: good quality, and very reasonable prices. What Decathlon adds to the mix is that the designs are usually attractive, if somewhat non-descript, and that the clothes are available to try on in their vast network of retail outlets.

That last factor is key: to conserve differentiation and therefore ability to compete against a determined behemoth like Amazon, offer something they don't. Even buying something as simple as a T-shirt online is a fraught experience, with the differences between European and US sizing. I can't imagine shopping online for anything more complicated, like a shirt or a suit. On the other hand, I buy stuff from Decathlon every season because I can check it out in store, try it on, feel the material.

The same goes for Zara, part of the giant Inditex group, which benefits from a joined-up online/offline strategy. Customers can try on clothes in store, but if the precise colour and size combination is not available, they can order it online through Zara's own retail site.

Another factor that might cause shoppers to hesitate before making an online purchase is returns. The cost of shipping can outweigh the cost of the item, and in any case requires shoppers to deal with packaging and labelling, before taking the item to an out-of-town shipper. Zara on the other hand allows online shoppers to manage exchanges or returns in-store, thereby further monetising their investment in real estate even for online-first shoppers.

Of course not every operator has to take this route - which is good, because by definition there is not much room at this end of the market, where margins are razor-thin.

Sticking with winter sportswear, for anything I care deeply about, I shop from named brands that I have built up trust with - the likes of Level or Hestra for gloves, Under Armor for mid-layers, Burton or Arcteryx for outerwear, and so on. These are fairly big-ticket items, certainly in proportion to the base layers I pick up by the three-pack at Decathlon, so I shop around and get all picky about them.

Part of what I buy into is of course the experience. Going into a North Face store, you feel like you are participating in strenuous and exciting activities, even if all you are doing is bumbling about in-bounds trying to get your kids to graduate from their preferred "Snowplow Everywhere!!!" technique. Neither Amazon nor Decathlon can offer this, but then again, they have staked out the price-sensitive end of the market.

There is not much room in the middle any more, especially when you consider that consumers are quite happy to cross back and forth, wearing base layers from no-name brands under much more expensive outer layers. On the other hand, it's not a foregone conclusion that Amazon will own everything below the top of the market. Price-focused outlets still have a role to play, if they capitalise on their strengths.

As with most aspects of the digital revolution, it's the middlemen that are in trouble. The trick is to have a unique value proposition and stick to it. Operators whose only proposition was ubiquity and convenience cannot match the actual ubiquity and convenience of web-scale operators.

Coming full circle, groceries will experience a similar transition over time. Amazon is good at undifferentiated goods - which is why they started out with books: a book is the same regardless of when and where it is bought. I fully expect them to take over a big chunk of the market for packaged goods. On the other hand, there will always be a need to pick out fruit and groceries in person, to feel whether this avocado is ripe to make guacamole tonight or whether that bunch of bananas is green enough to last out the week. By combining online scale with Whole Foods' local presence, Amazon is going for the Zara play: check out a few products locally, buy many products online, and have both parts of the experience supported by the same seamless massive back-end.

Anyone wanting to compete with Amazon should choose their terrain very carefully indeed. After all, your margin is their opportunity.

Wishing for a Wish List

Why does Apple hate wish lists so much?

The wish list is the main thing I miss since I fell out with Amazon and moved all of my media buying over to iTunes. Amazon not only has great management of its wish list, allowing you to sort it any way you like and highlighting deals, or sharing it with friends and family as suggestions; it also uses the contents of your wish list as inputs to its recommendation engine.

Over the decade or so that I used Amazon regularly, its recommendations grew to be uncannily accurate, alerting me to new books or albums that I might be interested in. The algorithm involved was clever enough to recommend not only new works by artists I had already bought from in the past, but also works by other artists I had not previously encountered. This was driven by their ability to identify that "other people who bought X also bought Y", based on their insight into all of our purchasing histories.

Of course this is a critical feature for Amazon, which explains why they spend so much time and effort on refining it. In fact, it was only when they messed with my wish list that I left in a huff.

I had continued to buy from Amazon’s UK site after leaving the UK, because with free shipping within the EU, it made no difference, while it allowed me to keep that all-important wish list history. A few years later, however, Amazon in their wisdom decided that many items would no longer be made available to ship outside the UK. Instead of simply tagging the items with a notice, they simply removed the items from users’ stored wish lists. In my case, this meant I lost nearly half of my wish list items.

I use wish lists as a way to spread out purchases or remind me of items that are due to come out in the future but that I am not committed enough to pre-order right away (or which may not yet be available to pre-order). Deleting half of my wish list in this high-handed way was enough for me to quit a triple-figure-per-month Amazon habit cold-turkey.

This coincided with the move to a new house, where even our existing media collections were overflowing the shelves once we had finished unpacking. The time was therefore ripe for a move to electronic content only, and given that I was cross with Amazon, Apple was the only real alternative.

It’s been a couple of years now, and I have not regretted it in any way. I adapted very quickly to reading on the iPad, and music and the occasional film are of course super-easy. There is only one glaring problem, and that is the utterly inconsistent handling of wish lists on the part of the Apple store apps.

iBooks app on iPhone - note lack of wish list button

The fact that it’s plural “apps" is a bit of a problem in its own right, actually. I have a Music app to listen to music, that I buy in the iTunes Store app. That is where I also buy videos, that I then watch in the Videos app. But if I want to buy books, I have to do that in a special tab of the iBooks app.

Historically this makes sense - iBooks came along much later than the rest of iTunes. But why the weird inconsistencies in when I can add something to my iTunes/iBooks wish lists? iBooks on iOS won’t allow this, but iBooks on the Mac will. On the other hand, iTunes on the Mac won’t let me add an album to my wish list, but the iTunes Store app on iOS will.

Same screen in iTunes Store app on iPhone - note "Add to Wish List" button

This is why I have a file in Notes with iTunes Store links to items that I wanted to add to my wish list, but couldn’t because I didn’t have access to the specific device that would let me do that at the time.

Workaround

This is admittedly a pretty minor niggle in the grand scheme of things, but I think it’s philosophically important for Apple to fix this inconsistency. It lies right at the heart of the iTunes ecosystem, and creates an unexpected and annoying discrepancy between MacOS and iOS platforms, and even between different devices on iOS.