Find the thread

AI and machine-learning (ML) are the hot topic of the day. As is usually the case when something is on the way up to the Peak of Inflated Expectations, wild proclamations abound of how this technology is going to either doom or save us all. Going on past experience, the results will probably be more mundane – it will be useful in some situations, less so in others, and may be harmful where actively misused or negligently implemented. However, it can be hard to see that stable future from inside the whirlwind.

Uhoh, This content has sprouted legs and trotted off.

In that vein, I was reading an interesting article which gets a lot right, but falls down by conflating two issues which, while related, should remain distinct.

there’s a core problem with this technology, whether it’s being used in social media or for the Mars rover: The programmers that built it don’t know why AI makes one decision over another.

The black-box nature of AI comes with the territory. The whole point is that, instead of having to write extensive sets of deterministic rules (IF this THEN that ELSE whatever) to cover every possible contingency, you feed data to the system and get results back. Instead of building rules, you train the system by telling it which results are good and which are not, until it starts being able to identify good results on its own.

This is great, as developing those rules is time-consuming and not exactly riveting, and maintaining them over time is even worse. There is a downside, though, in that rules are easy to debug. If you want to know why something happened, you can step through execution one instruction at a time, set breakpoints so that you can dig into what is going on at a precise moment in time, and generally have a good mechanical understanding of how the system works - or how it is failing.

I spend a fair amount of my time at work dealing with prospective customers of our own machine-learning solution. There are two common objections I hear, which fall at opposite ends of the same spectrum, but both illustrate just how different users find these new techniques.

Yes, there is an XKCD for every occasion

The first group of doubters ask to "see the machine learning". Whatever results are presented are dismissed as "just statistics". This is a common problem in AI research, where there is a general public perception of a lack of progress over the last fifty years. It is certainly true that some of the overly-optimistic predictions by the likes of Marvin Minsky have not worked out in practice, but there have been a number of successes over the years. The problem is that each time, the definition of AI has been updated to exclude the recent achievement.

Something of the calibre of Siri or Alexa would absolutely have been considered AI, but now their failure to understand exactly what is meant in every situation is considered to mean that they are not AI. Certainly Siri is not conscious in any way, just a smart collection of responses, but neither is it entirely deterministic in the way that something like Eliza is.¹

This leads us to the second class of objection: "how can I debug it?" People want to be able to pause execution and inspect the state of variables, or to have some sort of log that explains exactly the decision tree that led to a certain outcome. Unfortunately machine learning simply does not work that way. Its results are what they are, and the only way to influence them is to flag which are good and which are bad.

This is where the confusion I mentioned above comes in. When these techniques are applied in a purely technical domain - in my case, enterprise IT infrastructure - the results are fairly value-neutral. If a monitoring event gets mis-classified, the nature of Big Data (yay! even more buzzwords!) means that the overall issue it is a symptom of will probably still be caught, because enough other related events will be classified correctly. If however the object of mis-categorisation happens to be a human being, then even one failure could affect that person’s job prospects, romantic success, or even their criminal record.

The black-box nature of AI & ML is where very great care must be taken to ensure that ML is a safe and useful technique to use in each case, in legal matters especially. The code of law is about as deterministic as it is possible to be; edge cases tend to get worked out in litigation, but the code itself generally aims for clarity. It is also mostly easy to debug: the points of law behind a judicial decision are documented and available for review.

None of these constraints apply to ML. If a faulty facial-recognition algorithm places you at the heart of a riot, it’s going to be tough to explain to your spouse or boss why you are being hauled off in handcuffs. Even if your name is ultimately cleared, there may still be long-term damage done, to your reputation or perhaps to your front door.

It’s important to note that, despite the potential for draconian consequences, the law is actually in some ways a best case. If an algorithm kicks you off Google and all its ancillary services (or Facebook or LinkedIn or whatever your business relies on), good luck getting that decision reviewed, certainly in any sort of timely manner.

The main fear that we should have when it comes to AI is not "what if it works and tries to enslave us all", but "what if it doesn’t work but gets used anyway".

Photo by Ricardo Gomez Angel via Unsplash

What if all of those earnest post-Matrix philosophical discussions were more on point than we knew?

One of the central conceits of the Matrix films is that the machines simulate a late-twentieth-century environment for their human "batteries"…

Oh. Spoiler warning, I guess? Do we still need that for a film that came out in 1999? I’m calling it - anything from last century is now fair game.

As we were: all the humans live in a simulated late-90s world, complete with all sorts of weird and wonderful mobile phones, before we decided collectively that all phones should look like smooth rectangles of black glass.

This of course had nothing whatsoever to do with the fact that the late 90s were contemporaneous with when the films were being made, and therefore cheap to film, and everything to do with the late 90s apparently being recognised as the pinnacle of human civilisation.

Here’s the thing: what if the Wachowskis were right?

The twenty-first century is no longer the domain of a purely human civilisation. We are now a hybrid, cyborg civilisation, where baseline humans are augmented by artificial systems. I don’t think we are heading towards a Matrix-style takeover by the machines, but this is going to be a significant change, and one that is hard to fully comprehend from the inside, while it is happening. Also, once the change has happened, what came before will be fundamentally incomprehensible to anyone who comes of age in that future world.

The world they will inhabit will have bots and algorithms the way we baseline humans today have commensal bacteria in our guts. Our guts have enormous structures of neurons, second only to the brain itself:

Why is our gut the only organ in our body that needs its own "brain"? Is it just to manage the process of digestion? Or could it be that one job of our second brain is to listen in on the trillions of microbes residing in the gut?

Algorithms will begin to take part in this process too, as more and more of our cognition occurs outside our own biological minds. These off-board exo-selves will feel as much a part of us as our "gut feel" does today, but they will fundamentally change what it means - and how it feels - to be human.

We can see the beginnings of this process already: we drive where the algorithms tell us to drive, we exercise the way the algorithms tell us to exercise, and we even date whom the algorithms tell us to date. We buy films, music, and books that the algorithms recommend, go on holiday where they suggest, and take jobs that they set us up with. In the future, what other decisions will we hand over to algorithms - unquestioning and unconcerned?

The algorithms and bots may not be out to enslave us, but they do see things dramatically differently than we do. For an example, take a look at this map:

This is a snapshot of a map of the continental US doing the recent solar eclipse. The traffic algorithm has no idea of what an eclipse is, but it does know that something weird is happening: people are stopping their cars in the middle of roads across a wide strip of the US.

Famously, an algorithm figured out a teenage girl was pregnant before her dad did:

An angry man went into a Target outside of Minneapolis, demanding to talk to a manager:
"My daughter got this in the mail!" he said. "She’s still in high school, and you’re sending her coupons for baby clothes and cribs? Are you trying to encourage her to get pregnant?"
The manager didn’t have any idea what the man was talking about. He looked at the mailer. Sure enough, it was addressed to the man’s daughter and contained advertisements for maternity clothing, nursery furniture and pictures of smiling infants. The manager apologized and then called a few days later to apologize again.
On the phone, though, the father was somewhat abashed. "I had a talk with my daughter," he said. "It turns out there’s been some activities in my house I haven’t been completely aware of. She’s due in August. I owe you an apology."

And that’s not even the creepiest thing algorithms can do. They can identify your face, even when you hide it with a scarf to go to a protest - (unless of course they can’t ), and they can tell your sexual orientation from a photograph.

This is why DRM, privacy, and user control in general are such important topics: we are talking about our own future exoselves here. There are perfectly legitimate reasons not to want to broadcast your identity and all your particulars to all and sundry, especially in a world which is unfortunately still filled with prejudices against anyone who doesn’t fit in with the majority. And if something that is guiding your actions and your very thoughts belongs to a corporation that makes money from people who want to influence your actions and your thoughts, where does that leave you? About as enslaved as those human batteries in the Matrix, I’d say.

I’m a straight white middle-class dude, cis-het or whatever, and basically so square I’m practically cubic, so all of this is very far from affecting me personally. I’m at the very bottom of Niemöller’s poem - but I have friends and relatives who are much higher up, so I have both personal and selfish reasons for wanting to make sure this is done right. Personal, because don’t mess with my friends, and selfish, because as the Reverend Martin wrote, if we don’t fix it early, by the time it gets to causing problems for me, it will be way too late to do anything about it.

And of course there are all sorts of other aspects of this new future that we are building which all too few people are thinking about. Future historians will refer to these decades as "Digital Dark Ages": our history will be lost behind gratuitously incompatible file formats and DRM to which no living entity (human or corporate) has the keys any more. I was able to flip through my grandparents’ pictures and read a great-uncle’s book; as things stand, my grandchildren will not be able to have this experience.

The late twentieth century may indeed go down as the high-water mark of the purely human civilisation. The technologies that would make up the new world already existed - I played a full VR game, with goggles, 3D mouse, and a subwoofer in a backpack rig, in 1998 - but they were not yet fully joined up, and only vanishingly few people appreciated what would happen when they would all be connected up.

I have no intention of standing athwart history, yelling Stop - but we do need to think carefully about what kind of future we are building, and where it will take us. If the first couple of decades of this scary new century have taught us anything, it’s that the defences of "oh, that’ll never work" and "nobody would ever do that" are no defence at all, in cryptology, civil liberties, or anywhere else - if, indeed, they ever were.