Authorās Note: This short story was originally commissioned by Deakin College as part of an AI ethics course; they have since take it down. This is its new home. For a nonfiction analysis of the problems set forth herein, see my Guardian column on the subject. Hereās an audio edition.
inĀ·foĀ·sec (/ĖinfÅĖsek/): information security
coĀ·mo (/koh-moh/): content moderation
Content moderation is really, really hard.
Content moderators:
seek to set conversational norms, steering transgressors toward resources that help them better understand the local conversational rules;
respond to complaints from users about uncivil, illegal, deceptive or threatening posts;
flag or delete material that crosses some boundary (for example, deleting posts that dox other users, or flagging posts with content warning for adult material or other topics);
elevate or feature material that is exemplary of community values;
adjudicate disputes about impersonation, harassment and other misconduct.
Hereās a delicious story: CNN reports that Russian looters, collaborating with the Russian military, stole 27 pieces of John Deere farm equipment from a dealership in Melitopol, Ukraine, collectively valued at $5,000,000. The equipment was shipped to Chechnya, but it will avail the thieves naught, because the John Deere dealership reached out over the internet and bricked these tractors, using an in-built kill-switch.
Since that story ran last week, Iāve lost track of the number of people who sent it to me. I can see why: itās a perfect cyberpunk nugget: stolen tractors rendered inert by an over-the-air update, thwarting the bad guys. It could be the climax of a prescient novella in Asimovās circa 1996.
But Iām here to tell you: this is not a feel-good story.
I mean, sure. In the short term, itās really cool to think of those looters arriving in Chechnya only to discover that their looted tractors and combines and such are only good for spare parts (and maybe not even that).
But if you scratch the surface of that cinematic comeuppance, what you find is a far scarier parable about the way that cyberwarfare could extrude itself into the physical world. After all, if John Deereās authorized technicians can reach out and brick any tractor or combine, anywhere in the world, then anyone who suborns, hacks or blackmails a John Deere technicianāāāsay, Russiaās storied hacker army, who specialize in mass-scale infrastructure attacks, which they perfected by attacking Ukrainian embedded systemsāāācan do the exact same thing.