Pluralistic: AI can't do your job (18 Mar 2025)


Today's links


The capital building, crossed by a yellow hazmat tape emblazoned 'DANGER ASBESTOS.' Behind the tape stands a figure in an old-fashioned hazmat suit. In the background are the blown-up faces of a group of grubby, miserable child miners. The capital dome has been replaced with the glaring red eye of HAL 9000 from Kubrick's '2001: A Space Odyssey.'

AI can't do your job (permalink)

AI can't do your job, but an AI salesman (Elon Musk) can convince your boss (the USA) to fire you and replace you (a federal worker) with a chatbot that can't do your job:

https://www.pcmag.com/news/amid-job-cuts-doge-accelerates-rollout-of-ai-tool-to-automate-government

If you pay attention to the hype, you'd think that all the action on "AI" (an incoherent grab-bag of only marginally related technologies) was in generating text and images. Man, is that ever wrong. The AI hype machine could put every commercial illustrator alive on the breadline and the savings wouldn't pay the kombucha budget for the million-dollar-a-year techies who oversaw Dall-E's training run. The commercial market for automated email summaries is likewise infinitesimal.

The fact that CEOs overestimate the size of this market is easy to understand, since "CEO" is the most laptop job of all laptop jobs. Having a chatbot summarize the boss's email is the 2025 equivalent of the 2000s gag about the boss whose secretary printed out the boss's email and put it in his in-tray so he could go over it with a red pen and then dictate his reply.

The smart AI money is long on "decision support," whereby a statistical inference engine suggests to a human being what decision they should make. There's bots that are supposed to diagnose tumors, bots that are supposed to make neutral bail and parole decisions, bots that are supposed to evaluate student essays, resumes and loan applications.

The narrative around these bots is that they are there to help humans. In this story, the hospital buys a radiology bot that offers a second opinion to the human radiologist. If they disagree, the human radiologist takes another look. In this tale, AI is a way for hospitals to make fewer mistakes by spending more money. An AI assisted radiologist is less productive (because they re-run some x-rays to resolve disagreements with the bot) but more accurate.

In automation theory jargon, this radiologist is a "centaur" – a human head grafted onto the tireless, ever-vigilant body of a robot

Of course, no one who invests in an AI company expects this to happen. Instead, they want reverse-centaurs: a human who acts as an assistant to a robot. The real pitch to hospital is, "Fire all but one of your radiologists and then put that poor bastard to work reviewing the judgments our robot makes at machine scale."

No one seriously thinks that the reverse-centaur radiologist will be able to maintain perfect vigilance over long shifts of supervising automated process that rarely go wrong, but when they do, the error must be caught:

https://pluralistic.net/2024/04/01/human-in-the-loop/#monkey-in-the-middle

The role of this "human in the loop" isn't to prevent errors. That human's is there to be blamed for errors:

https://pluralistic.net/2024/10/30/a-neck-in-a-noose/#is-also-a-human-in-the-loop

The human is there to be a "moral crumple zone":

https://estsjournal.org/index.php/ests/article/view/260

The human is there to be an "accountability sink":

https://profilebooks.com/work/the-unaccountability-machine/

But they're not there to be radiologists.

This is bad enough when we're talking about radiology, but it's even worse in government contexts, where the bots are deciding who gets Medicare, who gets food stamps, who gets VA benefits, who gets a visa, who gets indicted, who gets bail, and who gets parole.

That's because statistical inference is intrinsically conservative: an AI predicts the future by looking at its data about the past, and when that prediction is also an automated decision, fed to a Chaplainesque reverse-centaur trying to keep pace with a torrent of machine judgments, the prediction becomes a directive, and thus a self-fulfilling prophecy:

https://pluralistic.net/2023/03/09/autocomplete-worshippers/#the-real-ai-was-the-corporations-that-we-fought-along-the-way

AIs want the future to be like the past, and AIs make the future like the past. If the training data is full of human bias, then the predictions will also be full of human bias, and then the outcomes will be full of human bias, and when those outcomes are copraphagically fed back into the training data, you get new, highly concentrated human/machine bias:

https://pluralistic.net/2024/03/14/inhuman-centipede/#enshittibottification

By firing skilled human workers and replacing them with spicy autocomplete, Musk is assuming his final form as both the kind of boss who can be conned into replacing you with a defective chatbot and as the fast-talking sales rep who cons your boss. Musk is transforming key government functions into high-speed error-generating machines whose human minders are only the payroll to take the fall for the coming tsunami of robot fuckups.

This is the equivalent to filling the American government's walls with asbestos, turning agencies into hazmat zones that we can't touch without causing thousands to sicken and die:

https://pluralistic.net/2021/08/19/failure-cascades/#dirty-data

(Image: Cryteria, CC BY 3.0; Krd, CC BY-SA 3.0; modified)

Hey look at this (permalink)


A Wayback Machine banner.

Object permanence (permalink)

#20yrsago NYT publishes anonymous remarks slamming anonymity https://wendy.seltzer.org/blog/archives/2005/03/19/nyt_catches_the_anonymous_wifi_is_evil_bug.html

#15yrsago Peter Watts found guilty https://www.rifters.com/crawl/?p=1186

#15yrsago New ACTA leak: It’s a screwjob for the world’s poor countries https://web.archive.org/web/20100320075536/http://www.michaelgeist.ca/content/view/4886/125/

#10yrsago Brute-force iPhone password guesser can bypass Apple’s 10-guess lockout https://www.youtube.com/watch?v=meEyYFlSahk

#10yrsago Terry Pratchett’s advice to booksellers https://web.archive.org/web/20150319205013/https://www.thebookseller.com/blogs/advice-booksellers

#5yrsago Magic in the time of coronavirus https://pluralistic.net/2020/03/19/gb-whatsapp/#corona-conjuror

#5yrsago Fox News is a suicide cult https://pluralistic.net/2020/03/19/gb-whatsapp/#fox-cult

#5yrsago How to structure a fair covid bailout https://pluralistic.net/2020/03/19/gb-whatsapp/#peoples-bailout

#5yrsago Data is the New Toxic Waste https://pluralistic.net/2020/03/19/gb-whatsapp/#oily-rags-r-us

#5yrsago Africa's Facebook modders are world leaders https://pluralistic.net/2020/03/19/gb-whatsapp/#self-determination

#5yrsago Canada Reads documentary on Radicalized https://pluralistic.net/2020/03/19/gb-whatsapp/#canadareads

#5yrsago The worst Democrat in Congress just lost his job https://pluralistic.net/2020/03/19/gb-whatsapp/#lipinski-slain

Upcoming appearances (permalink)

A photo of me onstage, giving a speech, pounding the podium.


A screenshot of me at my desk, doing a livecast.

Recent appearances (permalink)


A grid of my books with Will Stahle covers..

Latest books (permalink)


A cardboard book box with the Macmillan logo.

Upcoming books (permalink)

  • Enshittification: Why Everything Suddenly Got Worse and What to Do About It, Farrar, Straus, Giroux, October 7 2025
    https://us.macmillan.com/books/9780374619329/enshittification/

  • Unauthorized Bread: a middle-grades graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, 2026

  • Enshittification, Why Everything Suddenly Got Worse and What to Do About It (the graphic novel), Firstsecond, 2026

  • The Memex Method, Farrar, Straus, Giroux, 2026


Colophon (permalink)

Today's top sources:

Currently writing:

  • Enshittification: a nonfiction book about platform decay for Farrar, Straus, Giroux. Status: second pass edit underway (readaloud)

  • A Little Brother short story about DIY insulin PLANNING

  • Picks and Shovels, a Martin Hench noir thriller about the heroic era of the PC. FORTHCOMING TOR BOOKS FEB 2025

Latest podcast: With Great Power Came No Responsibility: How Enshittification Conquered the 21st Century and How We Can Overthrow It https://craphound.com/news/2025/02/26/with-great-power-came-no-responsibility-how-enshittification-conquered-the-21st-century-and-how-we-can-overthrow-it/

This work – excluding any serialized fiction – is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.

How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/@pluralistic

Medium (no ads, paywalled):

https://doctorow.medium.com/

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

ISSN: 3066-764X

Join the Discussion at ChinWag