Pluralistic: LLMs are slot-machines (16 Aug 2025)


Today's links


LLMs are slot-machines (permalink)

When LLM users describe their experience with their chatbots, the results are so divergent that it can sound like they're describing two completely different products.

Previously, I've hypothesized that this is because there are two distinct groups of users: "centaurs" (people who are assisted by a machine – in this case, people who get to decide when, whether and how to integrate an LLM into their work) and "reverse-centaurs" (people conscripted into being an assistant to a machine – here, people whose bosses have fired their colleagues and ordered the survivors to oversee an LLM that badly approximates the work of those departed workers):

https://pluralistic.net/2025/08/04/bad-vibe-coding/#maximally-codelike-bugs

But yesterday, I read "The Futzing Fraction," an essay by Glyph, that advances a compatible, but very different hypothesis that I find extremely compelling:

https://blog.glyph.im/2025/08/futzing-fraction.html

Glyph proposes that many LLM-assisted programmers who speak highly of the reliability and value of AI tools are falling prey to two cognitive biases:

  1. The "availability heuristic" (striking things are easier to remember, which is why we remember the very rare instances of kids being kidnapped and killed, but rarely think about the relatively common phenomenon of kids dying in boring car-crashes); and

  2. The "salience heuristic" (big things are easier to remember, which is why we double-check that the oven is turned off and the smoke alarms are working after our neighbor's house burns down).

In the case of LLM coding assistants, this manifests as an unconscious overestimation of how often the LLM saves you time. That's because a coding program that produces a bug that you have to "futz with" for a while before it starts working is normal, and thus unmemorable, while a coding tool that turns a plain-language prompt into a working computer program is amazing, so it stands out in your memory.

Glyph likens this to a slot-machine: when you lose a dollar to a slot-machine, that is totally unremarkable, "the expected outcome." But when a slot pays out a jackpot, you remember that for the rest of your life. Walk through a casino floor on which a player hits a slot jackpot, and the ringing bells, flashing lights, and cheering crowd will stick with you, giving you an enduring perception that slot-machines are paying out all the time, even though no casino could stay in business if this were the case.

Glyph develops this analogy to describe why LLMs are worse than slot machines. He says that (non-pathological) gamblers set a budget for the amount of money they're prepared to lose to the slots, while a coder who's feeling warmly disposed to an LLM coding assistant may not put any explicit limits on how much time they'll spend futzing with LLM-generated code (I'll add here that part of the seductive joy of coding is that it can put its practitioners into a kind of autohyptnotic fugue state where they don't notice the passing of time, a state that is also a feature of pathological gambling).

Glyph poses a hypothetical: if you have a coding project that you ask a chatbot to write, and the resulting code initially doesn't work, but does work after ten minutes of futzing, that feels amazing and you will remember it forever as the time you saved 3:50 by using a chatbot. But it's possible that you repeated the "well, I'll just futz with this for ten minutes" step to get to that final success so many times that the whole affair took six hours, two hours longer than it would have taken had you just written the program from scratch. It's like winning a $1000 jackpot after "just putting a dollar in," except that that was the one-thousand-and-first dollar that you fed to the machine.

Glyph says that in other business activities, the "let's just try this for 10 minutes more" strategy usually pays off, but that LLMs produce an "an emotionally variable, intermittent reward schedule" that subverts your ability to wisely deploy that tactic.

But that's not the only way in which an LLM coding assistant is like a slot machine. Reg Braithwaite proposed that AI companies' business model is also like a casino's, because they charge every time you re-prompt the AI. He writes:

When you are paying by the "pull of the handle," the vendor's incentive is not to solve your problem with a single pull, but to give the appearance of progress towards solving your problem.

https://social.bau-ha.us/@raganwald/115033262770049100

Jpeck likens the use of an LLM coding assistant to "a dense intern" who has to be walked through each step and then have their work double-checked:

https://universeodon.com/@boscoandpeck/115033787721848290

But there's an important difference between an intern and an LLM. For a senior coder, helping an intern is an investment in nurturing a new generation of talented colleagues. For a reverse-centaur, refining an LLM is either an investment in fixing bugs in a product designed to put you on the breadline (if you believe AI companies' claims that their products will continue to improve until they don't need close supervision), or it's a wasted investment in a "dense intern" who is incapable of improving.

(Image: Frank Schwichtenberg, CC BY 4.0Cryteria, CC BY 3.0; modified)

Hey look at this (permalink)


A shelf of leatherbound history books with a gilt-stamped series title, 'The World's Famous Events.'

Object permanence (permalink)

#20yrago PC disguised as a set of encyclopedia volumes https://www.mini-itx.com/projects/encyclomedia/

#20yrago Nobel economist on harm lurking in copyright monopolies https://web.archive.org/web/20060217023906/http://www.dailytimes.com.pk/default.asp?page=story_16-8-2005_pg5_12

#15yrsago Heinlein biography: LEARNING CURVE – the secret history of science fiction https://memex.craphound.com/2010/08/16/heinlein-memoir-learning-curve-the-secret-history-of-science-fiction/

#5yrsago Amazon bans podcasts that criticize Amazon #https://pluralistic.net/2020/08/16/combat-wheelchairs/#nondisparagement

#5yrsago Combat Wheelchairs https://pluralistic.net/2020/08/16/combat-wheelchairs/#combat-wheelchairs

#5yrsago How to Argue With a Racist https://pluralistic.net/2020/08/16/combat-wheelchairs/#race-realism

#5yrsago Self-driving cars are bullshit https://pluralistic.net/2020/08/16/combat-wheelchairs/#car-wreck

#1yrago MIT libraries are thriving without Elsevier https://pluralistic.net/2024/08/16/the-public-sphere/#not-the-elsevier

Upcoming appearances (permalink)

A photo of me onstage, giving a speech, pounding the podium.


A screenshot of me at my desk, doing a livecast.

Recent appearances (permalink)


A grid of my books with Will Stahle covers..

Latest books (permalink)


A cardboard book box with the Macmillan logo.

Upcoming books (permalink)

  • Canny Valley: A limited edition collection of the collages I create for Pluralistic, self-published, September 2025

  • Enshittification: Why Everything Suddenly Got Worse and What to Do About It, Farrar, Straus, Giroux, October 7 2025
    https://us.macmillan.com/books/9780374619329/enshittification/

  • Unauthorized Bread: a middle-grades graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, 2026

  • Enshittification, Why Everything Suddenly Got Worse and What to Do About It (the graphic novel), Firstsecond, 2026

  • The Memex Method, Farrar, Straus, Giroux, 2026

  • The Reverse-Centaur's Guide to AI, a short book about being a better AI critic, Farrar, Straus and Giroux, 2026


Colophon (permalink)

Today's top sources:

Currently writing:

  • "The Reverse Centaur's Guide to AI," a short book for Farrar, Straus and Giroux about being an effective AI critic. (1022 words yesterday, 32968 words total).

  • A Little Brother short story about DIY insulin PLANNING

This work – excluding any serialized fiction – is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.

How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/@pluralistic

Medium (no ads, paywalled):

https://doctorow.medium.com/

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

READ CAREFULLY: By reading this, you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

ISSN: 3066-764X

Join the Discussion at ChinWag