Pluralistic: When prophecy fails, election polling edition (26 Sep 2024)


Today's links


Giovanni Stradano's 1587 illustration of Canto 20 of Dante's *Inferno*, depicting the fortunetellers in the 4th Bolgia (pit) their heads rotated 180' on their necks, forced ever to walk in circles, looking backwards.

When prophecy fails, election polling edition (permalink)

In Canto 20 of Inferno, Dante confronts a pit where the sinners have had their heads twisted around backwards; they trudge, naked and weeping, through puddles of cooling tears. Virgil informs him that these are the fortunetellers, who tried to look forwards in life and now must look backwards forever.

In a completely unrelated subject, how about those election pollsters, huh?

Writing for The American Prospect, historian Rick Perlstein takes a hard look at characteristic failure modes of election polling and ponders their meaning:

https://prospect.org/politics/2024-09-25-polling-imperilment/

Apart from the pre-election polling chaos we're living through today, Perlstein's main inspiration is W Joseph Campbell 2024 University of California Press book, Lost in a Gallup: Polling Failure in US Presidential Elections:

https://www.ucpress.edu/books/lost-in-a-gallup/paper

In Campbell's telling, US election polling follows a century-old pattern: pollsters discover a new technique that works spookily well..for a while. While the new polling technique works, the pollster is hailed a supernaturally insightful fortune-teller.

In 1932, the Raleigh News and Observer was so impressed with polling by The Literary Digest that they proposed replacing elections with Digest's poll. The Digest's innovation was sending out 20,000,000 postcards advertising subscriptions and asking about presidential preferences. This worked perfectly for three elections – 1924, 1928, and 1932. But in 1936, the Digest blew it, calling the election for Alf Landon over FDR.

The Digest was dethroned, and new soothsayers were appointed: George Gallup, Elmo Roper and Archibald Crossler, who replaced the Digest's high-volume polling with a new kind of poll, one that sought out a representative slice of the population (as Perlstein says, this seems "so obvious in retrospect, you wonder how nobody thought of it before").

Representative polling worked so well that, three elections later, the pollsters declared that they could predict the election so well from early on that there was no reason to keep polling voters. They'd just declare the winner after the early polls were in and take the rest of the election off.

That was in 1948 – you know, 1948, the "Dewey Defeats Truman" election?

https://en.wikipedia.org/wiki/Dewey_Defeats_Truman

If this sounds familiar, perhaps you – like Perlstein – are reminded of the 2016 election, where Fivethirtyeight and Nate Silver called the election for Hillary Clinton, and we took them at their word because they'd developed a new, incredibly accurate polling technique that had aced the previous two elections.

Silver's innovation? Aggregating state polls, weighting them by accuracy, and then producing a kind of meta-poll that combined their conclusions.

When Silver's prophecy failed in 2016, he offered the same excuse that Gallup gave in 1948: when voters are truly undecided, you can't predict how they'll vote, because they don't know how they'll vote.

Which, you know, okay, sure, that's right. But if you know that the election can't be called, if you know that undecided voters are feeding noise into the system whenever you poll them, then why report the polls at all? If all the polling fluctuation is undecided voters flopping around, not making up their mind, then the fact that candidate X is up 5 points with undecided means nothing.

As the finance industry disclaimer has it, "past performance is no guarantee of future results." But, as Perlstein says, "past performance is all a pollster has to go on." When Nate Silver weights his model in favor of a given poll, it's based on that poll's historical accuracy, not its future accuracy, because its future accuracy can't be determined until it's in the past. Like Dante's fortune-tellers, pollsters have to look backwards even as they march forwards.

Of course, it doesn't help that in some cases, Silver was just bad at assessing polls for accuracy, like when he put polls from the far-right "shock pollster" Trafalgar Group into the highly reliable bucket. Since 2016, Trafalgar has specialized in releasing garbage polls that announce that MAGA weirdos are way ahead, and because they always say that, they were far more accurate than the Clinton-predicting competition in 2016 when they proclaimed that Trump had it in the bag. For Silver, this warranted an "A-" on reliability, and that is partially to blame for how bad Silver's 2020 predictions were, when Republicans got pasted, but Trafalgar continued to predict a Democratic wipeout. Silver's methodology has a huge flaw: because Trafalgar's prediction history began in 2016, that single data-point made them look pretty darned reliable, even though their method was to just keep saying the same thing, over and over:

https://www.ettingermentum.news/p/the-art-of-losing-a-fivethirtyeight

Pollsters who get lucky with a temporarily reliable methodology inevitably get cocky and start cutting corners. After all, polling is expensive, so discontinuing the polls once you think you have an answer is a way to increase the enterprise's profitability. But, of course, pollsters can only make money so long as they're somewhat reliable, which leads to a whole subindustry of excuse-making when this cost-cutting bites them in the ass. In 1948, George Gallup blamed his failures on the audience, who failed to grasp the "difference between forecasting an election and picking the winner of a horse race." In 2016, Silver declared that he'd been right because he'd given Trump a 28.6% chance of winning.

This isn't an entirely worthless excuse. If you predict that Clinton's victory is 71.4% in the bag, you are saying that Trump might win. But pollsters want to eat their cake and have it, too: when they're right, they trumpet their predictive accuracy, without any of the caveats they are so insistent upon when they blow it:

https://www.youtube.com/watch?v=1jDlo7YfUxc

There's always some excuse when it comes to the polls: in 1952, George Gallup called the election a tossup, but it went for Eisenhower in a landslide. He took out a full-page NYT ad, trumpeting that he was right, actually, because he wasn't accounting for undecided voters.

Polling is ultimately a form of empiricism-washing. The pollster may be counting up poll responses, but that doesn't make the prediction any less qualitative. Sure, the pollster counts responses, but who they ask, and what they do with those responses, is purely subjective. They're making guesses (or wishes) about which people are likely to vote, and what it means when someone tells you they're undecided. This is at least as much an ideological project as it is a scientific one:

https://prospect.org/blogs-and-newsletters/tap/2024-09-23-polling-whiplash/

But for all that polling is ideological, it's a very thin ideology. When it comes to serious political deliberation, questions like "who is likely to vote" and "what does 'undecided' mean" are a lot less important than, "what are the candidates promising to do?" and "what are the candidates likely to do?"

But – as Perlstein writes – the only kind of election journalism that is consistently, adequately funded is poll coverage. As a 1949 critic put it, this isn't the "pulse of democracy," it's "its baby talk."

Hey look at this (permalink)


A Wayback Machine banner.

This day in history (permalink)

#15yrsago D&D camp, circa 1982 https://web.archive.org/web/20091001142940/http://rpg.brouhaha.us/?p=925

#10yrsago Why #gamergate is bullshit https://www.cracked.com/blog/7-ways-gamergate-debate-has-made-world-worse

#10yrsago Eric Holder: creator of the “Too Big to Jail” bankster https://memex.craphound.com/2014/09/26/eric-holder-creator-of-the-too-big-to-jail-bankster/

#5yrsago The DoJ’s corporate “diversion” program is supposed to change bad corporate culture, but really, it enables repeat offenders https://www.citizen.org/article/soft-on-corporate-crime-deferred-and-non-prosecution-repeat-offender-report/

#5yrsago After the passage of the EU Copyright Directive, Google nukes Google News France https://www.bloomberg.com/news/articles/2019-09-25/google-to-change-french-news-results-under-copyright-rules

#5yrsago Sleuths discover the source of $28m in dark money lobbying in favor of emergency room “surprise bills”: private equity firms that own doctors’ practices https://hcrenewal.blogspot.com/2019/09/who-advocates-for-surprise-medical.html

#5yrsago Wework, Uber, Lyft, Netflix, Bird, Amazon: late-stage capitalism is all about money-losing predatory pricing aimed at creating monopolies https://web.archive.org/web/20190927033958/https://www.businessinsider.com/wework-is-a-prime-example-of-counterfeit-capitalism-2019-9

#5yrsago Bruce Sterling on Boris Johnson’s bizarre, cyberpunk dystopia address to the UN https://www.wired.com/beyond-the-beyond/2019/09/visionary-high-points-recent-boris-johnson-speech-united-nations/

#5yrsago Christopher Brown talking legal thrillers, dystopia, and science fiction https://narrativespecies.wordpress.com/2019/09/25/christopher-brown-rule-of-capture/

#1yrago Brian Merchant's "Blood In the Machine" https://pluralistic.net/2023/09/26/enochs-hammer/#thats-fronkonsteen

Upcoming appearances (permalink)

A photo of me onstage, giving a speech, holding a mic.


A screenshot of me at my desk, doing a livecast.

Recent appearances (permalink)


A grid of my books with Will Stahle covers..

Latest books (permalink)


A cardboard book box with the Macmillan logo.

Upcoming books (permalink)

  • Picks and Shovels: a sequel to "Red Team Blues," about the heroic era of the PC, Tor Books, February 2025

  • Unauthorized Bread: a middle-grades graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, 2025


Colophon (permalink)

Today's top sources:

Currently writing:

  • Enshittification: a nonfiction book about platform decay for Farrar, Straus, Giroux. September 26ths's progress: 761 words (54547 words total).

  • A Little Brother short story about DIY insulin PLANNING

  • Picks and Shovels, a Martin Hench noir thriller about the heroic era of the PC. FORTHCOMING TOR BOOKS JAN 2025

Latest podcast: Anti-cheat, gamers, and the Crowdstrike disaster https://craphound.com/news/2024/09/15/anti-cheat-gamers-and-the-crowdstrike-disaster/

This work – excluding any serialized fiction – is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.

How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/@pluralistic

Medium (no ads, paywalled):

https://doctorow.medium.com/

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

3 Comments