Pluralistic: Solving the Moderator's Trilemma with Federation (04 Mar 2023)


Today's links


A trilemma Venn diagram, showing three ovoids in a triangular form, which intersect at their tips, but not in the middle. The ovoids are labeled 'Avoid angering users,' 'Diverse userbase,' 'Centralized platforms.' In the center of the ovoids is the Mastodon mascot. The background is composed of dead Twitter birds on their backs with exes for eyes.

Solving the Moderator's Trilemma with Federation (permalink)

The classic trilemma goes: "Fast, cheap or good, pick any two." The Moderator's Trilemma goes, "Large, diverse userbase; centralized platforms; don't anger users – pick any two." The Moderator's Trilemma is introduced in "Moderating the Fediverse: Content Moderation on Distributed Social Media," a superb paper from Alan Rozenshtein of U of Minnesota Law, forthcoming in the journal Free Speech Law, available as a prepub on SSRN:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4213674#maincontent

Rozenshtein proposes a solution (of sorts) to the Moderator's Trilemma: federation. De-siloing social media, breaking it out of centralized walled gardens and recomposing it as a bunch of small servers run by a diversity of operators with a diversity of content moderation approaches. The Fediverse, in other words.

In Albert Hirschman's classic treatise Exit, Voice, and Loyalty, stakeholders in an institution who are dissatisfied with its direction have two choices: voice (arguing for changes) or exit (going elsewhere). Rozenshtein argues that Fediverse users (especially users of Mastodon, the most popular part of the Fediverse) have more voice and more "freedom of exit":

https://en.wikipedia.org/wiki/Exit,_Voice,_and_Loyalty

Large platforms – think Twitter, Facebook, etc – are very unresponsive to users. Most famously, Facebook polled its users on whether they wanted to be spied on. Faced with overwhelming opposition to commercial surveillance, Facebook ignored the poll result and cranked the surveillance dial up to a million:

https://www.nbcnews.com/tech/tech-news/facebook-ignores-minimal-user-vote-adopts-new-privacy-policy-flna1c7559683

A decade later, Musk performed the same stunt, asking users whether they wanted him to fuck all the way off from the company, then ignored the vox populi, which, in this instance, was not vox Dei:

https://apnews.com/article/elon-musk-twitter-inc-technology-business-8dac8ae023444ef9c37ca1d8fe1c14df

Facebook, Twitter and other walled gardens are designed to be sticky-traps, relying on high switching costs to keep users locked within their garden walls which are really prison walls. Internal memos from the companies reveal that this strategy is deliberate, designed to keep users from defecting even as the service degrades:

https://www.eff.org/deeplinks/2021/08/facebooks-secret-war-switching-costs

By contrast, the Fediverse is designed for ease of exit. With one click, users can export the list of the accounts they follow, block and mute, as well as the accounts that follow them. With one more click, users can import that data into any other Fediverse server and be back up and running with almost no cost or hassle:

https://pluralistic.net/2022/12/23/semipermeable-membranes/

Last month, "Nathan," the volunteer operator of mastodon.lol, announced that he was pulling the plug on the server because he was sick of his users' arguments about the new Harry Potter game. Many commentators pointed to this as a mark against federated social media, "You can't rely on random, thin-skinned volunteer sysops for your online social life!"

https://mastodon.lol/@nathan/109836633022272265

But the mastodon.lol saga demonstrates the strength of federated social media, not its weakness. After all, 450 million Twitter users are also at the mercy of a thin-skinned sysop – but when he enshittifies his platform, they can't just export their data and re-establish their social lives elsewhere in two clicks:

Mastodon.lol shows us how, if you don't like your host's content moderation policies, you can exercise voice – even to the extent of making him so upset that he shuts off his server – and where voice fails, exit steps in to fill the gap, providing a soft landing for users who find the moderation policies untenable:

https://pluralistic.net/2023/02/19/twiddler/

Traditionally, centralization has been posed as beneficial to content moderation. As Rozenshtein writes, there's a received wisdom that a company that can "enclose" its users and lock them in has an incentive to invest in better user experience, while companies whose users can easily migrate to rivals are less invested in those users.

And centralized platforms are more nimble. The operators of centralized systems can add hundreds of knobs and sliders to their back end and twiddle them at will. They act unilaterally, without having to convince other members of a federation to back their changes.

Centralized platforms claim that their most powerful benefit to users is extensive content moderation. As Tarleton Gillespie writes, “Moderation is central to what platforms do, not peripheral… [it] is, in many ways, the commodity that platforms offer":

https://yalebooks.yale.edu/book/9780300261431/custodians-of-the-internet/

Centralized systems claim that their enclosure keeps users safe – from bad code and bad people. Though Rozenshtein doesn't say so, it's important to note that this claim is wildly oversold. Platforms routinely fail at preventing abuse:

https://www.nbcnews.com/nbc-out/out-news/sexual-assault-harassment-bullying-trans-students-say-targeted-school-rcna7803

And they also fail at blocking malicious code:

https://www.scmagazine.com/news/threats/apple-bugs-ios-macos_new_class

But even where platforms do act to "keep users safe," they fail, thanks to the Moderator's Trilemma. Setting speech standards for millions or even billions of users is an impossible task. Some users will always feel like speech is being underblocked – while others will feel it's overblocked (and both will be right!):

https://www.eff.org/deeplinks/2021/07/right-or-left-you-should-be-worried-about-big-tech-censorship

And platforms play very fast and loose with their definition of "malicious code" – as when Apple blocked OG App, an Instagram ad-blocker that gave you a simple feed consisting of just the posts from the people you followed:

https://pluralistic.net/2023/02/05/battery-vampire/#drained

To resolve the Moderator's Trilemma, we need to embrace subsidiarity: "decisions should be made at the lowest organizational level capable of making such decisions."

https://pluralistic.net/2023/02/07/full-stack-luddites/#subsidiarity

For Rozenshtein, "content-moderation subsidiarity devolves decisions to the individual instances that make up the overall network." The fact that users can leave a server and set up somewhere else means that when a user gets pissed off enough about a moderation policy, they don't have to choose between leaving social media or tolerating the policy – they can simply choose another server that's part of the same federation.

Rozenshtein asks whether Reddit is an example of this, because moderators of individual subreddits are given broad latitude to set their own policies and anyone can fork a subreddit into a competing community with different moderation norms. But Reddit's devolution is a matter of policy, not architecture – subreddits exist at the sufferance of Reddit's owners (and Reddit is poised to go public, meaning those owners will include activist investors and large institutions that might not care about your little community). You might be happy about Reddit banning /r_TheDonald, but if they can ban that subreddit, they can ban any subreddit. Policy works well, but fails badly.

By moving subsidiarity into the technical architecture, rather than human policy, the fediverse can move from antagonism (the "zero-sum destructiveness" that dominates current online debate) to agonism, where your opponent isn't an enemy – they are a "political adversary":

https://www.yalelawjournal.org/article/the-administrative-agon

Here, Rozenshtein cites Aymeric Mansoux and Roel Roscam Abbing's "Seven Theses On The Fediverse And The Becoming Of Floss":

https://test.roelof.info/seven-theses.html

For this to happen, different ideologies must be allowed to materialize via different channels and platforms. An important prerequisite is that the goal of political consensus must be abandoned and replaced with conflictual consensus…

So your chosen Mastodon server "may have rules that are far more restrictive than those of the major social media platforms." But the whole Fediverse "is substantially more speech protective than are any of the major social media platforms, since no user or content can be permanently banned from the network and anyone is free to start an instance that communicates both with the major Mastodon instances and the peripheral, shunned instances."

A good case-study here is Gab, a Fediverse server by and for far-right cranks, conspiratorialists and white nationalists. Most Fediverse servers have defederated (that is, blocked) Gab, but Gab is still there, and Gab has actually defederated from many of the remaining servers, leaving its users to speak freely – but only to people who want to hear what they have to say.

This is the true meaning of "freedom of speech isn't freedom of reach." Willing listeners aren't blocked from willing speakers – but you don't have the right to be heard by people who don't want to talk to you:

https://pluralistic.net/2022/12/10/e2e/#the-censors-pen

Fediverse servers are (thus far) nonprofits or hobbyist sites, and don't have the same incentives to drive "engagement" to maximize the opportunties to show advertisements. Fediverse applications are frequently designed to be antiviral – that is, to prevent spectacular spreads of information across the system.

It's possible – likely, even – that future Fediverse servers will be operated by commercial operators seeking to maximize attention in order to maximize revenue – but the users of these servers will still have the freedom of exit that they enjoy on today's Jeffersonian volunteer-run servers – and so commercial servers will have to either curb their worst impulses or lose their users to better systems.

I'll note here that this is a progressive story of the benefits of competition – not the capitalist's fetishization of competition for its own sake, but rather, competition as a means of disciplining capital. It can be readily complimented by discipline through regulation – for example, extending today's burgeoning crop of data-protection laws to require servers to furnish users with exports of their follow/follower data so they can go elsewhere.

There's another dimension to decentralized content moderation that exit and voice don't address – moderating "harmful" content. Some kinds of harm can be mitigated through exit – if a server tolerates hate speech or harassment, you can go elsewhere, preferably somewhere that blocks your previous server.

But there are other kinds of speech that must not exist – either because they are illegal or because they enact harms that can't be mitigated by going elsewhere (or both). The most spectacular version of this is Child Sex Abuse Material (CSAM), a modern term-of-art to replace the more familiar "child porn."

Rozenshtein says there are "reasons for optimism" when it comes to the Fediverse's ability to police this content, though as he unpacked this idea, I found it much weaker than his other material. Rozenshtein proposes that Fediverse hosts could avail themselves of PhotoDNA, Microsoft's automated scanning tool, to block and purge themselves of CSAM, while noting that this is "hardly foolproof."

If automated scanning fails, Rozenshtein allows that this could cause "greater consolidation" of Mastodon servers to create the economies of scale to pay for more active, human moderation, which he compares to the consolidation of email that arose as a result of the spam-wars. But the spam-wars have been catastrophic for email as a federated system and produced all kinds of opportunities for mischief by the big players:

https://pluralistic.net/2021/10/10/dead-letters/

Rozenshtein: "There is a tradeoff between a vibrant and diverse communication system and the degree of centralized control that would be necessary to ensure 100% filtering of content. The question, as yet unknown, is how stark that tradeoff is."

The situation is much simpler when it comes to servers hosted by moderators who are complicit in illegal conduct: "the Fediverse may live in the cloud, its servers, moderators, and users are physically located in nations whose governments are more than capable of enforcing local law." That is, people who operate "rogue" servers dedicated to facilitating assassination, CSAM, or what-have-you will be arrested, and their servers will be seized.

Fair enough! But of course, this butts up against one of the Fediverse's shortcomings: it isn't particularly useful for promoting illegal speech that should be legal, like the communications of sex workers who were purged from the internet en masse following the passage of SESTA/FOSTA. When sex workers tried to establish a new home in the fediverse on a server called Switter, it was effectively crushed.

This simply reinforces the idea that code is no substitute for law, and while code can interpret bad law as damage and route around it, it can only do so for a short while. The best use of speech-enabling code isn't to avoid the unjust suppression of speech – it's to organize resistance to that injustice, including, if necessary, the replacement of the governments that enacted it:

https://pluralistic.net/2022/03/27/the-best-defense-against-rubber-hose-cryptanalysis/

Rozenshtein briefly addresses the question of "filter bubbles," and notes that there is compelling research that filter bubbles don't really exist, or at least, aren't as important to our political lives as once thought:

https://sciendo.com/article/10.2478/nor-2021-0002

Rozenshtein closes by addressing the role policy can play in encouraging the Fediverse. First, he proposes that governments could host their own servers and use them for official communications, as the EU Commission did following Musk's Twitter takeover:

https://social.network.europa.eu

He endorses interoperability mandates which would required dominant platforms to connect to the fediverse (facilitating their users' departure), like the ones in the EU's DSA and DMA, and proposed in US legislation like the ACCESS Act:

https://www.eff.org/deeplinks/2022/04/eu-digital-markets-acts-interoperability-rule-addresses-important-need-raises

To get a sense of how that would work, check out "Interoperable Facebook," a video and essay I put together with EFF to act as a kind of "design fiction," in the form of a user manual for a federated, interoperable Facebook:

https://www.eff.org/interoperablefacebook

He points out that this kind of mandatory interop is a preferable alternative to the unconstitutional (and unworkable!) speech bans proposed by Florida and Texas, which limit the ability of platforms to moderate speech. Indeed, this is an either-or proposition – under the terms proposed by Florida and Texas, the Fediverse couldn't operate.

This is likewise true of proposals to eliminate Section 230, the law that immunizes platforms from federal liability for most criminal speech acts committed by their users. While this law is incorrectly smeared as a gift to Big Tech, it is most needed by small services that can't possibly afford to monitor everything their users say:

https://www.techdirt.com/2020/06/23/hello-youve-been-referred-here-because-youre-wrong-about-section-230-communications-decency-act/

One more recommendation from Rozenshtein: treat interop mandates as an alternative (or adjunct) to antitrust enforcement. Competition agencies could weigh interoperability with the Fediverse by big platforms to determine whether to enforce against them, and enforcement orders could include mandates to interoperate with the Fediverse. This is a much faster remedy than break-ups, which Rozenshtein is dubious of because they are "legally risky" and "controversial."

To this, I'd add that even for people who would welcome break-ups (like me!) they are sloooow. The breakup of AT&T took 69 years. By contrast, interop remedies would give relief to users right now:

https://pluralistic.net/2021/11/14/jam-to-day/

Hey look at this (permalink)


A Wayback Machine banner.

This day in history (permalink)

#20yrsago Spectrum Etiquette: Two Proposals https://memex.craphound.com/2003/03/02/spectrum-etiquette-two-proposals/

#20yrsago Farber and Faulhaber’s argument for commons spectrum allocation https://memex.craphound.com/2003/03/01/farber-and-faulhabers-argument-for-commons-spectrum-allocation/

#20yrsago “Easement commons” isn’t enough https://memex.craphound.com/2003/03/01/easement-commons-isnt-enough/

#20yrsago Commons spectrum isn’t like a park! https://memex.craphound.com/2003/03/01/commons-spectrum-isnt-like-a-park/

#20yrsago Moot court on property versus commons for spectrum allocation https://memex.craphound.com/2003/03/01/moot-court-on-property-versus-commons-for-spectrum-allocation/

#20yrsago The people’s First Amendment rights should not be auctioned off to media barons https://memex.craphound.com/2003/03/01/the-peoples-first-amendment-rights-should-not-be-auctioned-off-to-media-barons/

#15yrsago Public broadcaster + Bittorrent = massive public savings https://nrkbeta.no/2008/03/02/thoughts-on-bittorrent-distribution-for-a-public-broadcaster/

#15yrsago How (and why) the Great Firewall of China works https://www.theatlantic.com/magazine/archive/2008/03/the-connection-has-been-reset/306650/

#15yrsago Nine Inch Nails goes Creative Commons remix-friendly with new album https://web.archive.org/web/20080305012426/http://ghosts.nin.com/main/home

#15yrsago Engineering approach to global climate change https://craphound.com/etech08_Saul_Griffith_energy_literacy.txt

#15yrsago Question Box: the Internet for remote places, no literacy or keyboards required https://www.questionbox.org

#15yrsago Why hardware ebook readers are a dead end (for now, anyway) https://www.locusmag.com/Features/2008/03/cory-doctorow-put-not-your-faith-in.html

#15yrsago Fake cold remedy Airborne settles lawsuit — get your cash back https://archive.nytimes.com/thelede.blogs.nytimes.com/2008/03/04/makers-of-airborne-settle-false-ad-suit-with-refunds/

#10yrsago Cops abduct 6-y-o for going to the store on her own, initially refuse to return to her dad https://www.freerangekids.com/cops-detain-6-year-old-for-walking-around-neighborhood-and-it-gets-worse/

#10yrsago Inside the prosecution of Aaron Swartz https://finance.yahoo.com/news/life-inside-aaron-swartz-investigation-005452894.html

#10yrsago What’s the most utopian fiction of all? https://locusmag.com/2013/03/cory-doctorow-ten-years-on/

#10yrsago How an algorithm came up with Amazon’s KEEP CALM AND RAPE A LOT t-shirt https://web.archive.org/web/20130305012153/http://iam.peteashton.com/keep-calm-rape-tshirt-amazon/

#10yrsago America’s six-strikes copyright system is a nightmare https://www.eff.org/deeplinks/2013/02/copyright-propaganda-machine-gets-new-agent-your-isp

#10yrsago Journalists took secret money for critical pieces about Malaysian opposition candidate https://www.buzzfeednews.com/article/rosiegray/covert-malaysian-campaign-touched-a-wide-range-of-american-m

#10yrsago US Trade Rep orders Canada to comply with the dead-and-buried ACTA treaty, Canada rolls over and wets itself https://www.eff.org/deeplinks/2013/03/us-trade-office-calls-acta-back-dead-and-canada-complies

#10yrsago Impulse: At long last, a new Jumper novel from Steven Gould https://memex.craphound.com/2013/03/01/impulse-at-long-last-a-new-jumper-novel-from-steven-gould/

#5yrsago Adblock will cache popular Javascript libraries, meaning adblocked pages will be faster and less janky https://www.bleepingcomputer.com/news/software/adblock-adds-feature-to-cache-popular-javascript-libraries/

#5yrsago Ajit Pai forced to return the gun the NRA gave him as a prize for his neutracidal rampage https://www.politico.com/story/2018/03/01/fccs-aji-pai-declines-nra-gun-award-381975

#5yrsago French Polynesia says it didn’t renew its deal with the Seasteaders, a group of libertarian separatists https://www.rnz.co.nz/international/pacific-news/351420/french-polynesia-sinks-floating-island-project

#5yrsago CEO of Trustico emails 23,000 HTTPS private keys, triggering panicked mass-revocation https://arstechnica.com/information-technology/2018/03/23000-https-certificates-axed-after-ceo-e-mails-private-keys/

#5yrsago United axes its employee bonus program, replaces it with a lottery, saving millions https://thepointsguy.co.uk/2018/03/united-cutting-employee-bonuses/

#5yrsago Don’t give a dime to the DCCC, they’ll just use to front DINOs and smear Justice Democrats https://www.thenation.com/article/archive/when-dccc-calls-hang-up-the-phone/

#1yrago How English libel law enables Russian kleptocrats https://pluralistic.net/2022/03/04/londongrad/#enablers

#1yrago Defi and Shadow Banking 2.0 https://pluralistic.net/2022/03/02/shadow-banking-2-point-oh/#leverage

#1yrago How "hollowed" hotels are destroying worker rights: REITs aren't just for money laundering https://pluralistic.net/2022/03/01/reit-modernization-act/#reit-makes-might


Colophon (permalink)

Today's top sources: Ji Fu (https://libranet.de/profile/fu).

Currently writing:

  • Picks and Shovels, a Martin Hench noir thriller about the heroic era of the PC. Friday's progress: 513 words (112141 words total)

  • The Bezzle, a Martin Hench noir thriller novel about the prison-tech industry. FIRST DRAFT COMPLETE, WAITING FOR EDITORIAL REVIEW

  • A Little Brother short story about DIY insulin PLANNING

  • Vigilant, Little Brother short story about remote invigilation. ON SUBMISSION

  • Moral Hazard, a short story for MIT Tech Review's 12 Tomorrows. FIRST DRAFT COMPLETE, ACCEPTED FOR PUBLICATION

  • Spill, a Little Brother short story about pipeline protests. ON SUBMISSION

Latest podcast: Twiddler https://craphound.com/news/2023/02/27/twiddler/

Upcoming appearances:

Recent appearances:

Latest books:

Upcoming books:

  • Red Team Blues: "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books, April 2023

  • The Internet Con: A nonfiction book about interoperability and Big Tech, Verso, September 2023

  • The Lost Cause: a post-Green New Deal eco-topian novel about truth and reconciliation with white nationalist militias, Tor Books, November 2023

This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.

How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/@pluralistic

Medium (no ads, paywalled):

https://doctorow.medium.com/

(Latest Medium column: "United We Stand: Creation is collective - and so is bargaining" https://pluralistic.net/2023/02/26/united-we-stand/)

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla


Today's links


A trilemma Venn diagram, showing three ovoids in a triangular form, which intersect at their tips, but not in the middle. The ovoids are labeled 'Avoid angering users,' 'Diverse userbase,' 'Centralized platforms.' In the center of the ovoids is the Mastodon mascot. The background is composed of dead Twitter birds on their backs with exes for eyes.

Solving the Moderator's Trilemma with Federation (permalink)

The classic trilemma goes: "Fast, cheap or good, pick any two." The Moderator's Trilemma goes, "Large, diverse userbase; centralized platforms; don't anger users – pick any two." The Moderator's Trilemma is introduced in "Moderating the Fediverse: Content Moderation on Distributed Social Media," a superb paper from Alan Rozenshtein of U of Minnesota Law, forthcoming in the journal Free Speech Law, available as a prepub on SSRN:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4213674#maincontent

Rozenshtein proposes a solution (of sorts) to the Moderator's Trilemma: federation. De-siloing social media, breaking it out of centralized walled gardens and recomposing it as a bunch of small servers run by a diversity of operators with a diversity of content moderation approaches. The Fediverse, in other words.

In Albert Hirschman's classic treatise Exit, Voice, and Loyalty, stakeholders in an institution who are dissatisfied with its direction have two choices: voice (arguing for changes) or exit (going elsewhere). Rozenshtein argues that Fediverse users (especially users of Mastodon, the most popular part of the Fediverse) have more voice and more "freedom of exit":

https://en.wikipedia.org/wiki/Exit,_Voice,_and_Loyalty

Large platforms – think Twitter, Facebook, etc – are very unresponsive to users. Most famously, Facebook polled its users on whether they wanted to be spied on. Faced with overwhelming opposition to commercial surveillance, Facebook ignored the poll result and cranked the surveillance dial up to a million:

https://www.nbcnews.com/tech/tech-news/facebook-ignores-minimal-user-vote-adopts-new-privacy-policy-flna1c7559683

A decade later, Musk performed the same stunt, asking users whether they wanted him to fuck all the way off from the company, then ignored the vox populi, which, in this instance, was not vox Dei:

https://apnews.com/article/elon-musk-twitter-inc-technology-business-8dac8ae023444ef9c37ca1d8fe1c14df

Facebook, Twitter and other walled gardens are designed to be sticky-traps, relying on high switching costs to keep users locked within their garden walls which are really prison walls. Internal memos from the companies reveal that this strategy is deliberate, designed to keep users from defecting even as the service degrades:

https://www.eff.org/deeplinks/2021/08/facebooks-secret-war-switching-costs

By contrast, the Fediverse is designed for ease of exit. With one click, users can export the list of the accounts they follow, block and mute, as well as the accounts that follow them. With one more click, users can import that data into any other Fediverse server and be back up and running with almost no cost or hassle:

https://pluralistic.net/2022/12/23/semipermeable-membranes/

Last month, "Nathan," the volunteer operator of mastodon.lol, announced that he was pulling the plug on the server because he was sick of his users' arguments about the new Harry Potter game. Many commentators pointed to this as a mark against federated social media, "You can't rely on random, thin-skinned volunteer sysops for your online social life!"

https://mastodon.lol/@nathan/109836633022272265

But the mastodon.lol saga demonstrates the strength of federated social media, not its weakness. After all, 450 million Twitter users are also at the mercy of a thin-skinned sysop – but when he enshittifies his platform, they can't just export their data and re-establish their social lives elsewhere in two clicks:

Mastodon.lol shows us how, if you don't like your host's content moderation policies, you can exercise voice – even to the extent of making him so upset that he shuts off his server – and where voice fails, exit steps in to fill the gap, providing a soft landing for users who find the moderation policies untenable:

https://pluralistic.net/2023/02/19/twiddler/

Traditionally, centralization has been posed as beneficial to content moderation. As Rozenshtein writes, a company that can "enclose" its users and lock them in has an incentive to invest in better user experience, while companies whose users can easily migrate to rivals are less invested in those users.

And centralized platforms are more nimble. The operators of centralized systems can add hundreds of knobs and sliders to their back end and twiddle them at will. They act unilaterally, without having to convince other members of a federation to back their changes.

Centralized platforms claim that their most powerful benefit to users is extensive content moderation. As Tarleton Gillespie writes, “Moderation is central to what platforms do, not peripheral… [it] is, in many ways, the commodity that platforms offer":

https://yalebooks.yale.edu/book/9780300261431/custodians-of-the-internet/

Centralized systems claim that their enclosure keeps users safe – from bad code and bad people. Though Rozenshtein doesn't say so, it's important to note that this claim is wildly oversold. Platforms routinely fail at preventing abuse:

https://www.nbcnews.com/nbc-out/out-news/sexual-assault-harassment-bullying-trans-students-say-targeted-school-rcna7803

And they also fail at blocking malicious code:

https://www.scmagazine.com/news/threats/apple-bugs-ios-macos_new_class

But even where platforms do act to "keep users safe," they fail, thanks to the Moderator's Trilemma. Setting speech standards for millions or even billions of users is an impossible task. Some users will always feel like speech is being underblocked – while others will feel it's overblocked (and both will be right!):

https://www.eff.org/deeplinks/2021/07/right-or-left-you-should-be-worried-about-big-tech-censorship

And platforms play very fast and loose with their definition of "malicious code" – as when Apple blocked OG App, an Instagram ad-blocker that gave you a simple feed consisting of just the posts from the people you followed:

https://pluralistic.net/2023/02/05/battery-vampire/#drained

To resolve the Moderator's Trilemma, we need to embrace subsidiarity: "decisions should be made at the lowest organizational level capable of making such decisions."

https://pluralistic.net/2023/02/07/full-stack-luddites/#subsidiarity

For Rozenshtein, "content-moderation subsidiarity devolves decisions to the individual instances that make up the overall network." The fact that users can leave a server and set up somewhere else means that when a user gets pissed off enough about a moderation policy, they don't have to choose between leaving social media or tolerating the policy – they can simply choose another server that's part of the same federation.

Rozenshtein asks whether Reddit is an example of this, because moderators of individual subreddits are given broad latitude to set their own policies and anyone can fork a subreddit into a competing community with different modeations norms. But Reddit's devolution is a matter of policy, not architecture – subreddits exist at the sufferance of Reddit's owners (and Reddit is poised to go public, meaning those owners will include activist investors and large institutions that might not care about your little community). You might be happy about Reddit banning /r_TheDonald, but if they can band that subreddit, they can ban any subreddit. Policy works well, but fails badly.

By moving subsidiarity into technical architecture, rather than human policy, the fediverse can move from antagonism (the "zero-sum destructiveness" that dominates current online debate) to agonism, where your opponent isn't an enemy – they are a "political adversary":

https://www.yalelawjournal.org/article/the-administrative-agon

Here, Rozenshtein cites Aymeric Mansoux and Roel Roscam Abbing's "Seven Theses On The Fediverse And The Becoming Of Floss":

https://test.roelof.info/seven-theses.html

For this to happen, different ideologies must be allowed to materialize via different channels and platforms. An important prerequisite is that the goal of political consensus must be abandoned and replaced with conflictual consensus…

So your chosen Mastodon server "may have rules that are far more restrictive than those of the major social media platforms." But the whole Fediverse "is substantially more speech protective than are any of the major social media platforms, since no user or content can be permanently banned from the network and anyone is free to start an instance that communicates both with the major Mastodon instances and the peripheral, shunned instances."

A good case-study here is Gab, a Fediverse server by and for far-right cranks, conspiratorialists and white nationalists. Most Fediverse servers have defederated (that is, blocked) Gab, but Gab is still there, and Gab has actually defederated from many of the remaining servers, leaving its users to speak freely – but only to people who want to hear what they have to say.

This is true meaning of "freedom of speech isn't freedom of reach." Willing listeners aren't blocked from willing speakers – but you don't have the right to be heard by people who don't want to talk to you:

https://pluralistic.net/2022/12/10/e2e/#the-censors-pen

Fediverse servers are (thus far) nonprofits or hobbyist sites, and don't have the same incentives to drive "engagement" to maximize the opportunties to show advertisements. Fediverse applications are frequently designed to be antiviral – that is, to prevent spectacular spreads of information across the system.

It's possible – likely, even – that future Fediverse servers will be operated by commercial operators seeking to maximize attention in order to maximize revenue – but the users of these servers will still have the freedom of exit that they enjoy on today's Jeffersonian volunteer-run servers – and so commercial servers will have to either curb their worst impulses or lose their users to better systems.

I'll note here that this is a progressive story of the benefits of competition – not the capitalist's fetishization of competition for its own sake, but rather, competition as a means of disciplining capital. It can be readily complimented by discipline through regulation – for example, extending today's burgeoning crop of data-protection laws to require servers to furnish users with exports of their follow/follower data so they can go elsewhere.

There's another dimension to decentralized content moderation that exit and voice don't address – moderating "harmful" content. Some kinds of harm can be mitigated through exit – if a server tolerates hate speech or harassment, you can go elsewhere, preferably somewhere that blocks your previous server.

But there are other kinds of speech that must not exist – either because they are illegal or because they enact harms that can't be mitigated by going elsewhere (or both). The most spectacular version of this is Child Sex Abuse Material (CSAM), a modern term-of-art to replace the more familiar "child porn."

Rozenshtein says there are "reasons for optimism" when it comes to the Fediverse's ability to police this content, though as he unpacked this idea, I found it much weaker than his other material. Rozenshtein proposes that Fediverse hosts could avail themselves of PhotoDNA, Microsoft's automated scanning tool, to block and purge themselves of CSAM, while noting that this is "hardly foolproof."

If automated scanning fails, Rozenshtein allows that this could cause "greater consolidation" of Mastodon servers to create the economies of scale to pay for more active, human moderation, which he compares to the consolidation of email that arose as a result of the spam-wars. But the spam-wars have been catastrophic for email as a federated system and produced all kinds of opportunities for mischief by the big players:

https://pluralistic.net/2021/10/10/dead-letters/

Rozenshtein: "There is a tradeoff between a vibrant and diverse communication system and the degree of centralized control that would be necessary to ensure 100% filtering of content. The question, as yet unknown, is how stark that tradeoff is."

The situation is much simpler when it comes to servers hosted by moderators who are complicit in illegal conduct: "the Fediverse may live in the cloud, its servers, moderators, and users are physically located in nations whose governments are more than capable of enforcing local law." That is, people who operate "rogue" servers dedicated to facilitating assassination, CSAM, or what-have-you will be arrested, and their servers will be seized.

Fair enough! But of course, this butts up against one of the Fediverse's shortcomings: it isn't particularly useful for promoting illegal speech that should be legal, like the communications of sex workers who were purged from the internet en masse following the passage of SESTA/FOSTA. When sex workers tried to establish a new home in the fediverse on a server called Switter, it was effectively crushed.

This simply reinforces the idea that code is no substitute for law, and while code can interpret bad law as damage and route around it, it can only do so for a short while. The best use of speech-enabling code isn't to avoid the unjust suppression of speech – it's to organize resistance to that injustice, including, if necessary, the replacement of the governments that enacted it:

https://pluralistic.net/2022/03/27/the-best-defense-against-rubber-hose-cryptanalysis/

Rozenshtein briefly addresses the question of "filter bubbles," and notes that there is compelling research that filter bubbles don't really exist, or at least, aren't as important to our political lives as once thought:

https://sciendo.com/article/10.2478/nor-2021-0002

Rozenshtein closes by addressing the role policy can play in encouraging the Fediverse. First, he proposes that governments could host their own servers and use them for official communications, as the EU Commission did following Musk's Twitter takeover:

https://social.network.europa.eu

He endorses interoperability mandates which would required dominant platforms to connect to the fediverse (facilitating their users' departure), like the ones in the EU's DSA and DMA, and proposed in US legislation like the ACCESS Act:

https://www.eff.org/deeplinks/2022/04/eu-digital-markets-acts-interoperability-rule-addresses-important-need-raises

To get a sense of how that would work, check out "Interoperable Facebook," a video and essay I put together with EFF to act as a kind of "design fiction," in the form of a user manual for a federated, interoperable Facebook:

https://www.eff.org/interoperablefacebook

He points out that this kind of mandatory interop is a preferable alternative to the unconstitutional (and unworkable!) speech bans proposed by Florida and Texas, which limit the ability of platforms to moderate speech. Indeed, this is an either-or proposition – under the terms proposed by Florida and Texas, the Fediverse couldn't operate.

This is likewise true of proposals to eliminate Section 230, the law that immunizes platforms from federal liability for most criminal speech acts committed by their users. While this law is incorrectly smeared as a gift to Big Tech, it is most needed by small services that can't possibly afford to monitor everything their users say:

https://www.techdirt.com/2020/06/23/hello-youve-been-referred-here-because-youre-wrong-about-section-230-communications-decency-act/

One more recommendation from Rozenshtein: treat interop mandates as an alternative (or adjunct) to antitrust enforcement. Competition agencies could weigh interoperability with the Fediverse by big platforms to determine whether to enforce against them, and enforcement orders could include mandates to interoperate with the Fediverse. This is a much faster remedy than break-ups, which Rozenshtein is dubious of because they are "legally risky" and "controversial."

To this, I'd add that even for people who would welcome break-ups (like me!) they are sloooow. The breakup of AT&T took 69 years. By contrast, interop remedies would give relief to users right now:

https://pluralistic.net/2021/11/14/jam-to-day/

Hey look at this (permalink)


A Wayback Machine banner.

This day in history (permalink)

#20yrsago Spectrum Etiquette: Two Proposals https://memex.craphound.com/2003/03/02/spectrum-etiquette-two-proposals/

#20yrsago Farber and Faulhaber’s argument for commons spectrum allocation https://memex.craphound.com/2003/03/01/farber-and-faulhabers-argument-for-commons-spectrum-allocation/

#20yrsago “Easement commons” isn’t enough https://memex.craphound.com/2003/03/01/easement-commons-isnt-enough/

#20yrsago Commons spectrum isn’t like a park! https://memex.craphound.com/2003/03/01/commons-spectrum-isnt-like-a-park/

#20yrsago Moot court on property versus commons for spectrum allocation https://memex.craphound.com/2003/03/01/moot-court-on-property-versus-commons-for-spectrum-allocation/

#20yrsago The people’s First Amendment rights should not be auctioned off to media barons https://memex.craphound.com/2003/03/01/the-peoples-first-amendment-rights-should-not-be-auctioned-off-to-media-barons/

#15yrsago Public broadcaster + Bittorrent = massive public savings https://nrkbeta.no/2008/03/02/thoughts-on-bittorrent-distribution-for-a-public-broadcaster/

#15yrsago How (and why) the Great Firewall of China works https://www.theatlantic.com/magazine/archive/2008/03/the-connection-has-been-reset/306650/

#15yrsago Nine Inch Nails goes Creative Commons remix-friendly with new album https://web.archive.org/web/20080305012426/http://ghosts.nin.com/main/home

#15yrsago Engineering approach to global climate change https://craphound.com/etech08_Saul_Griffith_energy_literacy.txt

#15yrsago Question Box: the Internet for remote places, no literacy or keyboards required https://www.questionbox.org

#15yrsago Why hardware ebook readers are a dead end (for now, anyway) https://www.locusmag.com/Features/2008/03/cory-doctorow-put-not-your-faith-in.html

#15yrsago Fake cold remedy Airborne settles lawsuit — get your cash back https://archive.nytimes.com/thelede.blogs.nytimes.com/2008/03/04/makers-of-airborne-settle-false-ad-suit-with-refunds/

#10yrsago Cops abduct 6-y-o for going to the store on her own, initially refuse to return to her dad https://www.freerangekids.com/cops-detain-6-year-old-for-walking-around-neighborhood-and-it-gets-worse/

#10yrsago Inside the prosecution of Aaron Swartz https://finance.yahoo.com/news/life-inside-aaron-swartz-investigation-005452894.html

#10yrsago What’s the most utopian fiction of all? https://locusmag.com/2013/03/cory-doctorow-ten-years-on/

#10yrsago How an algorithm came up with Amazon’s KEEP CALM AND RAPE A LOT t-shirt https://web.archive.org/web/20130305012153/http://iam.peteashton.com/keep-calm-rape-tshirt-amazon/

#10yrsago America’s six-strikes copyright system is a nightmare https://www.eff.org/deeplinks/2013/02/copyright-propaganda-machine-gets-new-agent-your-isp

#10yrsago Journalists took secret money for critical pieces about Malaysian opposition candidate https://www.buzzfeednews.com/article/rosiegray/covert-malaysian-campaign-touched-a-wide-range-of-american-m

#10yrsago US Trade Rep orders Canada to comply with the dead-and-buried ACTA treaty, Canada rolls over and wets itself https://www.eff.org/deeplinks/2013/03/us-trade-office-calls-acta-back-dead-and-canada-complies

#10yrsago Impulse: At long last, a new Jumper novel from Steven Gould https://memex.craphound.com/2013/03/01/impulse-at-long-last-a-new-jumper-novel-from-steven-gould/

#5yrsago Adblock will cache popular Javascript libraries, meaning adblocked pages will be faster and less janky https://www.bleepingcomputer.com/news/software/adblock-adds-feature-to-cache-popular-javascript-libraries/

#5yrsago Ajit Pai forced to return the gun the NRA gave him as a prize for his neutracidal rampage https://www.politico.com/story/2018/03/01/fccs-aji-pai-declines-nra-gun-award-381975

#5yrsago French Polynesia says it didn’t renew its deal with the Seasteaders, a group of libertarian separatists https://www.rnz.co.nz/international/pacific-news/351420/french-polynesia-sinks-floating-island-project

#5yrsago CEO of Trustico emails 23,000 HTTPS private keys, triggering panicked mass-revocation https://arstechnica.com/information-technology/2018/03/23000-https-certificates-axed-after-ceo-e-mails-private-keys/

#5yrsago United axes its employee bonus program, replaces it with a lottery, saving millions https://thepointsguy.co.uk/2018/03/united-cutting-employee-bonuses/

#5yrsago Don’t give a dime to the DCCC, they’ll just use to front DINOs and smear Justice Democrats https://www.thenation.com/article/archive/when-dccc-calls-hang-up-the-phone/

#1yrago How English libel law enables Russian kleptocrats https://pluralistic.net/2022/03/04/londongrad/#enablers

#1yrago Defi and Shadow Banking 2.0 https://pluralistic.net/2022/03/02/shadow-banking-2-point-oh/#leverage

#1yrago How "hollowed" hotels are destroying worker rights: REITs aren't just for money laundering https://pluralistic.net/2022/03/01/reit-modernization-act/#reit-makes-might


Colophon (permalink)

Today's top sources: Ji Fu (https://libranet.de/profile/fu).

Currently writing:

  • Picks and Shovels, a Martin Hench noir thriller about the heroic era of the PC. Friday's progress: 513 words (112141 words total)

  • The Bezzle, a Martin Hench noir thriller novel about the prison-tech industry. FIRST DRAFT COMPLETE, WAITING FOR EDITORIAL REVIEW

  • A Little Brother short story about DIY insulin PLANNING

  • Vigilant, Little Brother short story about remote invigilation. ON SUBMISSION

  • Moral Hazard, a short story for MIT Tech Review's 12 Tomorrows. FIRST DRAFT COMPLETE, ACCEPTED FOR PUBLICATION

  • Spill, a Little Brother short story about pipeline protests. ON SUBMISSION

Latest podcast: Twiddler https://craphound.com/news/2023/02/27/twiddler/

Upcoming appearances:

Recent appearances:

Latest books:

Upcoming books:

  • Red Team Blues: "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books, April 2023

  • The Internet Con: A nonfiction book about interoperability and Big Tech, Verso, September 2023

  • The Lost Cause: a post-Green New Deal eco-topian novel about truth and reconciliation with white nationalist militias, Tor Books, November 2023

This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.

How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/@pluralistic

Medium (no ads, paywalled):

https://doctorow.medium.com/

(Latest Medium column: "United We Stand: Creation is collective - and so is bargaining" https://pluralistic.net/2023/02/26/united-we-stand/)

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

1 Comment