Email could be the last federated internet technology — but it isn’t.
It feels like only yesterday that we were living through the Substack bubble, as mailing lists enjoyed a new renaissance (rebranded as “newsletters”), a tangible expression of the techlash and our collective disgust with the platforms and their attempts to enclose the internet and convert it to “five giant websites, each filled with screenshots of text from the other four.”
In the abstract, mailing lists/newsletters represent the promise of a return to a Jeffersonian internet, where each of us can garden own little patch, not subject to the whims of third parties. That, after all, is the original design brief of the internet, to be an “end-to-end network” where any party can connect to any other party without needing permission from anyone else.
That’s the promise of newsletters. The reality is a lot messier — and more alarming.
In early 2020, I left Boing Boing, a website I wrote on for 19 years, helping to build up a media platform that was not reliant on social media platforms for its audience (which isn’t to say we were fully insulated from Big Tech’s choices; periodic algorithmic changes at Google could have a huge effect on our reach and fortunes). The reasons for leaving were complicated, but ultimately, it was time. I’d done one thing for 19 years, now I wanted to do something else.
That something else is Pluralistic, a blog that publishes in daily(ish) editions consisting of 1–10 articles, ranging from brief blog mentions to 2–3,000 word essays. Pluralistic doesn’t have a couple decades’ worth of readership behind it, so I set about to find ways to reach readers that wouldn’t make me a digital sharecropper inside someone’s paywall, with an archive that could be vanished in an instant if a company kicked me off, or went bankrupt, or sold out, or “pivoted” (ugh).
The strategy I embraced is called “POSSE” — Post Own Site, Syndicate Everywhere. A POSSE publication has an official, canonical home on a server of the author’s control, but everything that appears on that permalink site is also published simultaneously on other platforms, to reach readers where they live.
For me, that means publishing the canonical day’s edition on a self-hosted WordPress blog on a server that my friend and sysadmin Ken Snider runs, in a rack at Torix, a giant, systemically important Toronto data-center where the world’s major backbone lines all cross over (readers of my fiction may recognize Torix as the backdrop for my story “When Sysadmins Ruled the Earth”).
This same edition is mirrored in near-realtime to a Twitter feed and a Mastodon feed (reformatted as threads), a Tumblr feed, a Medium feed, Discourse feed, a fulltext RSS feed, and a newsletter. Though I have some python scripts that help with this syndication it’s still a lot of manual work.
The reason for all this complexity and manual work is that I want Pluralistic to accomplish two contradictory goals: first, to spread as widely as possible, and second, to be my own publication, independent of corporate control and the risks of corporate incompetence, malevolence or indifference.
I could build a large following by focusing on just one platform and optimizing for it, but then if the platform terminated my account, my ability to publish and be read would disappear in an instant.
The idea, then, is to publish all my work to a variety of platforms, owned by a variety of corporate masters of varying degrees of competence, beneficence and conscientiousness, but to always be directing readers to the platforms I control: the blog, its RSS feed, and its newsletter.
I host that newsletter myself, because I can, and because hosting on someone else’s mailing list platform — Google Groups, Substack, Ghost, Tinyletter, Mailchimp, etc — exposes me to the risk of sudden termination, and exposes my readers to the risk of breach and privacy invasion.
Hosting your own mailing list used to be a common activity, but the growth of corporate mailing-list platforms, starting with ancient pioneers like Yahoo Groups, gradually siphoned off the demand for standalone mailing list software. Today, the only major, actively maintained free/open mailing list manager is the hoary Mailman, which has slipped behind the commercial offerings in flexibility and sophistication, but which still does the job.
Ken hosts a Mailman instance for me on his server as well, connected to his email server, which is also where I get my personal email for doctorow@craphound.com, an email address I’ve maintained for some 25 years now.
Mailman may be unpolished and retro, but it fills my needs very well: namely, it’s made it easy for thousands of people to sign up to get my daily newsletter, which I deliver to them using infrastructure I control. The daily editions are free from any tracking or ads — none of that creepy stuff that lets owners of commercial newsletters tell whether or not you’ve opened a message in your own mailbox (seriously, who the hell thought this was an acceptable idea?).
But the notion of newsletters as an end-to-end, self-hosted alternative to a Big Tech sharecrop is more theoretical than practical. It turns out that even if you have the wherewithal to stand up and secure an email server and a Mailman server on fast hardware in a leading data-center, you may still need to cede control to a commercial newsletter host.
Every week, I get a flood of messages from Mailman telling me about the users whose subscriptions have been automatically cancelled because their own email providers have rejected too many of my messages as suspected spam.
That’s a circuit-breaker built into Mailman: if you keep sending messages that a remote server classifies as spam, your whole mailserver can be added to one of the internet-wide “blackholes” — blocklists of servers accused of providing aid and comfort to spammers. The majority of the world’s mailservers subscribe to these blocklists, and woe betide you if your own mailserver ends up on one of these lists, because you will be cut off from the rest of the world, maybe permanently.
But my newsletter isn’t spam. Mailman uses a “double opt-in” verification system: to subscribe, you have to fill in and submit a form, then you have to click a link in a verification email. Everyone who subscribes to my newsletter took an active step to join it, and then confirmed that intent by taking a second active step.
What’s more, my mailserver doesn’t send spam. Ken provides email to a handful of close friends, old internet people like me, and none of us send out spam, nor have we ever.
Nevertheless, my newsletter editions keep getting blocked. For example, Comcast blocked all my messages for weeks because they subscribe to Vade’s blocklist, and Vade was misclassifying Mailman’s confirmation emails themselves as spam. That is, if you took steps to prevent spam, Vade would call you a spammer and block all your email for all of its customers, which include millions of Comcast subscribers.
This is a neat little example of the problems of email concentration, multiplied by the problems of automated email classification. The once-diverse world of email hosting —millions of similarly-sized email hosts and a handful of large institutional providers serving companies, universities and governments — has been denuded, with a couple of giant commercial providers dominating corporate, paid for services, and a couple more giant free webmail providers.
This concentration means that the communications of tens of millions of people are dependent on the invisible, inscrutable and inexplicable decisions of the administrators for a small coterie of giant firms.
That would be bad enough, but those administrators themselves no longer even make the decisions about what to block and what to let through. Instead, they outsource these choices to another, hyper-concentrated group of spamfighting services. Companies like Vade use automated, algorithmic filtering to (mis)identify emails; while realtime blocklists have a combination of public and secret heuristics for deciding which servers should be blocked from the internet altogether.
And because these decisions — algorithmic and human — can disrupt communications for millions of people, the individuals with the power to hear petitions to unblock a server or revise an algorithmic judgement are overwhelmed and insulate themselves from the majority of complaints, many of which, after all, come from actual spammers hoping to win back their right to spam.
Which leaves anyone running their own mailserver — with or without a newsletter that relies on it — in a situation of great precarity. I mentioned that I have been sending and receiving emails at doctorow@craphound.com for a quarter-century — but there was a three year period in which I could not send messages to anyone who relied on AT&T for their email. AT&T — and the hundreds of local ISPs it gobbled up and put under its corporate management — treated all mail from my mailserver, which had never, ever sent a single spam, as spam. Every month for three years, I sent fresh petitions to AT&T’s email management team, and never heard back (a loud Twitter campaign eventually got the company’s attention, but that hardly scales).
My newsletter hit a fresh peak of automated unsubscriptions earlier this month, and, after investigating, Ken emailed me, more discouraged than I’ve ever known him to be. He had discovered that one of the largest free email hosts — with more than a billion users — had been automatically marking all my newsletter messages as spam, apparently because I had taken two weeks off from publishing while recovering from hip-replacement surgery. The gap, and the restart, triggered some algorithmic tripwire, and without any human oversight or intervention, I was now cut off from thousands of my subscribers, each of whom had performed an elaborate double-opt-in to request and verify their subscriptions.
Ken wrote: “I think it may be time to consider a commercial mailhost for the pluralist mailing list…These aren’t things I can configure or protect against — these are almost certainly heuristics that, as a small mailhost, we are never going to escape. I imagine this is only going to get worse over time.”
The fact that I have a long history online means that I am in the top 0.00001% of people situated to solve these problems on their own. That free webmail provider? I know someone who’s pretty senior there, and that person got their colleagues to manually exempt my server from future blocking. In theory, I can do this for all of the major mail providers, one at a time, and cross my fingers.
I assume that’s how the proprietors of successful, standalone mailing lists manage: internet pioneer Dave Farber has run his Interesting People list since 1993. Its readership are a who’s-who of respected internet technology figures, the kinds of people who might intervene on Dave’s behalf if he gets blocked by a bad algorithm or a spurious blackhole accusation (he’s also hosted by Seclists, which is operated by and for the internet’s security community, the sort of people with after-hours unlisted numbers for the on-duty manager of every data center in the world).
With a lot of work, I might just be able to continue to host my own newsletter and email. Maybe. But almost no one else can.
Instead, they’ll pay for access to “email infrastructure.” This infrastructure isn’t technical — Ken’s setup and hosting are equivalent to these companies’ back-ends. It’s commercial.
These “email providers” have commercial relationships with the administrators of the world’s consolidated mail-hosting companies and the spam-detection services that sell to them, and they have staff whose job is to keep up with and participate in the committees that oversee the realtime blackholes.
When the spam-wars began and network administrators banded together to block “bad” email servers that allowed or facilitated spam, open internet advocates predicted that any victories they won against spammers would be short-lived.
Those open internet people were right: today, it’s harder than ever to send legitimate email unless you can convince (that is, pay) a handful of titans to relay your communications — and spam continues apace.
I get literally hundreds of PR pitches every day, almost all of them sent by publicists who host mailing lists on Mailchimp. I’ve never signed up for these lists, and never verified my subscription to them. PR professionals buy “press” mailing lists from sloppy vendors, then indiscriminately spam everyone on those lists with whatever harebrained crap they’re hoping to publicize today.
Mailchimp will let me unsubscribe from these lists — one at a time. But they won’t let me block PR companies from adding me to future lists, and they won’t show me all the lists I’m subscribed to. By any objective standard, Mailchimp is a commercial, mass-scale spam facilitator — and I’m not. But Mailchimp is a major mail provider, and if they get blocked, they get unblocked quickly. If AT&T started bouncing Mailchimp messages, it wouldn’t take them three years to get someone at Ma Bell to remove the block.
Large companies have been trying to enclose the open internet forever. Facebook wants to absorb the parts of the web it can and banish the rest. Spotify is removing podcasts from the open ecosystem and locking them inside its walled garden, and it’s reinvented payola for streaming music by replacing albums (which artists and labels control, and are the same whether you’re on Spotify or iTunes or just in your MP3 player) with playlists that are exclusive to its service, which you can’t bring with you if you quit.
These technical matters — how mailservers are configured, the appeals process for blackholes, the nature of large companies anti-spam algorithms — are niggling, abstract and obscure. But when we talk about digital sharecropping — about the ways that creative workers are corralled into making money for giant companies, beholden to their ongoing largese — these are the mechanisms that allow that to happen.