The hard problem of politics, religion, advertising and literature
Publishing is doing great
Publishing is doing great. Despite panic at the start of the lockdown, book sales were actually up during lockdown, as people turned to books to pass the time, joining online bookclubs and finding ways to support their local indie booksellers.
But authorship? Not so great.
Every part of the publishing supply chain has undergone radical concentration over the past 40 years, starting with consolidation of mass-market distribution in the 1980s. “Mass market” books are produced for sale in non-bookseller channels —pharmacies, grocery stores, news-stands, etc (books produced for sale in bookstores are called “trade books” because they’re sold through the bookselling trade).
The mass market
The rise of mass-market paperbacks represented an explosion of opportunities for authorship. Mass-market outlets were serviced by hyper-local distributors, hundreds of them across the USA. These small firms used unionized Teamsters to actually stuff the spinner-racks in dozens of non-bookstore retailers.
Being unionized, these book distributors had long-term job security and earned performance bonuses if the books they placed sold (unsold books were “stripped” of their covers which were returned for credit, with the rest of the book being pulped or snuck home by unscrupulous grocery store clerks, which meant that picking the wrong book didn’t just squander a potential sale — it actually cost money).
The combination of long-term experience and cash incentives turned many of these truck-drivers into shrewd predictors of what would sell and where. That’s how the science fiction fans of the sixties and seventies came to discover avant-garde, wildly transgressive “New Wave” sf novels in the spinner racks of their suburban drugstores.
The mass-market was far from perfect, but it was pluralistic: it spread out decision-making about which books were available to whom across a huge pool of people from a wide variety of backgrounds, beyond the bookstore chains and independent bookstores. When I was a baby writer, local science fiction conventions featured long-established pros passing on career advice about bringing boxes of donuts to the book warehouse in the morning as a bribe to the Teamsters to get your first novel into as many corner markets as possible.
Enter Sam Walton, twirling his mustache
But by 1990s, when I started selling stories and then books, that advice was long past its sell-by date. Nationally, groceries and drugstores had been transformed in a process that originated with Sam Walton, who took advantage of deregulation to expand Walmart nationwide. Walmart was followed by waves of copycats who used predatory pricing to drive local merchants out of business. Big-box stores became a fixture, and represented such an important part of the mass-market bookselling channel that they were able to restructure the entire market.
First among their demands was an end to regional distributing. A retailer with outlets in 50 states didn’t want to manage 50 distribution accounts (indeed, most distribution territories were citywide, or even neighborhood-by-neighborhood, so a national chain might need to open hundreds of distributor accounts to serve all its stores). Within the space of a few short years, the number of distributors nationwide fell from about 400 to fewer than ten, in an orgy of bankruptcies and mergers.
This had a profound effect on the mass-market. Decisions about which books would be sold where were no longer in the hands of thousands of drivers who knew their territories intimately through long experience — rather, they were centralized into the hands of a few buyers who used databases to track sales and make predictions about the most “efficient” titles to stock nationwide.
The number of titles for sale nationwide fell off a cliff, and woe betide an author whose book failed to meet sales targets. A single stumble could lead to the permanent exclusion of that author from a big box chain’s consideration. Without those big box stores, publishers could no longer profitably publish those authors. If you are a genre fan of a certain age, you’ll remember the wave of established writers who rebooted their careers by switching to pen-names in a bid to trick these all-powerful buyers into giving them another chance.
Monopolies beget monopolies
The effects of deregulation —the Reagan-initiated “consumer welfare” reconstruction of antitrust enforcement —weren’t confined to Walmart and other big box retailers. Bookstore chains devoured one another in a too-fast-to-follow blizzard that saw mall stores nationwide changing corporate ownership more often than they changed their window displays (Borders was bought by Kmart, merged with Waldenbooks, spun out again, renamed, merged with a toy retailer, expanded globally, merged with the UK chain Books etc, flipped to private equity, debt loaded and crushed; Barnes and Noble bought B. Dalton and Bookstop, went public, bought Gamestop — yes, that Gamestop — started haemorrhaging money, got flipped to private equity, and rebooted under new leadership with James Daunt at the helm).
These mergers weren’t just driven by deregulation, though: monopoly begets monopoly. With mass-market sales dominated by big-box retailers (who could use predatory pricing to discount books below their wholesale prices), booksellers’ share of mass-market revenues collapsed. Getting big enough to negotiate preferential terms from publishers — in the form of “co-op” payments to promote blockbuster titles, as well as sweetheart discounts, “incentives,” and favorable credit terms — was one way for bookselling to compete in the new market.
This was bad news for publishers, of course (it was even worse news for independent booksellers, who not only couldn’t get the favorable terms extorted by big box retailers and the Big Two national bookstore chains, but actually saw their terms get worse in this period, as publishers took their gains where they could get them).
Consolidation in both trade and mass-market retail and distributorship was met with consolidation in publishing. Publishing went from dozens of publishers to a handful that continues to dwindle to this day: the Big Six publishers of the 2010 are now the Big Four, thanks to Penguin-Random House’s acquisition of Simon & Schuster (the full name of this monstrosity is properly “Viking-Putnam-Berkeley-Avery-Ace-Avon-Grosset & Dunlop-Playboy Press-New American Library-Dutton-Jove-Dial-Warne-Ladybird-Pelican-Hamish Hamilton-Tarcher-Bantam-Doubleday-Dell-Knopf-Harold Shaw-Multnomah-Pocket-Esquire-Allyn & Bacon-Quercus-Fearon-Janus-Penguin-Random House-Simon & Schuster”).
Health-care-ificatoin of publishing
Meanwhile, both trade and mass-market retail were collapsing even further, as Amazon claimed the lion’s share of both markets, giving the final say over what books were connected with which readers to a single firm, whose executives used a mix of algorithms, superstition, vindictiveness and raw, anticompetitive bullying to determine which books would succeed or fail.
This is a familiar pattern. As David Dayen documents in his 2020 must-read book Monopolized, it’s a microcosm for what happened in many industries, notably health-care. Consolidation in the pharmaceutical industry led to a wave of price-gouging directed against hospitals, which underwent a rapid series of mergers that produced regional hospital monopolies that were able to negotiate lower drug prices. Hospital monopolies, in turn, put the screws to health insurers, spiking prices for routine procedures and supplies. Health insurers collapsed into a handful of companies, often dividing into territorial monopolies, and negotiated cheaper rates for hospital services.
Not only do the structural changes to the healthcare supply-chain parallel the changes in publishing, but so do the consequences: the climax of the orgy of consolidation is a coordinated attack on the people who can’t consolidate: workers and users of the system.
Consolidated health-care systems (insurance, pharma, hospitals, pharma benefit managers, labs, etc) are able to raise prices, lower the quality of care, and suppress wages of health-care workers while degrading their work environments with bureaucratic electronic health record boondoggles and horrific PPE shortages during a pandemic.
Likewise, a major impact of publishing consolidation is pressure on its workforce (many of whom have been moved from salaried positions with benefits and pensions to precarious contract work) and its supply-chain — that is, writers.
Shit flows downhill
As competition in publishing has faded, the deal for most writers has gotten worse. We’ve seen a rise in odious contracting terms, from binding arbitration waivers; to “joint accounting” that allows publishers to drain money owed for a successful book’s sales to cover the failure of another book; to non-negotiable inclusion of ebook, foreign, graphic novel and audio rights (with no increase in advances); to the abolishment of rights-reversion for out-of-print books. Advances and royalty rates have stalled.
Books are doing fine, authors (and publishing workers) are not — just as health insurers, hospitals and pharma companies are thriving, while patients and medical workers’ fortunes are growing steadily worse.
The decline in the author’s share of the pie is directly attributable to a decline in competition among publishers (which, in turn, is directly attributable to a decline in competition among retailers and distributors). In a world with four large publishers, if Publisher A passes on your book or makes a unacceptably low offer, you try your luck with Publishers B, C, and D, and, if four decision-makers all make no offer or a poor offer, you’re done.
This situation is bad for writers, readers and publishing workers (the same dynamic plays out for publishing workers — if you don’t get a job at Macmillan, Harpercollins, Hachette or Random Penguin, then you have been rejected by every major publisher). Many things have been tried to fix this system.
The myth of the benevolent giant
First came the search for an alternative to publishing itself (just as the mass-market was an alternative to trade publishing). Writers, readers and editorial workers sought out other sectors who’d get the books they wanted into the hands of readers. There were a lot of startups that tried to fill this niche, but apart from some religious publishers, the only one that attained liftoff was Amazon, which leveraged its dominance in every other area of publishing to create a successful alternative to the publishing industry itself.
But while Amazon produced some high-profile wins for indie writers, and a port of call for editorial workers shed by the major publishers in post-merger layoffs, the honeymoon was destined to be short and end bitterly.
After all, the reason companies in concentrated industries treat their customers, workers and suppliers badly is because they can. Google doesn’t shower its tech workers with stock options, free kombucha and massages because of its generous spirit —if the company was a champion of labor rights, then these same perks would extend to its low-waged workers, too. The fact that these workers are misclassified as independent contractors and paid through a staffing agency cutout reveals the true predictor of how Google will treat you: how hard you are to replace.
When the options for writers dwindled — as publishing concentrated into fewer hands — the treatment for the writers who defected to Amazon also declined. They became replaceable. Right on schedule, the company embarked on a program of wage theft that stole tens of millions of dollars from the indie authors who’d shackled themselves to Amazon’s platform.
The only way that the mass of disorganized readers, workers and suppliers of an industry can get a fair deal is for the industry itself to be disorganized — to consist of competing firms that have something to lose if we walk out of the door. Trading one monopolist for the other in the hopes that it would look more kindly upon us was doomed from the start.
The writer-friendly mid-tier
In parallel with this effort to pit one giant against another, many publishing workers embarked upon upon a very different project: to fill the gap between self-publishing and the Big Six^H^H^H Five^H^H^H^H Four publishers with “boutique” publishers staffed by publishing veterans and bright young publishing workers, scooping up the fantastic authors who had been turned away from the big publishers.
This project has been much more successful than the giant-seeking expedition that ended with Amazon devouring the writers who’d helped it build its indie platform.
Today’s publishing landscape boasts a very exciting mid-tier of publishers, some organized as nonprofits, others as commercial entities. Several of them — Canada’s Raincoast, the UK’s Bloomsbury — have been able to grow to regional juggernauts thanks to their willingness to publish JK Rowling’s first Harry Potter novel after it was rejected by major publishers.
These scrappy publishers are doing marvellous things, and the internet has allowed them to connect with audiences in unique ways, while a new tier of distributors have cropped up to serve as a bridge between the mid-tier and the nation’s struggling, tenacious, and utterly vital indie bookstores.
I’ve dealt with several of these mid-sized publishers and I can report that they are great to work with. To be fair, they can screw things up just as much as the big publishers (whom I also work with) do, and it’s true that from a writer’s perspective it doesn’t matter if something bad happens to your book because a giant publisher’s bureaucracy failed it or because a small publisher didn’t have the staff or ready cash to capitalize on an opportunity. But they can also score wins with books that the big publishers can’t or won’t figure out how to get into readers’ hands, and they represent a competitor and hedge against further intensification of the Big Four’s squeeze on writers.
The endangered mid-tier
But the existence of this thriving mid-tier doesn’t change the overall dynamics of publishing. The squeeze on workers and writers isn’t just the result of an industry dominated by four publishers — it’s also the consequence of an industry with one major distributor (Ingram), one major brick and mortar bookstore (B&N) and one major online bookseller (Amazon, whose audiobook dominance through Audible is even more extensive than its dominance of print and e-books).
The one advantage that the Big Four publishers have that the mid-tier of publishing does not is that they are big enough to push back (with limited effectiveness) against Amazon and Ingram’s worst practices. Lacking this might-checking-might, the mid-tier is in a precarious place indeed.
Amazon, recall, is the company that once created a “Gazelle Project,” to “approach the small publishers the way a cheetah would pursue sickly gazelle,” to get them to accept unfavorable terms for their books in the Kindle store. That was in 2004. Does anyone think the company has gotten less willing to play hardball since then?
Fewer people are attuned to the risk of all of distribution consolidating into one company, Ingram, but this should alarm anyone who cares about publishing. Distributors are incredibly powerful, and anyone who distributes on behalf of third parties has numerous opportunities to engage in funny accounting practices whereby they cream off a sneaky share of their own.
Advanced Marketing Services (an American horror story)
To see how wrong that can go, cast your mind back to the tale of Advanced Marketing Services, a one-time juggernaut in book distributors — when the big box retailers demanded consolidated distribution nationwide, AMS was the company that stepped in to serve that market, gobbling up regional companies. AMS had a grotesque “accounting scandal” (a polite way to describe ripping off its publishers) that led to the ouster of most of its top management in the early 2000s. Despite this, it was massively overcapitalized and it continued to buy up the mid-tier trade distributors, companies like Publishers Group West, which distributed virtually every mid-tier publisher in New York.
Then, in 2007, AMS declared surprise bankruptcy at the start of a holiday weekend. At least a dozen New York “boutique” publishers were bankrupt by the end of the next week.
That’s because publishers are “unsecured creditors” to their distributors — when a distributor goes bust, it gets to keep the books that have been shipped to it, even though it hasn’t paid for them. Then it gets to sell those books to the trade, at steep discounts, with the money going to its bankers and other secure creditors. Publishers don’t just lose their books (but still have to pay their printers) — they now have to compete with copies of their books being sold below cost into the same retail channels they serve.
The PGW bankruptcy allowed a couple of larger companies with access to capital— Perseus Books and Avalon Publishing Group — to buy up all those distressed publishers at bargain-basement prices, with ensuing waves of layoffs and orphaned books (I had a PGW-distributed book published during that week and was assigned three editors over the next month, as their publishers were rolled up into successively larger publishers and each of them was laid off, leaving me with an orphaned title).
Perseus and Avalon were subsequently rolled up by a private equity firm, then split out and sold to Hachette, one of the Big Four publishers, and Ingram, the only major distributor still extant in America.
I love mid-tier publishers and I am their biggest cheerleader, but they are small animals scampering between the legs of charging rhinos, and as nimble as they are, they are not safe.
Self publishing
With all this, you may be tempted to self-publish. After all, the mechanics of self-publishing have never been simpler or more extenisve. Lulu will print beautiful books onshore, quickly, and cheaply (I ran up some “author’s galleys” for a couple of my upcoming books to use in soliciting blurbs and feedback and discovered to my delight that I could print a 6 inch x 9 inch finished, perfect-bound book with a full-color cover for less than I pay to photocopy and side-staple a manuscript at my local copy shop!). Smashwords and Bookbaby offer extensive author services and ebook distribution.
And, of course, Amazon will take your book for the Kindle store, and Ingram will accept it as a print-on-demand title available to every bookstore in the country (if you can’t beat ’em, join ’em).
I often hear from friends and strangers who want to know what I think of self-publishing. Here’s what I tell them:
Publishing a professional-quality volume has never been easier. Working with publishing platforms, you can contract with excellent proofers, copyeditors, cover artists, book designers, typographers — the whole stack of professional services that go into making a book into a finished product (many of them are publishing veterans, still using their skills after merger-based layoffs). You can hire as many or as few of these professionals as you need, based on the skills you bring to the table.
But that still leaves you with a serious problem — perhaps the most serious problem in publishing. All of that stuff is writing and printing, but until the book finds its readers, it is not publishing.
As my beloved novel editor Patrick Nielsen Hayden told me long, long ago, “Publishing is the process of identifying a work and its audience, and then taking whatever steps are necessary to connect the two.” That may include cover design or marketing copy or copy-editing, but it also includes the huge, ill-defined, and nebulous world of marketing, sales, and publicity.
Put simply, you need to figure out why anyone, anywhere should give a shit that you wrote a book. This is a very hard problem. Indeed, it’s the hard problem of religion, advertising and politics: getting someone else to care about something you want them to care about.
Not only is this a hard problem, it never gets easier. The attention wars are a battle between novelty and adaptation — anything that gets your attention today will blend into the background with repetition (your fridge’s compressor is unbearably loud when it switches on, and five minutes later you stop hearing it, and won’t notice it again until it stops). The one thing that definitely won’t work today is the thing that worked yesterday.
Here’s where publishers have an advantage: they have a longitudinal view of how books and audiences find one another. They publish lots of books. They try variations on their marketing, sales and publicity with each book, see which tactics show the most promise, and refine them. They can iterate.
That’s the single largest disadvantage faced by self-publishers. You go into your marketing and publicity plan without any precedents to have learned hard lessons from. You are a data-set of one.
The method
Some people do very well with self-publishing, of course. Sometimes that’s down to luck, but after years of peripheral experience with self-publishing and a few successful direct experiments, I’ve evolved a checklist for would-be self-publishers that makes success more than a matter of pure luck.
- Observe the publishing fortunes of books whose audiences you imagine to be similar to your book’s audience;
- From these observations, formulate a falsifiable hypothesis about how you will reach a similar audience;
- Based on this hypothesis, formulate a plan to get your book to that audience;
- Execute your plan, and measure its progress by comparing your book’s performance to your hypothesized performance;
- As new data comes in about where your hypothesis was mistaken, revise your hypothesis and make a new plan, and execute that;
- Go to step 4. and repeat.
This won’t guarantee that you succeed, but without something like this, you will almost certainly fail.
The corollary of this method is simple: unless you feel you can figure out how to market your book, unless you want to devote as much energy to that marketing plan as you did to its authorship and production, unless you are prepared to sustain your marketing effort through constant iteration and refinement, you probably shouldn’t self-publish.
The mass market (again)
Generations ago, publishing was upended by the creation of the mass-market. Selling books to people who didn’t identify as readers — people who didn’t go into bookstores, but might buy a book from a pharmacy or grocery spinner-rack —surfaced entire new literatures and audiences that had no home in the world of bookselling. Moreover, it moved those literatures and audiences into the bookstore, creating a funnel for the strange world of self-identification as a “reader.”
Publishing is — by definition — very good at targeting readers, but when it comes to targeting the vastly larger world of non-readers, publishing’s expertise is far patchier. After all, understanding “non-readers” involves understanding the whole world, the motivations not just of people who do buy books, but people who don’t.
Mega-bestsellers are just books that a small proportion of non-readers read. “Airport novels,” books that Oprah pitches, books that get made into movies — these are all books that are exposed to groups of non-readers that are orders of magnitude larger than the people who consider themselves “readers.” And they’re funnels: the Harry Potter novels and 50 Shades of Grey both introduced vast numbers of non-readers to books, and induced a small proportion of those non-readers to become readers.
This is where self-publishing has a potential advantage relative to publishing. You may be in touch with a group of non-readers — a faith group, members of a subculture or fandom, a professional association or a political movement — that you have well-developed ideas for reaching and convincing to give a shit about your book. It’s entirely possible that you are the first person who’s ever considered the potential pathway to engaging that group of people, that is, you might be the world’s leading expert on the subject.
In that instance, a publisher brings a lot less to the table. What’s more, if you are successful in selling to that non-reader group, you can use that accomplishment as leverage in negotiating a deal with a publisher to introduce your book to readers, something they almost certainly know more about than you do.
Reply hazy, try again
This is an odd, unsettled moment in our cultural history. Audiences have self-organized in online communities and forums that are more addressable than at any time in history, and mid-sized publishers have access to more talent — both in terms of staff and writers — than at any time in history. The big publishers have global networks that reach further than any publisher in history. Indie bookstores experienced a pre-pandemic resurgence and indie-friendly electronic distributors like Libro.fm and Bookshop.org gave them a lifeline during lockdown. Barnes and Noble is under management of a man who founded the UK’s best indie bookstore chain and then rescued its last national bookstore chain.
But there’s still just one brick-and-mortar chain, one dominant electronic bookstore, four major publishers, and one major distributor. The mass market is dominated by four or five big box retailers.
To top it all off, there’s a surging new antitrust movement that wants to tear apart the cross-industry monopolies and force them to compete.
I love the way my Big Four publisher handles my novels, and I love the way my nonprofit mid-sized publisher is handling my next nonfiction book, and I love the way my odd little experimental pamphlet from Medium is performing, finding audiences and paying handsomely besides. I still get a steady stream of payments from my self-publishing experience, and my self-publishing Kickstarter last year literally broke records.
I know I’m lucky. Yes, I work hard, but every writer I know works hard. More than anything, I’m keenly aware that, despite my success with large publishers, mid-tier publishers and self-publishing, the system is precarious and my future in it is anything but assured.
A viable future for publishing — like a viable future for online services, internet delivery, health care and every other industry — depends on our ability to restructure our markets to serve the public good, rather than monopolists’ shareholders. Until then, I’ll be right here with the rest of you, observing the market, formulating hypotheses and turning them into plans.
And crossing my fingers.