A discussion of why I don’t intend to turn Gwern.net into a book, and how trying to write a book can harm writers.
As well as asking about doing podcasts, people sometimes ask me whether I am going to write a book or turn Gwern.net into a book. To their surprise, I have no such intention and remain a writer manqué. Writing a book can be harmful; for many writers, the opportunity cost & risk mean that writing a book is about the last thing they should do (unless they feel a book bursting out of them, much like a chestburster).
But how can I not want to write a book? And I get it: writing a book is sacred and unquestionable, the ultimate achievement for Western intellectuals—better than being arrested in a protest (because you don’t have to get sweaty), better than a PhD (because not so devalued), and better even than going to Harvard (because that mostly means you got lucky in admissions).
It’s something I’ve definitely aspired to since I became a bookworm: imagining joining the pantheon of authors shelved in my local library, to be able to hold my hardcover book in my hands (perhaps even one with… gilt-edged pages?), and carp about how ‘the publisher chose the cover’.
Bad Reasons For Books
I will be Chateaubriand1 or nothing!
Victor Hugo (age 14)
Yet, that’s the problem: I am in love with the idea of having published a book, but not with publishing a specific book. Like Douglas Adams, I want to have published a book and have the identity of ‘author’2, but not so much be writing a book—in the same way one might want to have ‘learned to read Mandarin’ or ‘become a bodybuilder’, but not to actually sweat through memorizing (and then forgetting) endless arbitrary characters or hours in the gym (followed by gluttony as cruel as the starvation).
One can dabble in such things as a hobby and perhaps learn Chinese to the level of a kindergartener or do one’s health good without much sacrifice, but to gain those accomplishments to the level that would excite the envy & admiration of one’s friends & acquaintances is quite another thing.
Costs of Books
There were shelves on shelves of books on every conceivable subject…The [late] owner had read them all, and many of them were full in their margins with cross-references in pencil. There pigeon-holed desks and cabinets with literally thousands of compartments..These volumes were apparently coming in at the rate of 10 or so per week…For years, apparently, he had been endeavoring to keep up with everything that was being written—a Sisyphean task. Over all there were brown Holland sheets, a thin coating of dust, the moths dancing in the pale September sun. There was a faint aroma of mustiness, proceeding from thousands of 17th & 18th-century books in a room that had been locked up since the owner’s death. I never saw a sight that more impressed on me the vanity of human life and learning.
Sir Charles Oman, On The Writing Of History 1939
It is easy for someone like a book agent or editor to suggest “hey, you should publish a book!”, since they aren’t the one writing it, and (like a venture capitalist) they have plenty on their plate if you fail. But what book, exactly, would I publish? “Gwern.net”? Leaving aside all the fun hypertext features, what would such a book be? It would have to be some sort of anthology, I suppose; but what a mongrel of an anthology it would be, if I look over my past few essays: a review of a children’s movie about unicorns, then the moral philosophy of self-consistent time travel, then there’s the detailed technical tutorial about generating fonts with 2023 generative AI models… (The anthology would not be ordered chronologically, safe to say.)
I don’t feel enthused about it; but to make a book I would not be ashamed of, I would have to spend many hours revising, rewriting, stripping out hypertext features and deciding what should stay, reformatting for some arcane publisher workflow like Microsoft Word or EPUB (or worse).3 This is true whether it’s a big professional publication or a small boutique style, like the LessWrong anthologies—which are nice indeed, but I recall Oliver Habryka ballparking the effort at hundreds of man-hours. (For already-written LW posts!) And while I was in the middle of that, if I wanted to write something else, well, now I would get to feel extra-guilty about procrastinating on The Book™. I definitely won’t have time or energy to follow up on weird anomalies or interesting new rabbit-holes, and even if my output appears unaffected by The Book™, I am in reality eating my intellectual seed-corn and gradually falling out of date, as I neglect the ordinary work of reading & discussion that may flower years later. (Authors who write books have a distinct tendency to ‘go stale’.)
I finish all this, and then what? Now it has to spend months working its way through the pipeline, still a distraction, and then—mirabile dictu!—it is done.
Uh oh—that’s just the start, isn’t it? Now I have to go out and shill for pre-orders (as we are endlessly assured by apologetic authors that pre-orders are about the only sales number that matters these days and oh god if you ever liked my writing please pre-order and don’t wait until afterwards just think of it as a donation if you have to but pre-order now). If you opt for a low-key launch, skipping the pre-publication publicity campaign, well, you’re still going to have to publicize it once it’s out (free copy to the first person to retweet this!), and deal with reviews (soliciting them, specifically—you should be so lucky anyone actually wrote a review, good or bad, unsolicited) etc.
And of course, the double-edged blade of the physical immutability of books as artifacts begins to cut both ways—now that your thoughts are committed to a book, you can never take them back or change them or tweak them. You are now in some respects frozen and reduced to a caricature, to a much greater degree watching your caricature self go around and about (“Borges and I”). Francis Fukuyama, for example, became famous for his ‘end of history’ essay/book; the result was to be mostly insulted & totally misunderstood (despite being correct about the failure of alternative ideologies & rise of identity politics as a substitute), and his better work (eg. 2014 or The Origins of Political Order) overshadowed.
The larger your audience, the simpler your writing must become, because it is read by more dumb, ignorant, or hostile readers, and things like sarcasm or understatement or deliberate errors become things you can’t countersignal—they will simply infer that you are as dumb, ignorant, or evil as they assume (and indeed, many of those readers will be malicious and simply reading for things they can weaponize).
Publishing can be a trap for the unwise. We build our lives like we guess our whys: glimpses in the eyes of passerbys. If you decide to ride the tiger, you find yourself increasingly the creature of those you sought to use, called on to deliver more of what made you popular (regardless of the personal cost), slave to the audience and the social media algorithm, always demanding something now, at the cost of the future; and fame, one may find, is not all that it’s cracked up to be, as inevitably the marginal reader drawn by things like prizes is less keen on your writings than the firstcomers, and there are a lot of drawbacks to fame (like all the crazies).
So, a book is a lot of work for a writer, even if it is mostly already-written writing, which crowds out new writing or exploration, and which tends to freeze them in place. But it gets worse.
A book commits you to a single task, one which will devour your time for years to come, cutting you off from readers and from opportunity; in the time that you are laboring over the book, which usually you can’t talk much about with readers or enjoy the feedback, you may be driving yourself into depression.
…Lundkvist later said he thought the award [for Aniara] brought the authors to a too early death…According to academy member Lars Gyllensten, both authors were badly affected by the negativity and criticism following the award.
The sensitive Martinson found it hard to cope with the criticism following his award, and died on 11 February 197846ya…after cutting his stomach open with a pair of scissors in what has been described as a “hara-kiri-like manner”.
Wikipedia
Case Studies
It is a laborious madness and an impoverishing one, the madness of composing vast books—setting out in 500 pages an idea that can be perfectly related orally in 5 minutes. The better way to go about it is to pretend that these books already exist, and offer a summary, a commentary on them.
Jorge Luis Borges, Fictions forward (194480ya)
PhD attrition (and mental illness) rates are well-known, and much of it happens after qualifications, while writing the thesis; those who pay attention to biographies of writers will recall many more instances attested, if only in those odd lacuna, where biographers dispatch entire years with a line or two about a discarded manuscript.
To these, I’ll add two case-studies that I sometimes think about.
Wait But Why?
The long-form essayist/blogger Tim Urban of “Wait But Why” (WBW) made a name for himself in the mid-2010s by writing long enthusiastic essays, leavened with crude but effective comics, about various rabbit-hole topics like cryonics, the rise of deep learning, willpower/procrastination etc; at his peak he became a favored mouthpiece of Elon Musk. He was often compared to Scott Alexander (then blogging at Slate Star Codex); Scott Alexander has gone from strength to strength and is more widely read than ever (at Astral Codex Ten). WBW… not so much. The reader will have one of two reactions: “who’s that?” and “oh yeah, whatever happened to him?”
The answer is simple: on 2016-06-18, disgusted by the 2016 US Presidential election and American political trends (especially the Great Awokening), as he explains in his post-mortem, he decided to begin writing what would be a book about politics, What’s Our Problem?: A Self-Help Book for Societies (2023). This was a mistake.
Re-emerging over half a decade later (or 2,440 days) to break his silence, he explains at length how it was a mistake: the scope kept sprawling, he kept thinking he was almost done only to redo more, and sunk cost fallacy kept him from killing it, while he kept procrastinating and trying to do it all, to the extent that:
People in my life were worried about me. They tried encouraging me, shaming me, setting deadlines for me, reminding me that one post really shouldn’t take multiple years. Nothing seemed to help.
|
Year |
n |
|---|---|
|
2014 |
15 |
|
2015 |
11 |
|
2016 |
13 |
|
2017 |
1 |
|
2018 |
1 |
|
2020 |
4 |
|
2021 |
1 |
|
2023 |
2 |
|
2024 |
2 |
Eventually, his girlfriend, then fiancee, then wife, got pregnant. The thought of having a kid while still writing the book terrified him into finishing it off before the pregnancy did.
Was the book any good? I couldn’t tell you. I’ve never read it or seen anyone mention it; when I brought it up in a chat, only one person had read it (and they didn’t like it4). I suspect many more people read his post about writing the book than will ever read the book.
And one can see the impact looking at his archives. You do not need advanced statistics to see a change ~2016–2017 in the posting rate. (And this is neglecting that the later WBW posts are often short; counting words & illustrations would probably show a larger loss.)
Another way to put the opportunity cost is to note that if WBW had kept up its lowest pre-book annual rate (2015’s 11 posts), then instead of a total of 11 posts to show for 2017–72024 (ie. less than an average year), it would have 55 more (66 total) posts.5
Anonymous
In another case, the author has not written a post-mortem, but I think the same thing happened.
Around roughly the same time as WBW, there was another popular Internet blogger and writer, who gave well-received talks; he too decided to write a book to sum it all up and popularize his views, and went silent.
About 2–3 years later, I went to dinner with him; I tried to avoid bringing up the book at all, and felt I could sense his constant strain and anxiety that I would ask about it, which made conversation a little difficult. This depression was also clearly visible in their minimal online activity. I wondered if they would ever publish the book, or if the moment had already passed.
And then 4 years after that, somewhat to my surprise, the book finally came out—and it was good! It was positively reviewed and I still see it mentioned (which is more than >99% of books can say), and people still ask occasionally “whatever happened to him?” (If you google his name, “what happened to X” is the first suggested search query.)
So, the reward was real but modest. What was the cost? Unfortunately, he has not returned to his previous activities; so in his case, the opportunity cost appears to have been… everything. Everything he would’ve written if he hadn’t been writing the book. As far as I can tell, several years later, he has not begun writing another book nor taken up any major non-writing activities; he just exists—burned out by the trauma of writing the book. And you can’t fix burnout by doing the same kind of thing, I think, only totally different things.
This is regrettable.
As a third case, one could mention Scott Alexander again—at the same time as WBW he was writing, and finishing (on 2017-05-17), his online fantasy novel Unsong, which I enjoyed. He has since been editing and polishing it up for publication, according to occasional mentions on SSC and later on ACX up to the present. After 7 years of ‘editing’ and negotiations for publication, during which “various things went wrong and I procrastinated”, a fan volunteered to help get it self-published on Amazon. It’s nice that a book now exists, technically, but it feels like the mountain gave birth to a mouse, and one wonders if Scott Alexander would’ve been even more prolific 2017–72024 had he not had Unsong hanging from his neck…?
And a fourth case would be The Last Psychiatrist: he stopped blogging in May 2014, a book having been mentioned; which arrived only in December 2020, almost 7 years later. Reviews were mixed (eg.), but one thing is clear—as of September 2024, 4 years later, he has not returned to any known public writing.
Also worth mentioning is Andrew Hussie’s Homestuck graphic novel series, which took a year-long break (the “gigapause”) at its height in 201311ya to work on a videogame whose first installment would not come out for several years to mixed reviews (and accusations of mismanagement & diversion of the crowdsourced funds), and is apparently not yet finished as of November 2024 ; Hussie finished Homestuck proper, but has since abandoned the franchise (and does not appear to have been all that productive since, nor have any of his projects enjoyed even a tiny fraction of Homestuck’s success).
Wildbow’s Worm, supposedly undergoing intense editing—admittedly, quite necessary in Worm’s case—since 201311ya for ebook & print publication. Last I knew, it was still being edited, with no sign of Worm publication imminent; although meanwhile Wildbow has written at least 3 other web serials, of vast length & at similar update speed, so it is unclear if the ‘editing’ has done him any harm.
Keeping The Book Inside Them
Now of course, there is a second kind of author: authors who write books with little discernible impact on their newsletter or blog. They sit down for an hour (statistically, in the morning), pound out their quota, and turn to the next task (sometimes a dayjob, sometimes a previous book later in ‘the pipeline’, sometimes just leisure), like an Anthony Trollope. The book does not weigh on their minds, and they are confident that they will have it out, as usual, in 12 months. More power to them—but I also think that most would-be book authors already know if they are one of that latter kind of writers, or if they are the former kind I’ve been discussing. (I am not such a writer; I rely on divine inspiration.)
If they are the former kind, then they should seriously doubt any plan to write a book: why do they think they will finish? Or that it will be worthwhile? Is there a clear professional reason that the writing has to be a book? (Unless they are named “J. K. Rowling”, they can usually rule out “sales” as an adequate professional reason.)
More importantly, what will the effect on them be? And what is the opportunity cost of writing a book: if they spend the next 5 years devoting all their energies to a book, what will be excluded? Would they happily throw everything they wrote in the previous 5 years into a fireplace in exchange for a finished manuscript of the book they are fantasizing about? Would they willing to wager everything they might ever write on the book going well, rather than becoming an albatross and burning them out? If I had decided in 2019 to write up “a book about AI” in the wake of my GPT-2 experiments, say, and only published it in 2024, and sacrificed writing like “The Scaling Hypothesis” (visionary in June 2020, hopelessly obsolete in August 2024), would I be better off than I am now?6
I suspect that if many writers took seriously writing a book as a real project, rather than an aspirational cultural ideal of ‘being an author’ along the lines of ‘being a PhD’ or ‘creating a video game’, they would realize that it is a serious risk which can backfire. Everyone may have a book inside them, but some should keep it inside them.
And we should not put book-writing up on a pedestal, but acknowledge it is often a bad idea and that good writers should not be guilt-tripped into writing a book—you shall not crucify my kind on a cross of gilt!
And for now, should a book come out of me despite my best efforts, I will take off and nuke it from orbit. (It’s the only way to be sure.)
External Links
-
A once-universally-read French writer, now principally remembered for the beef dish named after him.↩︎
-
Tanner Greer describes this mentality in characterizing Washington DC elites:
↩︎In Washington people never read books—they just write them…Washington intellectuals are masters of small mountains. Some of their peaks are more difficult to summit than others. Many smaller slopes are nonetheless jagged and foreboding; climbing these are a mark of true intellectual achievement. But whether the way is smoothly paved or roughly made, the destinations are the same: small heights, little occupied. Those who reach these heights can rest secure. Out of humanity’s many billions there are only a handful of individuals who know their chosen domain as well as they do. They have mastered their mountain: they know its every crag, they have walked its every gully. But it is a small mountain. At its summit their field of view is limited to the narrow range of their own expertise.
The social function of such a book is entirely unrelated to its erudition, elegance, or analytical clarity. It is only partially related to the actual ideas or policy recommendations inside it. In this world of small mountains, books and reports are a sort of proof, a sign of achievement that can be seen by climbers of other peaks. An author has mastered her mountain. The wonk thirsts for authority: once she has written a book, other wonks will give it to her.
-
Gwern.net is licensed under CC-0 and anyone could make a book out of it if they wanted to. But if I the author am not enthusiastic enough to do it, why would anyone else?↩︎
-
Scott Alexander reviewed it but in a paywalled post, so I don’t know if he liked it or not.↩︎
-
This is a good place to note the “equal-odds rule”: that for creative outputs under normal conditions, each output has a surprisingly equal chance of being a hit (without a clear quantity-quality tradeoff); so each of those 55 foregone WBW posts could have been his biggest hit yet. But they were all sacrificed for a single project, and one without particularly high odds of success.↩︎
-
This is not a completely hypothetical example. Consider NYT AI journalist Cade Metz’s book The Genius Makers, written mostly 2017–32020, and published 2021-03-16.
It was obsolete on publication, and its main interest now is “how could Metz have missed the DL scaling revolution so completely, even while interviewing so many of the principals as it was happening?” (The short answer seems to be a curious lack of curiosity & pursuit of a “Narrative” meant he was committed to the narrative of the book and couldn’t throw it out & rewrite from scratch as necessary to regain relevancy.)↩︎