2021-09-06 My experience with book writing

This is a rather atypical post in that it is just a bunch of thoughts about how to write a book, how not to write a book, how I’ve been writing my book on Emacs Lisp, what was easy, what was difficult etc. If you want to write a book, it might be useful to learn where a fellow author struggled – note however, that this post reflects my personal experiences and your situation, personality, experience, work style etc. may be similar or completely different.

Also, this post is quite long, and – let’s say – not extremely structured. It is just a collection of thoughts, some of which might be useful, entertaining or interesting to someone – or not. You have been warned;-).

So, let’s start with the things that turned out to surprise me, either positively or negatively. First of all, I was completely unprepared to the reactions of other people. Since I decided to use Leanpub, which is a platform to sell books-in-progress, I expected (hoped) a few people would pay me, and I expected to gather some early feedback which might help me with the process. Both expectations were wrong – the number of people who trusted me with their actual money was much higher than I thought (which was a tremendous boost to my motivation!), and the number of people who provided valuable feedback was close to zero (well, I got some encouraging emails etc., but almost no concrete suggestions or signals like “I like this part, but I don’t like that part” or anything similar). This meant that I didn’t really know anything about the reception (apart from the fact that quite a few people paid me and didn’t want their money back, so I do hope that I did a few things right!), and thus the fact that I published the book incrementally didn’t help me as much as I hoped for. (Still, let me emphasize that I consider it a good decision, since even an indirect feedback in the form of paying me actual money helped me. As any writer surely knows, an “I’m unworthy of anything and what I write is essentially garbage” feeling likes to strike from time to time, and being able to counter this with “but there are people who think otherwise, or else they would want their money back!” is really valuable!) Let me also mention that this is my first book that I was not really sure is “wanted” by anyone. The first one, which is a textbook on hyperconvex metric spaces, and the third one, which I think I’m not yet allowed to talk about a lot, were commissioned by a publisher, so I had the comfort of not caring if anybody would actually buy it. The second one I wrote (with a friend) filled an obvious and glaring gap in the market (no free and modern textbook on LaTeX in Polish except the translation of the Not so short introduction, which has a different goal).

Another thing that took me by surprise (and it really shouldn’t!) was how often I needed to go back and change things I’ve already written. I am not talking editing changes, like fixing typos or awkward sentences – I’ve been an editor for over a decade and I think I know how that works – but the overall structure. It turned out that all three books I (co)authored earlier were a bit different in that regard. The first one basically wrote itself – the book grew naturally from my master’s and Ph.D. theses, and while it is not a direct translation (even if many a proof in it is), it was basically the third large text I wrote on this subject, so I had quite a clear vision of what to present in what order etc. The second one was basically a set of LaTeX examples, so it was very easy to structure (and wouldn’t even suffer much if we had shuffled the chapters or sections, within reason of course). As for the third one, I think I can say this: it is going to have three names on the cover, and while we had many, many discussions about the structure, my personal contributions were rather on a smaller scale of individual chapters, sections or even paragraphs, not thinking up the overarching structure of the whole book, which was more of someone else’s responsibility.

Coming back – I knew that I would need to rewrite things or add them to earlier parts, but I thought that I had a more or less clear vision of the book and that changing what I already wrote wouldn’t be necessary so often. It probably shouldn’t have surprised me so much, but it still did. Of course, it is very good for the book’s quality (or so I hope) – many such changes are just finding opportunities to explain more things.

Let me now say a bit about how I approached writing when I first started, and what experience taught me later. At first, I had a very vague idea about the structure of the book, and I just thought “well, I’ll definitely need a chapter on this, and on this, and on that”, and wrote large portions of text devoted to those particular ideas. It turned out to be a very bad approach – it left me with a bunch of “chapters”, no way of joining them and still no idea about the overall structure. Then I abandoned the project for several years (because life intervened, and also that another book took much more work and time than I initially expected). When I finally came back, I was a bit more humble and quite a bit more experienced. I realized that Elisp is a huge thing, and so I gave up any hope for any kind of “completeness” – some (in fact, many) areas would have to be left out unless I wanted to rewrite the whole Elisp reference (only much, much bigger – with more explanations, examples etc.), which I obviously didn’t. This was liberating – I decided to write several chapters (and by “several” I mean rather 3–5 and not, say, 12–15), each describing some piece of code from its first, draftish lines to a more or less complete state, explaining the concepts needed as I went. I am still not 100% sure, but I am pretty convinced that this was a very good decision. Perhaps most importantly, it allowed me to largely avoid the (in)famous “writer’s block” – since I had a pretty clear vision of what every “package” would do, I always knew (more or less) what to write next – just code the next feature needed and explain it. Also, it makes reading the book closer to the actual experience of someone learning on their own and researching just the portions of the language they need. On the other hand, it definitely makes the book more “chaotic” – but this is unavoidable when you explain things at the exact point when they are needed. I’d argue that this is actually a good thing, though – most Elisp features are pretty independent, for example, you don’t really need to understand destructive functions before knowing how to define minor modes (or the other way round, for that matter). When you know the language core (which is very well explained in the late Robert J. Chassell’s An introduction to programming in Emacs Lisp, which I can’t recommend enough!), you can just learn what you need in just about any order.

The drawbacks are obvious – some things will not be interestinguseful to some people at all/, and hence large parts of the book may be boring for them. Still, the book should be read in order, and skipping large parts may not work (because in later parts I just use the concepts introduced earlier, usually without many comments). I want to prepare a pretty detailed index at some later point (and not doing it step-by-step while writing was probably a mistake), which should at least partially alleviate the problem.

Another drawback is that one could argue that in an average (good) textbook, the main value is that the author made a selection of topics and prepared them carefully in a logical order (and also, in an increasing order of difficulty) – and hence my book has less value compared to just learning by yourself, since the order is not that logical. This may be true, although I still think (hope?) that reading it is better in some way than just studying the Elisp reference and actual Elisp code. The main strength, I believe, lies in the detailed explanations and examples (which are – unsurprisingly – rather scarce in the reference), in the fact that I try to introduce various concepts one by one and not all of them at the same time, but also in the fact that I skipped things which I think may be less important for someone starting their adventure with Elisp, and conversely, included things that I think should be learned at a pretty early stage. For instance, I included a whole set of tricks helping with finding information about existing Elisp functions doing useful things so that you don’t have to reinvent the wheel (and this is in the first chapter, since it is of utmost importance!). Another example is the fact the the book doesn’t even mention defining major modes, since this is not something you would need on a daily basis (in fact, I made a lot of customizations in my Emacs, but I almost never had a need to write a major mode, and when I did, it was always derived from special-mode). On the other hand, I spend some time showing how to define a minor mode (since I have a feeling that this is way more useful and needed way more often from the viewpoint of a user who just wants to customize their Emacs a bit).

One thing I guess is very important to keep in mind is how to make strategic decisions with a book. What to include and what to omit? How concise or how verbose should the explanations be? I know basically one way to answer these questions (obviously, I didn’t come up with it myself) – imagine a very concrete person you are writing for, and ask yourself what this person would need. And since imagining what a person different from yourself might need is next to impossible, I just decided to write a book I would like to read when I was starting to learn Elisp. I strongly suspect this is one of the best ways to write any textbook. (Another one is writing a book for someone you personally know very well, like your spouse or child. In other words, you don’t have to imagine your reader from scratch – you just take a preexisting one. Then, you may actually ask them questions. I haven’t tried that idea yet, but I do toy with the idea.)

A completely different issue with book writing is how to organize your work. Some things are obvious – you want to write it with a tool you know and like (I use Emacs and Org-mode with a Leanpub exporter written by Diego Zamboni, of course, though AUCTeX could also work if not for the format used by Leanpub), and you need to use version control (I use Git with Magit, of course). Some things are less obvious, though. One famous question is, do you write when you feel inspired or do you have a regular schedule? Well, I don’t believe in the “write-when-you-feel-like-it” thing at all. (In fact, our times are way too much about feeling than about thinking and doing. Emotions are important, but it is the reason and action that make it possible for us to achieve things, not just “feeling”.) So, I do have a writing schedule. I admit that it’s not extremely regular, but it works pretty well. First of all, I use (unsurprisingly) Beeminder to make sure that I spend at least 20 minutes per day on writing on average. It seems not very much, but 20 minutes per day goes a really, really long way over time – it’s more than a hundred hours over the course of a year, and you can write a pretty good book in a hundred hours. I usually write in short bursts of about 25 minutes (clearly inspired by the famous Pomodoro technique), and I usually tried to reserve two such 25-minute cycles in quick succession so that I don’t break my focus. This is very difficult, though, if you have kids, a day job and a few other side projects, so there’s room for improvement there. (Still, I think it’s better to spend 25 minutes on writing almost every day than two and a half hours once per week – I was afraid that if I wrote only once or maybe even twice a week, the breaks between the writing sessions would be too long and I would forget too much context. I don’t have any real experiments to back this sentiment up, though.)

Then, I have an overall weekly plan saying when exactly I spend time on writing. For instance, I usually had 2×25 minutes reserved for it every Monday from 5:15 to 6:15 and also every Wednesday from 6:45 to 7:45, etc. (I definitely prefer working in the morning than in the evening.) It’s pretty flexible in the sense that if a need arises, I change my plan (sometimes I have something urgent to do, so writing has to take a back seat that day) – but I don’t worry about a slippery slope, since Beeminder takes care of that (and if needed, I plan additional time outside the usual schedule). Now that vacation is over, my time for writing is going to shrink, but I still try to schedule it and do it regularly.

I am pretty lenient, however, in what I count as “time spend writing” – rebasing in Git (if necessary) counts, editing and moving stuff between sections counts, studying related manuals, email threads and code counts, even sitting with my eyes closed and thinking counts. I’ve just found out that I don’t want to cheat myself (nor my readers!), so I don’t have strict rules with this. If I was tending to procrastinate even more, I could set up some precise rules, or count e.g. words produced instead of time spent. This, however, is pretty risky, since sometimes I needed to spend a lot of time thinking how to put something in words only to decide that I should just delete it. This can also be a valuable portion of the writing work, even though the word count is zero (or even negative). In fact, counting time spent instead of words produced goes along with the general idea that it’s better to commit to certain amount of effort put in rather than to any amount of effect produced.

Finally, let me tell you what was the easiest and the most difficult part of writing my book. The easiest thing was (probably) just coding the stuff I wanted to show and explain. It should be no surprise that I like programming, and the topics I chose for the book were things that I personally found useful and interesting. This is also a common advice for people who want to learn programming: find some project you find interesting or useful to you personally and solve it using a computer. Of course, this means that the projects I coded in the book might be boring and useless for someone else – but that’s ok, I don’t see how I could avoid that. (Well, I tried a bit by selecting topics that are not tied to a particular problem, language, job etc.)

The most difficult thing (I think) was – also unsurprisingly – persisting in writing when attacked by doubts about the book. Does it even make sense to write it? Isn’t it too easy, too difficult, too boring, too niche, too concise, too verbose…? Would anyone find it worthwhile? I have to admit that using Leanpub (which enabled feedback, if only in the form of money paid) helped tremendously with that. (That, and also a firm commitment to myself that I really want to write it, even if it turns out that I’m not going to earn enough money with it to justify the whole idea financially.)

So, that’s it as far as my thoughts on book-writing go. Now let’s get back to work!

CategoryBlog, CategoryEnglish, CategoryEmacs, CategoryEmacsBook