2025-11-17 Showing size of Org mode subtrees
I’ve been using Org mode for way over 10 years now, and some of my Org files grew to a considerable size. My largest Org files are over 3 megabytes, which means that working with them tends to be a bit slow. I’d like to be able to check which parts of the Org file are the largest and which are smaller. With files on the disk, the classical tool to achieve this is du, along with its interactive counterparts like ncdu or the insanely fast gdu.
I hoped for the existence of some kind of org-du which would do a similar thing with Org files. In fact, a tool with that name (written by the famous Karl Voit!) does exist, but it has three disadvantages which pretty much disqualify it for me: it counts lines instead of bytes, it is not working inside Emacs, and it is written in Python.
Let’s write an org-du command which would work a bit like org-clock-display, that is, it would show the size of each subtree next to it using overlays.
Now, there are two problems to solve here. One is to analyze the code of org-clock-display to learn how it performs creating overlays. The other is to perform the actual computations.
The first one seemed a bit tricky at first, but it boiled down to looking at org-clock-put-overlay, copying its code and changing it a bit. I decided to reuse the org-clock-overlays variable in spite of its name not really matching what my code will do. The reason is that org-ctrl-c-ctrl-c at one point checks if org-clock-overlays is bound and non-nil, and if yes, it runs org-clock-remove-overlays. Probably a slightly cleaner way would be to introduce another variable to collect my “size overlays”, define a suitable function to remove them and add it to org-ctrl-c-ctrl-c-hook. This, however, would result in having to write more code.
The second one was actually a bit tricky. I could do it in a naive way – start at the beginning of a headline, remember the position, go to the next headline on the same level, calculate the difference between point and the remembered position, show it in the overlay, and move to the next headline (possibly on a lower level). I decided to be a bit more fancy and move one heading at a time (irrespecitve of their levels), but use recursion to calculate the sizes of the subtrees. The idea was to write a function (I called it org-du--calculate-subtree-size-and-move) which would move to the next headline on the same level and put an overlay with the computed size on the one it started on, but instead of using org-forward-heading-same-level, it uses itself in a loop which ends when the level of the subheading is the same as the one we started at. This means that while moving, it calculates (and shows using overlays) the sizes of each subtrees as it passes over them.
And that’s pretty much it. The only remaining part is to copy the trick from org-clock-display to make sure that the overlays will be removed when the user modifes the buffer (for example, starts typing) – this means adding org-clock-remove-overlays to the before-change-functions hook. Of course, C-c C-c also removes them.
The code that does all this is actually quite short – just a bit over 50 lines, and about a third is the org-du--put-overlay function which is almost identical to org-clock-put-overlay. Here it is, in all its recursive glory
:
(defun org-du ()
"Compute (recursively) the size of the subtree at point.
Assume point is at the very beginning of the subtree. Move point to the
next subtree at the same level. Recursively create overlays with the
computed size."
(interactive)
(save-excursion
(when org-remove-highlights-with-change
(add-hook 'before-change-functions 'org-clock-remove-overlays
nil 'local))
(goto-char (point-min))
(unless (org-at-heading-p)
(outline-next-heading))
(while (not (eobp))
(org-du--calculate-subtree-size-and-move))))
(defun org-du--calculate-subtree-size-and-move ()
"A helper function for `org-du'.
Perform the actual computations for this subtree and each of its
subtrees."
(let ((level (org-current-level))
(beg (point)))
(outline-next-heading)
(while (progn
(when (> (org-current-level) level)
(org-du--calculate-subtree-size-and-move))))
(org-du--put-overlay beg (- (point) beg))))
;; Copied from `org-clock-put-overlay' and slightly changed
(defun org-du--put-overlay (pos size)
"Put an overlay on the headline at point, displaying SIZE.
Create a new overlay and store it in `org-clock-overlays', so
that it will be easy to remove. This function assumes point is
on a headline."
(save-excursion
(goto-char pos)
(org-match-line org-complex-heading-regexp)
;; the following `or's are a workaround for a bug when the actual
;; headline is empty
(goto-char (or (match-beginning 4) (match-beginning 2)))
(let* ((headline (or (match-string 4) ""))
(text (concat headline
(org-add-props
(make-string
(max (- (- 60 (current-column))
(org-string-width headline)
(length (org-get-at-bol 'line-prefix)))
0)
?\·)
'(face shadow))
(org-add-props
(format " %9s " (org-du--format-size size))
'(face org-clock-overlay))))
(o (make-overlay (point) (line-end-position))))
(org-overlay-display o text)
(push o org-clock-overlays))))
(defun org-du--format-size (size)
"Format SIZE in a human-friendly way."
(cond ((> size 1048576)
(format "%.1fM" (/ size 1048576.0)))
((> size 1024)
(format "%.1fk" (/ size 1024.0)))
(t size)))
And this is it for today. I hope you’ll find this useful!
CategoryEnglish, CategoryBlog, CategoryEmacs, CategoryOrgMode
2025-11-10 ASCII art timeline diagrams
A few weeks ago I was desiging some time-related code, and that code needed to handle “events” happening on various days. Without getting into too much detail, it had to do various things depending on stuff like “did this event happen at least a week after another one” or “how many days after «day zero» did that event happen”. Since there were quite a few rules concerning that, I wanted to document them in a clear, visual way.
I decided that a “timeline diagram” would be a perfect way to visualize these sorts of relationships. Of course, I didn’t want to draw my diagrams on paper and scan them (well, this is actually a viable option – pencil and paper are hard to beat as a UI! – just not what I wanted), so I needed to select some tool. There are many, many tools which can help with that – my teammates like Excalidraw a lot, there are many others, both web-based and desktop-based. Me being me, I decided to use Emacs and ASCII art. While I would definitely not call myself an “artist”, I have a sort of history of drawing stuff in Emacs with ASCII characters. Let’s add to that history!
(defun insert-time-axis ()
"Insert a \"time axis\" at point."
(interactive)
(let ((points ()) point (max 0))
(while
(progn
(setq point (read-number
"Time point in days (0 to finish): "))
(push point points)
(when (> point max)
(setq max point))
(plusp point)))
(setq points (sort points))
(unless (bolp)
(insert "\n"))
(dotimes (i (1+ max))
(insert
(if (memq i points)
"| " " ")))
(insert "\n")
(dotimes (i (1+ max))
(insert
(if (memq i points)
"+-" "--")))
(insert "->\n")
(dotimes (i (1+ max))
(insert
(if (memq i points)
(format "%-2d" i) " ")))
(insert " time [days]\n")))
As you can see, this code is exceptionally crude – it’s written in a very imperative style (and there’s nothing wrong with that in this particular case!), with all the setqs and dotimes loops. Also, instead of accepting a list of integers and some clever trickery in the interactive clause, it just doesn’t accept any arguments and uses read-number directly in the body of the function to get the list of numbers to mark with “ticks”. This means my command cannot be used non-interactively – but that’s ok, since I don’t intend to ever use it that way!
I can now say M-x insert-time-axis and type a few numbers, ending them with a zero (which makes sense – I always want a zero on my time axis) and get a “picture” like this:
| | | | +-------------+-----+-------+--> 0 7 10 14 time [days]
I can then manually edit it to add descriptions to the days marked with ticks etc., for example:
one week
| ten days
start | | two weeks
| | | |
+-------------+-----+-------+-->
0 7 10 14 time [days]
Was it faster to type this command (and make a few editions to the diagram itself) than to learn enough Excalidraw to make this diagram? Maybe, maybe not. Is it more cool to have a little command to create ASCII art timeline diagrams like this? Definitely. Would it be much more complicated to code automatic support for tick labels like the ones I added manually above? A bit, yes. Would it be difficult to write a command to create other diagrams programmatically, say Venn diagrams, or graphs of functions, or flowcharts etc.? Not really – if the diagram is too complex to draw using just plain inserts, I can always resort to calling Artist mode functions like I did several times before.
By the way, in the old days when I worked in academia and used LaTeX extensively, I would have probably used TikZ for diagrams like this. (I very highly recommend reading the section titled “Guidelines on Graphics” in its documentation, even if you do not plan to use TikZ or LaTeX at all – it is a solid collection of very no-nonsense tips on creating graphics in whichever tool you like, aimed especially at scientific or engineering documents.) And in fact, TikZ can create graphics in svg format, so it can be used for web-based documents, too. (Now that I think about it, I might be tempted to write another blog post showing how to draw similar timeline diagrams using TikZ and how to generate them in svg format. We’ll see!) In the meantime, if you ever write an engineering document requiring diagrams of any sort, remember that using Emacs – either interactively, using Picture mode or Artist mode, or programmatically – to draw them with ASCII art is a perfectly viable option.
2025-11-03 Beeminder Org timer
Once I wrote about what I called “Org mode burst timer”, another idea occurred to me. That code is good, and may be useful for some people – if it is for you, feel free to use it! But I can do something better for me. Instead of requiring myself to set the burst property manually, I could tie this code to my Emacs Beeminder client. When a clock is started on a beeminded task, it could retrieve the necessary data about the goal associated with the current headline and notify me when I work on it enough.
(Note: the following requires some knowledge about how Beeminder works, and in particular it uses a bit of Beeminder jargon.)
Now, this could work in several ways. The simplest way would be just to notify me after I work on a task for an amount of time equal to my daily goal, no matter how much I have worked on it today. For example, assume that I have my writing goal set up to write for 20 minutes per day, I have one day of buffer, I need to write for 17 minutes to increase the buffer to two days, and I’ve already been writing for 15 minutes today. With this approach the timer would be set to 20 minutes anyway. On the other hand, one could argue that the “correct” way would be to set it to 5 minutes (since I’ve already done 15 today), or even 2 (since this is what I need to increase the safety buffer). It is very tempting to use the first, simplest approach, but I decided to go a bit fancy and use a mixture of the second and third ones. I want to be notified as soon as the amount of time I have worked on a goal today is equal to my daily rate. Additionally, if my goal is on a beemergency, I want to be notified as soon as I did enough to dispatch it.
So, I decided to add such a feature to my Beeminder client. I wrote most of it in 2015 and 2016 when I was a beginning Elisp programmer, and many choices I made back then are… less than ideal, so to speak. (For example, it uses eval in some places.) Coming back to code you wrote 10 years ago is painful in more than one way – not only do you see your stupidity, but you have to collaborate with your stupid self from 10 years ago. But I really want to have this feature, so I did it anyway.
I’ve introduced two options. First of all, beeminder-org-notify-when-enough-p. Setting this to non-nil (which is the default now) will make Emacs notify me when the amount of work I’ve done on the goal is equal to the daily rate. Additionally, there is beeminder-org-notify-when-buffer-increased, firing a notification when I’ve done enough to increase my safety buffer by one day. This can be set to nil (to disable that feature), the symbol beemergency (the default, to enable it only for eep days) or anything else (to always enable it). Finally, I introduced beeminder-org-notification-sound, which has the same semantics as org-clock-sound (and is set to t by default).
And that’s it for today. If you happen to use my Emacs Beeminder client, you can download the newest version from GitHub and start using these options now. (Note that that version also includes a few other minor updates, the most visible being that the “Beeminder goal details” window shows a few aggregates – by default, the sum, median, average and daily average of the displayed datapoints. See beeminder-goal-template-fields-alist if you want to add yours.) At least for me, they seem to be quite helpful (even if a tiny bit distracting). While at that, I fixed some minor issues in the code (mainly adding necessary options to a few defcustoms), so even if you don’t want to use the new features, you might want to upgrade.
CategoryEnglish, CategoryBlog, CategoryEmacs, CategoryOrgMode