2015-11-21 The Emacs widget library and automatic modification of editing fields

A long time ago, I read about the Emacs Widget library. My first thought was “Wow, that is so cool!”. I didn’t have any use for that, however.

Until recently.

I work for a journal (together with my colleague), and one of our duties is to take submissions and enter them into our system (then, we try to turn them into something compatible with our LaTeX template etc.). We don’t have any automatic submission system – it would probably be an overkill, mainly because we strongly suspect that most authors would not use it anyway (the usual modus operandi for them is to send their papers by email to one of the editors, who then sends it to us).

Therefore, what we have to do for each submission, is to initialize a Mercurial repository with a suitable name, put the file(s) in there, and mould them into our template. This is something that would benefit from automation, so I decided to use Widget to facilitate entering article metadata; my (yet-to-be-written) function will then seed the repository with an article template.

One of the pieces of said metadata is the author’s name. As is often the case, we have two variants of it: the full name and the abbreviated name (for the running header). I decided to have two editing fields for these two strings, but since 99% of the time the abbreviated name can be generated automatically, I wanted the second one to be autofilled when the user enters the full name.

This seemed perfectly doable. Each widget has a :notify property, which is a function called each time the user changes the state of that widget. (See the manual for details.) So this is (more or less) what I did:

(defun widget-autofilling-example ()
  "An example of what happens if the autofilling is used in
a naive way."
  (interactive)
  (switch-to-buffer "*Widget-autofill-examle*")
  (kill-all-local-variables)
  (let ((inhibit-read-only t))
    (erase-buffer))
  (remove-overlays)
  (setq widget-author-list
	(widget-create 'editable-list
		       :entry-format "%i %d %v"
		       :value '(("" ""))
		       '(group (editable-field :size 48 :format "Full name:  %v\n"
					       :notify update-shortname)
			       (editable-field :size 36 :format "Short name: %v\n"))))
  (use-local-map widget-keymap)
  (widget-setup))

(defun update-shortname (name-widget &rest ignore)
  "Update the shortname -- the naive version."
  (save-excursion (widget-value-set (cadr (widget-get (widget-get name-widget :parent) :children))
				    (shorten-name (widget-value name-widget)))))

(defun shorten-name (name)
  "Make NAME shorter by abbreviating each word besides the first
  one to one letter."
  (let ((case-fold-search nil))
    (replace-regexp-in-string "\\(\\b[[:upper:]]\\)[[:lower:]]+\\b\\(.\\)" "\\1.\\2" name)))

A few things might need clarifying. The shorten-name function is the usual regex fun, which shortens all but the first words (starting with a capital letter) to the first letter. Notice that we temporarily set case-fold-search to nil so that the search actually distinguishes lower and upper case. This way, “John Smith” gets shortened to “J. Smith”, but “Johann von Gutenberg” becomes “J. von Gutenberg”. The second group (containing only the dot, i.e., matching any character) is there to ensure that the last word beginning with a capital letter is not modified – it won’t match since there’s no character after it to be matched by the dot operator. (Emacs regexen have \', which matches the empty string, but only at the end of the string. They don’t have the reverse: a construct which would match an empty string not at the end of the string.)

The first thing really related to widgets is the widget-autofilling-example function. It is modelled after the example in the manual, but contains one twist: an editable list whose elements are groups of fields. If I didn’t give the :value keyword, it would be initialized as an empty list; this is not what we want, we assume that it is best to assume one author as the default. (In the rare case when there are no authors – it happens in some unusual circumstanes – the user can always delete the only element of the editable list anyway, using the [DEL] button). The :notify property is the name of the function which will be called when any change occurs to the containing widget (see the docs for details on its argument list).

The actual function doing the work is update-shortname. Notice the magic cadr: since there are no functions in the widget library (at least, not documented ones) to walk the widget tree, getting the next widget requires going one level up (so you first get the :parent and then its :children), and then selecting the right one from the children (in this example, the second one). It is ugly, it is unmaintainable, and it is implementation-dependent, but I know no other way to accomplish this. (If this wasn’t part of one of the elements of the editable list, things would be different – I could just say (setq widget-name (widget-create ...)), like in the case of widget-author-list.) The save-excursion makes sure the point stays where it should.

So, everything looks ok, right? But it’s not. Try it yourself and you’ll see why: with each character automatically entered in the shortened name field, the field seemingly grows by one character, and with each character deleted (either by pressing backspace, or by shorten-name trickery) it shrinks.

I have to say that I was clueless what to do with it. The “obvious” solution – calling widget-setup in update-shortname – didn’t help. Inspecting the whole code of the widget library seemed too much work.

Happily, I found what I needed rather quickly. Since C-h c a in a field shows that a is bound to the usual self-insert-command, I suspected that the adjusting of the field size (which works when normally typing) must be either an advice (but grepping for “advice” in the widget sources returned no results) or a hook. Grepping for “hook” yields (after a minute of searching) the function widget-after-change, which is added to the hook after-change-functions.

Looking for that hook in the Elisp reference manual I learned that it doesn’t seem trivial to use it. I was wrong, however; it just required a few minutes of thinking. Each function in that hook should receive three arguments: the position of the beginning and the end of the text that changed, and the length of the text before the change. In normal circumstances, Emacs calls those functions itself, with proper arguments. In our case, they are not called. The reason for that is (more or less) clear: calling those hooks is (temporarily) disabled while they are running, probably in order to avoid infinite loops.

So, here’s the way I figured out to solve my problem:

(defun update-shortname (name-widget &rest ignore)
  "Update the shortname.  A working, though needlessly complicated version."
  (let* ((short-name-widget
	  (cadr (widget-get (widget-get name-widget :parent) :children)))
	 (old-overlay (widget-get short-name-widget :field-overlay))
	 (old-beg (overlay-start old-overlay))
	 (old-end (overlay-end old-overlay))
	 (old-beg-mark (copy-marker old-beg))
	 (old-end-mark (copy-marker old-end))
	 (old-size (- old-end old-beg)))
    (save-excursion (widget-value-set short-name-widget
				      (shorten-name (widget-value name-widget)))
		    (widget-after-change (marker-position old-beg-mark)
					 (marker-position old-end-mark)
					 old-size))))

As you can see, there’s a lot of juggling of markers and overlays: I first obtain the field overlay, then I convert its beginning and end to markers, then perform the change (so that the markers get relocated properly), and finally convert the markers to integers back again and feed them into the widget-after-change function (which normally sits in after-change-functions). Quite a mouthful.

It turns out that this is not really necessary. My final implementation is much better (and it addresses the root of the problem, not its symptoms). It turns out that the inhibition of the modification hooks is governed by the aptly named inhibit-modification-hooks variable. The best solution (at least the best I could invent) is therefore to bind it to nil for a moment when autoupdating the shortened name field:

(defun update-shortname (name-widget &rest ignore)
  "Update the shortname -- the naive version."
  (save-excursion
    (let ((inhibit-modification-hooks nil))
      (widget-value-set (cadr (widget-get (widget-get name-widget :parent)
					  :children))
			(shorten-name (widget-value name-widget))))))

The bottom line is that (as usual) Emacs is very customizable, and the widget library is yet another proof for that. It might be not extremely easy to use, but you can do very nice things with it.

CategoryEnglish, CategoryBlog, CategoryEmacs