2016-05-17 Emacs Lisp closures demystified

It is often claimed that one of the advantages of closures (as in “lexical scoping”) is information hiding (a.k.a. encapsulation). This is true, but as this post shows, you can’t really hide anything in Emacs Lisp;-).

Consider the classical example.

; -*- lexical-binding: t; -*-

(let ((counter 0))
  (defun increase-counter ()
    (setq counter (1+ counter)))
  (defun get-counter ()
    counter))

It is well known how this works: (increase-counter) increases an “internal” counter, and (get-counter) returns its value. (Here, “internal” means “internal to these two functions”.)

As the manual says, however,
Currently, an Emacs Lisp closure object is represented by a list with the symbol ‘closure’ as the first element, a list representing the lexical environment as the second element, and the argument list and body forms as the remaining elements.

(It says also that this is considered an internal implementation detail and hence we recommend against directly examining or altering the structure of closure objects.)

Let’s see what happens when we evaluate (symbol-function 'increase-counter):

(closure ((counter . 4) t) nil (setq counter (1+ counter)))

And here’s the result of (symbol-function 'get-counter):

(closure ((counter . 4) t) nil counter)

(Notice that the mysterious nils are just empty argument lists.)

And here’s where things start to get interesting. Before you evaluate this expression, try to guess the result.

(eq (cadr (symbol-function 'increase-counter))
    (cadr (symbol-function 'get-counter)))

See? Not only does the second part of the closure list look the same, it also is the same object! That’s why the “sharing” of the variable counter between these two closures actually work. Here’s the proof:

(fset 'increase-counter
      '(closure ((counter . 4) t) nil (setq counter (1+ counter))))
(fset 'get-counter
      '(closure ((counter . 4) t) nil counter))

Now symbol-function gives aparently the same results, but the counter variable (in fact, the lexical environment) is no longer shared between the two closures – no wonder, since they are now different (though identically looking) objects.

And finally, let us perform one more trick, just for fun – as the manual explicitly says, we should not rely on the exact structure of the closure objects. We are now going to manually recreate our two closures!

(let ((env '((counter . 0) t)))
  (fset 'increase-counter
	(list 'closure env nil '(setq counter (1+ counter))))
  (fset 'get-counter
	(list 'closure env nil 'counter)))

Try it out, it’s almost magic! We first set up the lexical environment in a temporary variable, then put the carefully crafted lists into the function cells of the symbols increase-counter and get-counter, and that’s it!

(In case you are wondering whether it would be possible to do the same without a temporary variable, the answer is yes. Fasten your seatbelt and look at this:

(fset 'increase-counter
      (list 'closure
	    '((counter . 0) t)
	    nil
	    '(setq counter (1+ counter))))
(fset 'get-counter
      (list 'closure
	    (cadr (symbol-function 'increase-counter))
	    nil
	    'counter))

Don’t try this at home, kids.)

CategoryEnglish, CategoryBlog, CategoryEmacs