The art of replacing long lambdas III: nested anaphoras and threading annoyances

(See Part I and Part II.)

Sometimes you come across nested lambdas or threading macros, and you happen to be fond of anaphoras. What do you do?

Anaphoras in nested lambdas

It helps if we distinguish between two different situations. In one situation, the lambdas can resolve themselves independently. In another, the resolution of the inner lambda is dependent on the outer lambda.

An analogy with nested for-loops in shell scripts may be helpful here.

Here are two "independent" loops:

for i in a b c; do
    printf "$i "     # only i here
    for j in 1 2; do
        printf "$j " # only j here
    done
done

Which gives us: a 1 2 b 1 2 c 1 2

And here are two "dependent" loops:

for i in a b c; do
    for j in 1 2; do
        printf "$i$j " # both i and j show up here
    done
done

Which gives us: a1 a2 b1 b2 c1 c2

But a problem shows up when we transport these into Emacs Lisp, with its mapping functions and anaphoric alternatives.

Defining i and j is fine when using full lambdas, where we can explicitly name arguments:

(lambda (i)
  (lambda (j)
    ...))

But explicitly naming arguments is exactly what we can't do when using anaphorics. What then?

When arguments from the outer lambda do not show up in the inner

In this case, we can use the exact same solution outside and inside. There will be no confusion between the anaphoras, which stay inside their respective lambdas.

Lambdas inside let*

Before nesting, let's start with this:

(let* ((list-of-vectors '([a b c d e] [1 2 3 4 5]))
       (list-of-lists (mapcar (lambda (vector)
                                (append vector '()))
                              list-of-vectors))
       (results (mapcar (lambda (list)
                          (reverse (butlast list 3)))
                        list-of-lists)))
  results)
=> '((b a) (2 1))

This is about as traditional, long, and boring as it gets.

This can be shortened a lot if we get rid of the let-bound variables and then nest the mapcars.

Nested mapcars, two long lambdas

(mapcar (lambda (lst) (reverse (butlast lst 3)))
        (mapcar (lambda (vec) (append vec ()))
                '([a b c d e] [1 2 3 4 5])))
=> '((b a) (2 1))

Perfectly readable, right?

First, the inner mapcar solves the vectors into lists.
Then, the outer mapcar does stuff with each of these lists.

Notice that:

  • The two lambdas don't intersect.
  • The second mapcar is outside the first lambda.

So these independent nestings look like this:

(mapcar (lambda (var1) …)          ; this lambda starts and ends here
        (mapcar (lambda (var2) …)  ; this lambda starts and ends here
                some-list)

Now, we could, if we wanted, rewrite our example like this:

(mapcar (lambda (x) (reverse (butlast x 3)))
        (mapcar (lambda (x) (append x ()))
                '([a b c d e] [1 2 3 4 5])))
=> '((b a) (2 1))

But this is not as immediately readable: you could now become confused as to whether the inner x has anything to do with the outer ones.

In general, then, you don't want to do that if you're using long lambdas.

Nested mapcars, two llamas

But maybe you're comfortable with that, in which case you could do this:

(mapcar (##reverse (butlast % 3))
        (mapcar (##append % ())
                '([a b c d e] [1 2 3 4 5])))
=> '((b a) (2 1))

sacrificing some readability for the sake of shortening those big lambdas.

And since % is a shorthand version for %1, you could also do this:

(mapcar (##reverse (butlast % 3))
        (mapcar (##append %1 ())
                '([a b c d e] [1 2 3 4 5])))
=> '((b a) (2 1))

just to make clear to the reader that the variables are not the same. This is because these two aren't allowed inside the same llama:

(##+ %1 %1) => (lambda (%1) (+ %1 %1))
(##+ % %)   => (lambda (%) (+ % %))
(##+ % %1) !!> error   ; ‘%’ and ‘%1’ are mutually exclusive

So if both % and %1 show up in an expression without throwing an error, as is the case of the nested mapcars above, this means that they're not part of the same lambda.

Nested anaphorics (--map)

Anyway, since the lambdas and llamas above work, then dash's anaphoric will also work:

(--map (reverse (butlast it 3))
       (--map (append it ())
              '([a b c d e] [1 2 3 4 5])))
=> '((b a) (2 1))

Nested anaphorics (--map) inside a -->

Now, we could go further and bring that list of vectors up, and use dash's anaphoric threading macro -->, so we'll have this:

(--> '([a b c d e] [1 2 3 4 5])
     (--map (reverse (butlast it 3))
            (--map (append it ())
                   it)))
=> '((b a) (2 1))

which is nice, because we start with the input.

Problem is, we now have three it, and each of them refers to a different thing!

So nested anaphoras work fine, but they can drive you insane if you haven't done enough of these to get a good feeling of what's going on.

Nested mapcars (a -cut and a llama) inside a -->

Now, here's the nice part.

Since we have several lambda-shortening weapons at our disposal, we can greatly improve readability by using different ones:

(--> '([a b c d e] [1 2 3 4 5])
     (mapcar (##reverse (butlast % 3))
             (mapcar (-cut append <> ())
                     it)))
=> '((b a) (2 1))

It's now very clear that the:

  • it can only belong to the anaphoric threading macro (-->)
  • <> is restricted to dash's combinator (-cut)
  • % is part of the llama (##)

And although we could replace either or both mapcar with -map, mapcar makes it clearer.

When arguments from the outer lambda show up in the inner

In this case, we cannot use the exact same solution outside and inside. There will be confusion between the anaphoras.

A long lambda inside another

This is the usual, with regular lambdas.

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (mapcar (lambda (lst)                 ;<-- lst
            (mapcar (lambda (fun)       ;<-- fun
                      (apply fun lst))  ;<-- fun AND lst
                    funs))
          lsts))
=> '((13 42) (42 441))

Notice that:

  • The two lambdas do intersect.
  • The second mapcar is inside the first lambda.

So these dependent nestings look like this:

(mapcar (lambda (var1)               ; the 1st lambda starts here
          (mapcar (lambda (var2) …)  ; the 2nd lambda starts and ends here
                  list-two))         ; the 1st lambda ends here
        list-one)

A llama inside a long lambda

Here we make the inner lambda a llama.

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (mapcar (lambda (lst)
            (mapcar (##apply % lst)
                    funs))
          lsts))
=> '((13 42) (42 441))

A long lambda inside a llama

Here we make the outer lambda a llama.

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (mapcar (##mapcar (lambda (fun)
                      (apply fun %))
                    funs)
          lsts))
=> '((13 42) (42 441))

A llama inside another

Here we try to make both lambdas a llama, and...

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (mapcar (##mapcar (##apply % %)
                    funs)
          lsts))
!!> wrong-number-of-arguments

...it won't work, of course.

We cannot possibly disambiguate the two %, each of which supposed to be part of a different lambda.

This is what's happening:

(##mapcar (##apply % %) funs)
=> (lambda () (mapcar #'(lambda (%) (apply % %)) funs))

A -cut inside another

And -cut also won't work, throwing a similar error.

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (-map (-cut -map (-cut apply <> <>)
              funs)
        lsts))
!!> wrong-number-of-arguments

Because that expands to:

(-cut -map (-cut apply <> <>) funs)
=> (lambda () (-map #'(lambda (D1 D2) (apply D1 D2)) funs))

A --map inside another

Finally, anaphorics won't help us here either, for a similar reason:

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (--map (--map (apply it it)
                funs)
         lsts))
!!> wrong-type-argument

Now, if you're determined to go out of your way to avoid long lambdas, there are a couple of alternatives. What they have in common is that they combine different approaches to short lambdas, each of which uses a different anaphora — so the ambiguity is gone.

A -cut inside a --map

The next one uses only dash.

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (--map (mapcar (-cut apply <> it)
                 funs)
         lsts))
=> '((13 42) (42 441))

Note that in this and in the next examples, you could replace any mapcar with -map — but readability would suffer a bit.

A llama inside a --map

A combination of dash and llama is also possible.

You can either nest them:

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (--map (mapcar (##apply % it)
                 funs)
         lsts))
=> '((13 42) (42 441))

A llama of a --map

Or you can compose them in hash-hash-dash-dash fashion:

(let ((funs '(+ *))
      (lsts '((6 7) (21 21))))
  (mapcar (##--map (apply it %)
                   funs)
          lsts))
=> '((13 42) (42 441))

They have the same length:

(--map (-map (##apply % it) '(+ *)) '((6 7) (21 21))) => '((13 42) (42 441))
(-map (##--map (apply it %) '(+ *)) '((6 7) (21 21))) => '((13 42) (42 441))
;;       ^definitely not weird

These two are as short as the thing gets — yet ditching the let and one-lining it sacrifices some readability.

All of this was inspired by a real case that shows up in a giant example that tests all of xht's hash table equality functions.

Anaphoras when threading

We already saw a case where anaphorically threading through separate lambdas is made much clearer by using different lambda-shortening weapons instead of letting three different it lying around.

Here's a similar case.

Traditional "threading" with let* using mapcar

The usual dash-less way to do things.

(let* ((lst '(3 4 5))
       (lst (mapcar (lambda (n) (expt n 2)) lst))  ;  '(9 16 25)
       (sum (apply #'+ lst)))                      ; (+ 9 16 25) => 50
  (/ sum 2.0))
=> 25.0

Anaphorically threading plus anaphoric --map

We could dial up the dash by adding -sum, anaphoric threading (-->), and anaphoric mapping (--map):

(--> '(3 4 5)
     (--map (expt it 2) it)
     -sum
     (/ it 2.0))
=> 25.0

It's clear that the second it cannot belong to the --map, since --map's it always shows up only inside the form. So that second it must be from the threading macro.

This is perfectly readable — but only if you've done a lot of them. And even then, you need to pause a bit to untangle the it in your mind. Maybe there's something a bit better?

Anaphorically threading plus -cut

(--> '(3 4 5)
     (-map (-cut expt <> 2) it)
     -sum
     (/ it 2.0))
=> 25.0

The ambiguity is now gone: the two it can only be from the threading macro.

But we notice that the (now) first it is the last argument, whereas the (now) second it is the first argument. Maybe we can change the order?

A generic solution is available with -flip:

(--> '(3 4 5)
     (-map (-cut expt <> 2) it)
     -sum
     (funcall (-flip #'/) 2.0 it))
=> 25.0

But that last one is unnecessary in this specific case, because division has a well-known inverse operation (multiplication) that is indifferent to argument order. Therefore:

(--> '(3 4 5)
     (-map (-cut expt <> 2) it)
     -sum
     (* 0.5 it))
=> 25.0

So now both are the last argument.

Threading last plus -cut

We can then get rid of the more flexible (but noisier) anaphoric threading, simplifying it into threading last (->>):

(->> '(3 4 5)
     (-map (-cut expt <> 2))
     -sum
     (* 0.5))
=> 25.0

Threading last plus --map

Having done that, we can reverse that -map + -cut combo, replacing it with the shorter (and more familiar to most) anaphoric --map:

(->> '(3 4 5)
     (--map (expt it 2))
     -sum
     (* 0.5))
=> 25.0

which fits nicely into a single line:

(->> '(3 4 5)  (--map (expt it 2))  -sum  (* 0.5)) => 25.0

Alternative: -as->

The above is simpler, but there was another way out.

We had this:

(--> '(3 4 5)
     (--map (expt it 2) it)
     -sum
     (/ it 2.0))
=> 25.0

From which we could have gone to this:

(-as-> '(3 4 5) acc
       (--map (expt it 2) acc)
       -sum
       (/ acc 2.0))
=> 25.0

The choice of acc here is only because it's already fontified by dash (so is other, but that's longer).

It could have been anything: foo, res, results, or even _, so this would also work:

(-as-> '(3 4 5) _
       (--map (expt it 2) _)
       -sum
       (/ _ 2.0))
=> 25.0

Another let*, but with --map

For comparison, the closest let*-threaded-but-still-dash version of the above would be:

(let* ((_ '(3 4 5))
       (_ (--map (expt it 2) _))
       (_ (-sum _)))
  (/ _ 2.0))
=> 25.0

which is just a bit longer, and similarly readable.

Enjoy your anaphoric nestings and threadings.