[Comps] continuation passing style

[Hamilton Link]

I'm forced to take issue with the misinformation that is in some of the
CPS notes that were distributed... how this person got into MIT I don't
know (well actually I'm sure she's plenty smart, this is just wierd
stuff she didn't understand), but this person said the stuff in
email-style quotes below.

Sorry if this is long-winded, I'm trying to explain it with lots of
explanation instead of a terse *poof*-we're-done kind of thing. If this
is not as clear as a friggin' Albuquerque summer's blue sky, ask me
questions (if you care, it might very well not be on the tests but I
didn't want anyone reviewing incorrect material) and/or give me feedback
on my delivery style.

I apologize for my funky indentation in advance, Netscape doesn't know
about lisp.

Anyway...

> Some rules of thumb
> -------------------
> - Continue is a procedure of one argument
> - only primitive operations can occur in the expression that is an argument to continue
> - procedures must contain only conditionals (if, cond, and, or), and other direct calls to procedures
> (including continue), with no pending operations
> - the tests in the conditionals should only involve primitive operations
>

I have to object, because these statements are all rather misleading if
not incorrect. Well, actually this isn't true -- the examples and the
above statements associated with them are WAY wrong in pure CPS. No
offense intended.

Here's the straight skinny. I have blatently plagerized my own
explaination of CPS from the other night, which fortunately I wrote
enough of down to bootstrap myself again. This is some wierd stuff.

Continuation passing style.

Imagine, for a moment, a universe in which functions (and in fact
everything else, apart from the archangel Lambda) never return when
called. These functions must be passed their arguments as usual, along
with an explaination of what do do with the result, since of course the
function can never return to you with the result for you to deal with as
you like. This is how continuation passing works. All CPS functions can
be thought of as requests to do something, after which something else
(which you have to say up front) will be done, after which something
else (which you have to say even earlier) will be done, etc. I say all
CPS functions because at some level we have to bootstrap ourselves into
CPS, and the implementation of CPS primitives can be done without
resorting to CPS.

OK. A digression is in order to properly formulate some examples, so
let's look at factorial

(define (fact1 n) (if (< n 2) 1 (* n (fact1 (- n 1)))))

and it's tail-recursive accumulating equivalent, which I will not bother
wrapping in syntactic sugar -- call it initially with an accumulator of
1

(define (fact2 n acc) (if (< n 2) acc (fact2 (- n 1) (* n acc))))

What is the difference between these, just so we can come back to this
difference later to draw some parallels? fact1 builds up a stack, and
after each call returns there is someone waiting to multiply to do the
last bit of work on the way to a solution, and return again. Fact2
reuses each stack frame (well, it can in a system where tail-calls are
compiled in this way -- a tail call is a function call whose result,
when returned, is immediately returned to the next level up. A good
compiler can take this situation and simply reuse the stack frame after
each call, and remember the original caller to whom the returned
returned returned etc. value would eventually have gotten to anyway, and
return directly to that place after the tail recursion bottoms out) and
carries along an accumulated value, which will be augmented and passed
along as we go until we're done and have our answer.

okie dokie. let's take a small piece of this, at first, and transform it
into CPS. then we'll work our way towards the entire thing for fact1,
and then do it again for fact2, and then we'll discuss how the CPS
versions are similar at run-time to the original.

(* n (fact1 (- n 1)))

Look at this piece of code. What does it do? well, the evaluator digs
down into the multiplication and discovers that it doesn't have
everything it needs to do the work, so it recursively calls the
evaluator to get working on the fact1 form... which also doesn't have
what it needs, so the evaluator is called AGAIN to work on the
subtraction, at which point we can actually do something (I'm assuming
we have a value for n handy in the lexical environment). Each function
then returns a value to the waiting wrapper so it can begin to do it's
work.
    So basically, as written this piece of code is backwards from the
order in which things are actually done, and we're way off base from CPS
because stuff has to be returned in order to do the next steps.

So anyway, we have to start somewhere. What has to happen first? Well,
the value (- n 1) has to be computed first, because everything else
depends on it. But we can't use "-" in CPS, because it returns. Let's
make a "c-" CPS version of "-" that takes three arguments (we'll worry
about implementing it later :) ), the two things to subtract and a
continuation to pass the result into:

(define (c- a b cont) ...)

in this case our continuation is what? we don't know yet, let's call it
c1 and move on with

(c- n 1 c1)

What's the continuation? Well, the next thing we have to do is compute
factorial and pass it's value into the multiplication. Let's make a
cfact function that takes a value and a continuation (we're obviously in
the process of defining it yet, so let's not write it's body down here)
and passes it's result into the continuation c2 -- now we've got

(cfact (c- n 1 c1) c2) ... ack!

ack, because c- doesn't return anything -- EVER. Well, poop. What to do,
what to do? As we're standing here scratching our heads, the archangel
Lambda comes down and says "hail, I can actually return something for
you to use right when you need it" -- so that's what we'll do. The
continuation c1 is a function that computes the factorial value and
passes it into another continuation, which if we go back and wrap the
cfact code in a lambda with a new variable, is exactly what lambda will
provide:

(c- n 1 (lambda (x) (cfact x c2)))

So what's c2? Stop reading here and figure it out for yourself if you
can...

What do we do with the cfact's result? we have to multiply it by
something.
What will do the mulitplication for us? not "*", we need to pretend we
have a "c*"
How does the c* get it's values to multiply? One is n, which we're
assuming has a value available, the other is the value passed into c2...
so we need to wrap a lambda around the c* which will bind a variable to
the value from cfact, making a continuation that we can use for c2.

(did I mention that Lambda is not the only archangel? so is Let -- but
when let shows up it's really lambda in disguise, did you know that?)

So our CPS version of the original form (* n (fact (- n 1))) is

(let ((c2 (lambda (y) (c* n y c3))))
    (let ((c1 (lambda (x) (cfact x c2))))
        (c- n 1 c1)))
... or written another way ...
(c- n 1 (lambda (x) (cfact x (lambda (y) (c* n y c3)))))

Sigh. Where's c3 come from? at some level we have to say "whatever you
want to do with the ultimate result that this form we're converting
would generate" -- so let's wrap this up before we move on (renaming
stuff and using let to make it purty):

(lambda (n c)
    (let ((c2 (lambda (y) (c* n y c))))
        (let ((c1 (lambda (x) (cfact x c2))))
            (c- n 1 c1))))

So in fact we have a boolean operator, a constant, and a conditional
left. Well, the boolean operator < is just another standard conversion
like these, but the conditional is something else again and the constant
is easy but different, so let's just look under the hood a second...

(define (c- a b c)
    (c (- a b)))
(define (c* a b c)
    (c (* a b)))

What's going on here? we've switched into the implementation layer,
which is not in CPS. - and * are normal functions that return a value,
which is then passed into the continuation we've made that never comes
back - the argument c. Thus c- and c* never come back, either, and we've
managed to get a smidge of work done on the road to nowhere. But here's
the implementation of c< and cif -- they're just the same, mostly:

(define (c< a b c)
    (c (< a b)))
(define (cif bool cthen celse c)
    (if bool
        (cthen c)
        (celse c)))

OK, look at cif again. What does it mean? again, we've hit bedrock, so
we're calling a real if that can chew on a boolean value and finish in
time to do something else. So it calls one or the other continuations
that are being passed in, passing into them the continuation of what to
do after the condition is done -- note that cthen and celse don't take
any values other than the continuation, and that whatever is passed into
c is in the hands of the conditional path taken. Which is just what we
want -- the path of computation taken is supposed to be generating the
values we'd keep going with.

Now that we know what cthen and celse look like, we can try using it on
this -- most of fact1, really, all if we deal with the constant 1 return
value where we want a continuation.

(if (< n 2) 1 (* n (fact1 (- n 1))))
->
to convert this, we again start by asking "what happens first?" The
boolean operation is evaluated.

(c< n 2 c?)

Now, the c? continuation must take a single value, because that's all c<
passes along -- it has to have the if inside it, plus the pre-packaged
consequence and alternative clauses. Fortunately we know what the
alternative clause looks like -- we converted that first. So it goes
something like this

(c< n 2 (lambda (bool) ?))
... put the cif in, with the eventual continuation being the original
c...
(c< n 2 (lambda (bool) (cif bool ? ? c)))
... let's split out the alternative so we don't get overly messy ...
(let ((celse (lambda (n c)
                    (let ((c2 (lambda (y) (c* n y c))))
                        (let ((c1 (lambda (x) (cfact x c2))))
                            (c- n 1 c1))))))
    (c< n 2 (lambda (bool) (cif bool ? celse c))))

now how do you turn a 1 into a continuation that takes a continuation as
it's only argument and continues? same as anything, wrap a lambda around
it that takes one argument and

(lambda (c)
    (c 1))

so our final factorial, once we get the outermost arguments on it, looks
like this (note the disappearance of the n argument to celse, which was
otherwise copied from above)

(define (cfact1 n c)
    (let ((celse (lambda (c)
                        (let ((c2 (lambda (y) (c* n y c))))
                            (let ((c1 (lambda (x) (cfact1 x c2))))
                                (c- n 1 c1))))))
        (c< n 2 (lambda (bool) (cif bool (lambda (c3) (c3 1)) celse
c)))))

So, what does this do when we run it? All the recursive calls into celse
end up building another c2 and recursing, which passes along a larger
and larger continuation until we hit bottom, at which point 1 is passed
into this continuation along with our original c and we start
multiplying like crazy. *gasp* which is exactly what happens when we
call fact1, when you think about it -- the stack gets built up, along
the lines of the lambdas we're building, each of which will eventually
do a multiplication and then go away. When we get to it, the CPS cfact2
that carries along an accumulator will work the same way as the original
cfact2 -- the continuations will be built just in front of the calls,
each passed in the decremented n and the accumulator, along with the
original c, and when the process is completed the only continuation to
be passed a result will be the original c -- just like the
tail-recursive version of fact2 shortcuts all it's stack frame
construction and reuses them, just to return directly to the parent with
the result at the end.

Happy? Of course you aren't, you want to see cfact2! Well, you should
derive it yourself -- really, you should rederive all of this yourself.
Here's the *poof* of it:

(define (cfact2 n acc c)
    (c< n 2 (lambda (bool)
                    (cif bool (lambda (cresult) (cresult acc))
                                (lambda (cresult)
                                    (let ((c2 (lambda (y) (c* n y
cresult))))
                                        (let ((c1 (lambda (x) (cfact1 x
c2))))
                                            (c- n 1 c1))))
                                c))))

I hope after the above explaination of the way the accumulating CPS
cfact2 works you can puzzle through this.

Some final discussion about intermediate values, and compilers, with
respect to the CPS transformation. All intermediate variables are made
explicit in the transformation to CPS -- note x, y, bool, etc. Great,
huh? well, for a compiler, it is great -- the compiler has to figure out
what to do with all the partials generated by the assembly code during
computation anyway, this just has it already mostly done and we can get
straight to register allocation. Plus the stuff that gets done first
comes first, another nice thing for a compiler, since the evaluator
really doesn't want to be crawling over code at run time. This is why
there's a book on "Compiling with Continuations". All good lisp and
scheme systems, I suspect, transform code into CPS during compilation,
or might as well.

Some things I glossed over...
some detailed accounts of how the other paper's examples are way
wrong-oh...
what's a lexical environment, exactly?
how's Lambda dress up as Let?
how does this relate to implementing a scheduler (slightly off topic, it
just provides an example of how continuations are like other things you
might understand better)?
how do continuations ever get passed more than one value? the examples
didn't show this because I wasn't calling any operations that wanted to
return more than one value... but at work I love
multiple-value-returning functions, and CPS supports this without any
(well, I should say any _more_) fancy language support

If you care, write me email, it's getting late.