From Newsgroup: comp.lang.lisp

On Sun, 20 Sep 2026 16:31:49 -0400, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:

I've never figured out what makes "delimited" continuations different
from general continuations except that the implementation constrains
where a "delimited" continuation may be invoked. But that seems to be
a difference in usage rather than in kind.

Full continuations are, conceptually, a copy of the whole stack, wrapped
as a kind of function that never returns.
Delimited continuations offer, conceptually, a copy of a chunk of the
stack, wrapped as a kind of function that does return.

=== Stefan

I appreciate the explanation ... but I'm a bit dense and I don't see
what delimited continuations give you vs just creating the equivalent
closure.


I look at continuations from the POV of a compiler writer: I see them
as co-routines. Calling a co-routine is a "sideways" jump into a
different call chain. If the co-routine is to finish and return, then obviously its originating call chain must be preserved.

You mentioned above that DCs are wrapped in a function, so they can
finish and "return" [for some definition].


But if the intent of keeping the stack is simply to allow the
[co-routine] entry point to be in a nested function ... well then it
seems that having the compiler perform closure conversion would get
you to a result equivalent to the DC without needing to preserve the
runtime call stack - which typically contains (lots of) extraneous
state unrelated to the needs of the closure.


Dunno. Perhaps implementations that heap allocate stack frames just
don't bother with closure conversion, choosing instead to waste the
address space and eat the extra GC processing.

I don't see any clear win for DC here. What am I missing?
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.lang.lisp

On 2026-09-22, George Neuner <gneuner2@comcast.net> wrote:

On Sun, 20 Sep 2026 16:31:49 -0400, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:

I've never figured out what makes "delimited" continuations different
from general continuations except that the implementation constrains
where a "delimited" continuation may be invoked. But that seems to be
a difference in usage rather than in kind.

Full continuations are, conceptually, a copy of the whole stack, wrapped
as a kind of function that never returns.
Delimited continuations offer, conceptually, a copy of a chunk of the >>stack, wrapped as a kind of function that does return.

=== Stefan

I appreciate the explanation ... but I'm a bit dense and I don't see
what delimited continuations give you vs just creating the equivalent closure.

Delimited continutaions close over the entire context/environment
between the capture point and some dynamic contour (the delimiting
prompt). The delimiting prompt can be in a parent function, or
grandparent, etc.

When we invoke ("revive") a delimited continuation, the execution inside
that continuation appears to pick up from the same point where the
continuation was captured. This code has available to it the original callling environment up to the delimiting prompt.

Here is the cool/key part: when the revived delimited continuation
returns to its original callers, and finally bubbles out to the capture contour, at that point it terminates. And whatever value that emerges
from the capture contour will be returned to the caller which revived
the continuation.

I look at continuations from the POV of a compiler writer: I see them
as co-routines. Calling a co-routine is a "sideways" jump into a
different call chain. If the co-routine is to finish and return, then obviously its originating call chain must be preserved.

If we must preserve the entire call chain, then we have an "undelimited" regular continuation. A regular continuation is like a delimited
continuation delimited by the dynamic top level of the enclosing form:
the top of the current thread.

The difference is that a regular continuation does not return when
it reaches its capture limit: the thread or process terminates there!

Like if you have some (defun startup () ...) main function and a
top-level form which calls (startup). If inside startup a continuation
is captured, and that continuation is then invoked, it will return all
the way to (startup) which terminates and then the image stops.

A delimited continuation doesn't stop the thread or image; it reaches
its contour, and then that value is returned to whatever invoked the continuation.

One obvious application for delimited continuations provides a way to
help understand them. Using delimited continuations, a block of code (surrounded by a delimiting prompt) can speculate about multiple ways of resuming itself. Say that block of code is Boolean, in that it returns
true or NIL for success or failure. Inside the block, a delimited
continuation can be captured. Then the block can try invoking the
continuation for various values, until it obtains a true.
It's essentially saying, "If I continue my computation with the value
42, will that make me return true? Oops, no it deosn't. Okay then,
suppose I continue myself with the argument value 73? Still nope ..."
Code can perpetrate a search of its future executions, with different
values, to find some condition, like a success indication.

This is how John MacCarthy's "amb" operator can be implemented
with delimited continuations.

You mentioned above that DCs are wrapped in a function, so they can
finish and "return" [for some definition].

Return mean to bubble out of the last enclosing frame that was
included in the capture, and then return to the caller which
invoked the continuation.

Importantly: all those frames that were returned from remain live: the continuation can be used again!

(That creates issues for scoped-based resource and other
scope-based things.)

But if the intent of keeping the stack is simply to allow the
[co-routine] entry point to be in a nested function ... well then it
seems that having the compiler perform closure conversion would get
you to a result equivalent to the DC without needing to preserve the
runtime call stack - which typically contains (lots of) extraneous
state unrelated to the needs of the closure.

Dunno. Perhaps implementations that heap allocate stack frames just
don't bother with closure conversion, choosing instead to waste the
address space and eat the extra GC processing.

Delimited continuations don't require heap-allocated stack frames.

One possible implementation is that stack frames are just stack
allocated, and so nothing is different when DC are not used.

When a DC is captured, the actual captured
part of the stack is copied out to a heap object.

When a DC is revived, stack material is copied out of its
heap object to the current top of the stack: the stack is extended
with all those captured frames, and the topmost frame inside of
that is made current. As the DC finishes executing up to its
delimiting prompt, that material naturally pops off.
(But the heap copy remains, for repeated uses of the DC!)

Thus, a spaghetti stack is never required.
--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.lang.lisp

Kaz Kylheku <046-301-5902@kylheku.com> wrote:

Delimited continutaions [...]

You've just made delimited continuations make sense to me for the first
time: thank you!

(That creates issues for scoped-based resource and other
scope-based things.)

If I understand this right (and this agrees with the mental model I've
built from all the excised bits of your post) this means that you still
need things like dynamic-wind with delimited continuations, and various resource-scoping problems are not solved by them.

But they sound hugely better than standard Scheme-style continuations. If
I didn't have a cat sitting on me I'd go up & play with them right away (I think Racket has them).
--
tfeb.org/computer/
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.lang.lisp

On 2026-09-24, tfb <tfb@work.it.out> wrote:

Kaz Kylheku <046-301-5902@kylheku.com> wrote:

Delimited continutaions [...]

You've just made delimited continuations make sense to me for the first
time: thank you!

(That creates issues for scoped-based resource and other
scope-based things.)

If I understand this right (and this agrees with the mental model I've
built from all the excised bits of your post) this means that you still
need things like dynamic-wind with delimited continuations, and various resource-scoping problems are not solved by them.

That is correct, because you can re-use a delimited continuation
(meaning, invoke it as a function) so that you can replay its captured
slice of future computation between the capture point and the delimiting prompt. The first resumption will blow out to the top, and unwind
everything.

I solved the problem without dynamic wind. The problem is not solved
for raw delimited continuations. Rather, I provide tools for solving
the problem for a coroutine/generator-like abstractions built on
delimited continuation, which I call "suspended execution contexts".

I provide an operator called "obtain" which creates a suspended
execution out of its dynamic contents and returns a lambda.

When we invoke the lambda, the interior of the suspension starts
executing, until it hits a (yield ...). The yield transfers
control to the original caller; the argument to the yield appears
as the return value of invoking the suspension.

The caller can then invoke the function again to resume the suspension,
and pass it a value. That value bubbles out of the (yield ...) call:
two-way communication. The suspension goes on to the next yield and so
on. Eventually it bubbles out. The programmer has to have some way of indicating "this is the last yield, don't call this any more".

These suspensions are based on DLC's. The original function is a DLC.
The subsequent (yield ...) replaces the suspension's function with
a brand new DLC capture. And every subsqeuent yield does the same.
Every quantum of computation between two yields "rides" a new DLC.

Exceptions, unwinding, dynamic scoped variable binding ... all
the dynamic stuff works properly inside the suspension as it hops
between yields with different DLC's.

What happens is that the yield operation uses an unsafe form of
dynamic control transfer which I named "abscond". By using
abscond, yield can return control to the client context that is using
the suspension.

Three second demo of abscond:

(block foo (unwind-protect (return-from foo 42) (prinl "cleanup!")))

"cleanup!"
42

(block foo (unwind-protect (sys:abscond-from foo 42) (prinl

"cleanup!")))
42

sys:abscond-from is an alternative return-from; it looks exactly like it
with the same convention, and uses a named block as its target.

However, unwinding is not done by sys:abscond-from; as you can see
the (print "cleanup!") int the unwind-protect is not happening.

So to reiterate, I have not solved the resource problem for the
low-level DLC's; there is no dynamic wind. But we have a working
solution for thread-like abstractions built on top of DLC's.
(Abstractions which do not reuse a continuation; do not repeat/replay
the same captured computation.)

I recommend that when a solution is built using raw DLC's, that dynamic
state that tears down should be avoided; or else that the programmer
invent an abstraction which solves the problem somehow.

Dynamic resources which are not destroyed will work properly.
For instance if we bind a dynamic variable (let ((*dyn-var* 42)) ...)
inside a DLC, such that when the DLC finishes that variasble is
unbound, when we re-enter the DLC, the variable will be correctly bound!

This is because TXR Lisp uses deep binding for dynamic variables;
the execution context has a pointer to a dynamic environment chain,
and when we re-animate the DLC, that pointer is restored.

Similarly other language-built-in resources that are dynamically
set up like exception catches are also properly re-established.
If you replay a DLC that has previously thrown an exception,
that DLC can throw that exception again.

However, any clean-up that mutates global state, like with-open-stream
closing a stream, is permanent, making it a one-shot.

But they sound hugely better than standard Scheme-style continuations. If
I didn't have a cat sitting on me I'd go up & play with them right away (I think Racket has them).

They are controlled; like continuations with a leash.

A regular continuation is like throwing a rock; but a delimited
continuation is like a boomerang or yo-yo.

A rock will return if someone throws it back, but a boomerang or yo-yo
have a built in limit.

The DLC gives us a clear contour where everything happens, such that
the outside is not affected; a caller that sets up a contour for
a DLC knows that the DLC cannot escape beyond that. And because it
returns, it is a function; DLC's can be composed. We can (mapcar
<some-dlc> list) etc.

The DLC "thinks" it has the whole execution future ahead of itself,
but it suddenly hits a brick wall at the delimited prompt, returning
control to the caller, as well as the value that bubbled out.
--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.lang.lisp

On 2026-09-24, Kaz Kylheku <046-301-5902@kylheku.com> wrote:

So to reiterate, I have not solved the resource problem for the
low-level DLC's; there is no dynamic wind. But we have a working
solution for thread-like abstractions built on top of DLC's.
(Abstractions which do not reuse a continuation; do not repeat/replay
the same captured computation.)

Here is a demo. We use a suspension to walk a cons-based tree
recursively and yield the items.

First, we just show how the items are yielded.

Then we repeat the exercise, but this time we turn on tracing
for the recursive function that does the tree walking.

We see how the tracing is correctly interleaved with our
calls to the suspension.

Because the suspension is absconding back to the top, all the trace stuff is intact; it can be resumed like a thread to run its next time slice
and produce its next slice of trace output.

At the very end, it bubbles out, closing out all the traces, and
t is returned (because of the cond clause ((null obj))).

Each time we call [fn], a new delimited continuation is used,
which was prepared ahead of time by the previous yield-from
(or the initial obtain). The replacement of one delimited
continuation by another inside the fn object is destructive:
it has a single context, just like a thread's instruction pointer
being destructively updated.

$ txr
This is the TXR Lisp interactive listener of TXR 302.
Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet.

(defun yflatten-rec (obj)

(cond
((null obj))
((atom obj) (yield-from yflatten obj))
(t (yflatten-rec (car obj))
(yflatten-rec (cdr obj)))))
yflatten-rec

(defun yflatten (obj) (yflatten-rec obj))

yflatten

(defvar fn (obtain (yflatten '(1 2 (3 4 (5) 6)))))

fn

[fn]

1

[fn]

2

[fn]

3

[fn]

4

[fn]

5

[fn]

6

[fn]

t

(trace yflatten-rec)

nil

(defparm fn (obtain (yflatten '(1 2 (3 4 (5) 6)))))

fn

[fn]

(yflatten-rec ((1 2 (3 4 (5) 6)))
(yflatten-rec (1)
1

[fn]

nil)
(yflatten-rec ((2 (3 4 (5) 6)))
(yflatten-rec (2)
2

[fn]

nil)
(yflatten-rec (((3 4 (5) 6)))
(yflatten-rec ((3 4 (5) 6))
(yflatten-rec (3)
3

[fn]

nil)
(yflatten-rec ((4 (5) 6))
(yflatten-rec (4)
4

[fn]

nil)
(yflatten-rec (((5) 6))
(yflatten-rec ((5))
(yflatten-rec (5)
5

[fn]

nil)
(yflatten-rec (nil)
t)
t)
(yflatten-rec ((6))
(yflatten-rec (6)
6

[fn]

nil)
(yflatten-rec (nil)
t)
t)
t)
t)
t)
(yflatten-rec (nil)
t)
t)
t)
t)
t

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.lang.lisp

On 2026-09-24, Kaz Kylheku <046-301-5902@kylheku.com> wrote:

Each time we call [fn], a new delimited continuation is used,
which was prepared ahead of time by the previous yield-from
(or the initial obtain). The replacement of one delimited
continuation by another inside the fn object is destructive:
it has a single context, just like a thread's instruction pointer
being destructively updated.

By the way, this is pretty awfully expensive. While, since we are Lisp programmers, we know the value of every [fn] call, we may not be aware
of the cost. Making a delimited continuation is not cheap (in this implementation): we are consing up a new heap object which contains a
snapshot of a slice of the run-time stack. Reviving it requires that
memory to be copied to the current stack top, and various fix-ups to
take place.

This is a kind of "big gun" to solve a problem where that way of
expressing it is worth the tradeoffj.
--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.lang.lisp

On Tue, 22 Sep 2026 19:03:18 -0000 (UTC), Kaz Kylheku <046-301-5902@kylheku.com> wrote:

... about delimited continuations ...


Hi Kaz,

Apologies for the delay - it's been one of those weeks.

I want to thank you for expounding on your explanation. I think I
understand better now, but I'm going to have to play with it a bit.


It would help greatly if I could look inside a compiler / runtime at a
real implementation. Decsriptions are fine, but seeing it done is how
I learn best.

I know Racket has DCs, I just have no idea where in the code to look.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: comp.lang.lisp

On 2026-09-27, George Neuner <gneuner2@comcast.net> wrote:

On Tue, 22 Sep 2026 19:03:18 -0000 (UTC), Kaz Kylheku
<046-301-5902@kylheku.com> wrote:

... about delimited continuations ...

Hi Kaz,

Apologies for the delay - it's been one of those weeks.

I want to thank you for expounding on your explanation. I think I
understand better now, but I'm going to have to play with it a bit.

It would help greatly if I could look inside a compiler / runtime at a
real implementation. Decsriptions are fine, but seeing it done is how
I learn best.

The implementation I made is likely poorly comprehensible. It's a giant
hack that directly manipulates the C stack, and (on reviving a
continuation) does a conservative scan to fix up things that look like pointers.
In the TXR source file "unwind.c", it starts at the definition of
"struct cont". It is a small amount of code ending at the function uw_capture_cont.

The function uw_push_cont_copy is also part of the implementation,
outside of that code range.

I will now take a moment to explain what these copy frames are about,
and why they are needed.

In compiled code, lexical variable bindings are on the stack, and so
they get naturally copied when continuations are captured and revived:
there is no sharing.

However, in interpreted code, bindings are not stack allocated;
they are dynamically allocated environment frames pushed onto their
own environment stack by the evaluator. If our stack-copying
continuation implementation deos not do anything special for these, a
very Bad Thing will happen: continuations will share lexical variables:
if a continuation mutates a variable, the original context as well
as other instances of continuations which captured across that same
context, will see the mutation!

The solution is a copy-on-capture implementation: the interpreter (code
in eval.c) installs copy handler frames when binding variables;
these copy handlers will deeply copy the environment frames when
a delimited continuation is captured, as well as when it is revived.
--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
--- Synchronet 3.22a-Linux NewsLink 1.2