I've yet to see examples [of closures] that have real uses and that
don't make code harder to grasp.
On Fri, 10 Oct 2025 20:04:14 +0100, bart wrote:
I've yet to see examples [of closures] that have real uses and that
don't make code harder to grasp.
I used a class factory in my Python wrapper for Cairo. It was
convenient to define higher-level Python wrappers around lower-level C structs, where there was a lot of commonality across those different wrappers, to save having multiple copies of the same code in the
source. Thus, defining a rCLFontExtentsrCY wrapper class around the rCLfont_extents_trCY struct became as simple as
FontExtents = def_struct_class \
(
name = "FontExtents",
ctname = "font_extents_t"
)
And rCLTextExtentsrCY as a wrapper around rCLtext_extents_trCY; but here it became convenient to add extra attributes, to return entire Rect and
Vector components as single objects, where the C code had the coordinates
as separate fields. So I define the derived attributes in an additional temporary helper class:
class TextExtentsExtra :
# extra members for TextExtents class.
@property
def bounds(self) :
"returns the bounds of the text_extents as a Rect."
return \
Rect(self.x_bearing, self.y_bearing, self.width, self.height)
#end bounds
@property
def advance(self) :
"returns the x- and y-advance of the text_extents as a Vector."
return \
Vector(self.x_advance, self.y_advance)
#end advance
#end TextExtentsExtra
and then merge that into the wrapper:
TextExtents = def_struct_class \
(
name = "TextExtents",
ctname = "text_extents_t",
extra = TextExtentsExtra
)
del TextExtentsExtra
The definition of rCLdef_struct_classrCY is 79 lines of code. For more details, see <https://gitlab.com/ldo/qahirah>.
This is a comparatively simple example; I have more complex ones
elsewhere, if you want them.
On 10/10/2025 05:18, Waldek Hebisch wrote:<snip>
bart <bc@freeuk.com> wrote:
It is not clear to me what you measure. I used:
Last week I posted a link to a 'man or boy' test; this a version in C:
https://rosettacode.org/wiki/Man_or_boy_test#C
This test requires local functions with full closures to implement
natively. The first C version emulates what is needed.
I tweaked that version so that is used N=13 instead of N=10, and
evaluates the function 10,000 times. The second version uses C
extensions to provide the higher level functionality.
These are timings I got for the two versions:
First C version (gcc -O2: 0.8 seconds
(Tiny C): 1.8 seconds
Second C version (gcc -O2): 590 seconds (extrapolated from 1K loops) >>> (gcc -O0): 940 seconds
So avoiding those complex functional features makes even TCC's poor code >>> 300 times that optimised C. And nearly 800:1 better comparing -O2 with -O2. >>
#include <stdio.h>
#define INT(body) ({ int lambda(){ body; }; lambda; })
int
main(){
int a(int k, int xl(), int x2(), int x3(), int x4(), int x5()){
int b(){
return a(--k, b, xl, x2, x3, x4);
}
return k<=0 ? x4() + x5() : b();
}
int res = 0;
int i;
for(i = 0; i < 10000; i++) {
res += a(13, INT(return 1), INT(return -1), INT(return -1), INT(return
1), INT(return 0));
}
printf("res = %d\n", res);
}
Compiling this using:
gcc -O -Wall man1.c -o man1
I get 0.895s. Using -O2 I get 0.693s. Version without loop, but using
13 as an argument (as above) takes 0.002s. This is on a fast machine,
on really slow mini-PC I get 10.117s at -O2. I suspect that your time is
due to operationg system, the results above are for Linux on x86_64.
On Linux on 1GHz Risc-V at -O2 I get:
real 0m28.910s
user 0m14.512s
sys 0m14.381s
which means that about half of execution time is in system calls.
Looking at assembly shows that the machine code is calling
'__riscv_flush_icache', that is it flushing cache to make sure
that processor will execute freshly generated instructions
(not needed on PC-s). System calls are expensive, and flushing
cache also adds slowdown, so this adds substantial cost. Still,
the Risc-V machine is quite slow, and can do this much faster
than your estimate of execution time.
For me on fast machine first (emulated) version takes 0.331s when
compiled by gcc at -O2 and 0.767s when compiled by tcc. On Risc-V
emulated version (compiled by gcc -O2) needs 3.553s.
So emulated version is clearly faster, especially if version
using nested functions need support from operating system.
But the point of using nested functions/closures is that code
is simpler.
Well, the code is smaller; that doesn't mean it is simpler! It can be
harder to know what's going on with a cryptic bit of code like this.
(I believe it was Knuth who failed to correctly predict the result of
N=10 (?), at some point in history when computing it was not practical.)
The test is doing nonsense operation, so one
can miss simplicty, but clearly emulated version is longer.
More important, in bigger program all involved calls must be
dane in nonstandard way, which adds work (potentially quite a lot).
In version with nested functions only efort is in definition
of nested functions, calls are standard calls and rest of program
can be written as if there were no use of nested functions.
As you can see, when properly supported extra cost of nested
functions is moderate.
BTW: Some language implementation use approach which is
equivalent to emulated version. For heavy use of nested
functions this may lead to faster code (like in this example).
But when "emulation" is part of language implementation
all calls would need to do extra work, slowing down normall
calls. In language like C this is deemed unacceptable,
nested function are rarely use so they must pay extra,
while normal call work at maximal speed.
I'm testing the version show below (I added the += to make it match
yours). Results on Windows are:
c:\c>gc d opt
Invoking: gcc -s -O2 d.c -o d.exe -lm -ldl -fno-strict-aliasing
Compiled d.c to d.exe
c:\c>tm d
-64200
TM: 6.01
So 6 seconds to do 100 iterations. If I try it in WSL, using 10000 iterations, then I get:
c:\c>wsl
root@DESKTOP-11:/mnt/c/c# gcc -O2 d.c -od
root@DESKTOP-11:/mnt/c/c# time ./d
-6420000
real 0m1.184s
user 0m1.180s
sys 0m0.001s
So under WSL it's 500 times faster. I've no idea why.
However, I work under Windows! So do lots of people. And there, the fact that these exotic features might make programs significantly slower has
to be taken into consideration.
I tried a Python version too. There, 200 iterations took 5.4 seconds on
both OSes. PyPy on Windows was twice as fast.
On 10/10/2025 22:45, Lawrence DrCOOliveiro wrote:
The definition of rCLdef_struct_classrCY is 79 lines of code. For more
details, see <https://gitlab.com/ldo/qahirah>.
I can't see any use of closures above, and I can't find those 79 lines
of code in that link. Not that it would help.
But I've found in the past with languages like Python and C++ which
offer a plethora of advanced features, that people will use them because they're there.
On Sat, 11 Oct 2025 00:29:55 +0100, bart wrote:
On 10/10/2025 22:45, Lawrence DrCOOliveiro wrote:
The definition of rCLdef_struct_classrCY is 79 lines of code. For more
details, see <https://gitlab.com/ldo/qahirah>.
I can't see any use of closures above, and I can't find those 79 lines
of code in that link. Not that it would help.
ThatrCOs where closures are used. I could copy and paste the whole thing, if you like.
But I've found in the past with languages like Python and C++ which
offer a plethora of advanced features, that people will use them because
they're there.
You could prove your point by doing what my code does more simply, in C
say, without using that rCLplethora of advanced featuresrCY. But you canrCOt manage that, can you?
I'm not quite clear what you code does, or whether I would need to do
the same.
I need simple, solid, intuitive features that anyone can understand
and use confidenly. Or if necessary, implement a different way.
On Sat, 11 Oct 2025 01:59:28 +0100, bart wrote:
I'm not quite clear what you code does, or whether I would need to do
the same.
You said you couldnrCOt see a need for this sort of functionality. This
is an example of this sort of functionality.
I need simple, solid, intuitive features that anyone can understand
and use confidenly. Or if necessary, implement a different way.
Hint:
def def_struct_class(name, ctname, extra = None) :
# defines a class with attributes that are a straightforward mapping
# of a ctypes struct. Optionally includes extra members from extra
# if specified.
ctstruct = getattr(CAIRO, ctname)
class result_class :
...
#end def_struct_class
Can you see the closure yet?
bart <bc@freeuk.com> wrote:
On 10/10/2025 05:18, Waldek Hebisch wrote:<snip>
bart <bc@freeuk.com> wrote:
Last week I posted a link to a 'man or boy' test; this a version in C: >>>>
https://rosettacode.org/wiki/Man_or_boy_test#C
This test requires local functions with full closures to implement
natively. The first C version emulates what is needed.
I tweaked that version so that is used N=13 instead of N=10, and
evaluates the function 10,000 times. The second version uses C
extensions to provide the higher level functionality.
These are timings I got for the two versions:
First C version (gcc -O2: 0.8 seconds
(Tiny C): 1.8 seconds
Second C version (gcc -O2): 590 seconds (extrapolated from 1K loops) >>>> (gcc -O0): 940 seconds
So avoiding those complex functional features makes even TCC's poor code >>>> 300 times that optimised C. And nearly 800:1 better comparing -O2 with -O2.
It is not clear to me what you measure. I used:
#include <stdio.h>
#define INT(body) ({ int lambda(){ body; }; lambda; })
int
main(){
int a(int k, int xl(), int x2(), int x3(), int x4(), int x5()){
int b(){
return a(--k, b, xl, x2, x3, x4);
}
return k<=0 ? x4() + x5() : b();
}
int res = 0;
int i;
for(i = 0; i < 10000; i++) {
res += a(13, INT(return 1), INT(return -1), INT(return -1), INT(return
1), INT(return 0));
}
printf("res = %d\n", res);
}
Compiling this using:
gcc -O -Wall man1.c -o man1
I get 0.895s. Using -O2 I get 0.693s. Version without loop, but using
13 as an argument (as above) takes 0.002s. This is on a fast machine,
on really slow mini-PC I get 10.117s at -O2. I suspect that your time is >>> due to operationg system, the results above are for Linux on x86_64.
On Linux on 1GHz Risc-V at -O2 I get:
real 0m28.910s
user 0m14.512s
sys 0m14.381s
which means that about half of execution time is in system calls.
Looking at assembly shows that the machine code is calling
'__riscv_flush_icache', that is it flushing cache to make sure
that processor will execute freshly generated instructions
(not needed on PC-s). System calls are expensive, and flushing
cache also adds slowdown, so this adds substantial cost. Still,
the Risc-V machine is quite slow, and can do this much faster
than your estimate of execution time.
For me on fast machine first (emulated) version takes 0.331s when
compiled by gcc at -O2 and 0.767s when compiled by tcc. On Risc-V
emulated version (compiled by gcc -O2) needs 3.553s.
So emulated version is clearly faster, especially if version
using nested functions need support from operating system.
But the point of using nested functions/closures is that code
is simpler.
Well, the code is smaller; that doesn't mean it is simpler! It can be
harder to know what's going on with a cryptic bit of code like this.
(I believe it was Knuth who failed to correctly predict the result of
N=10 (?), at some point in history when computing it was not practical.)
As I wrote below, this test is doing nonsense operation. Most real
uses of higher order functions are simpler (some higher order may
even disregard them as too simple). For example, you want to
perform some operation on all elements of an aggregate. For a list
one can have mostly standard C code like:
typedef struct gen_lst gen_lst;
struct gen_lst {gen_lst * next;};
void
gen_map(void (*f)(gen_lst * lst, void * extra), gen_lst * lst, void * extra) {
for(; lst; lst = lst->next) {
f(lst, extra);
}
}
typedef struct my_lst my_lst;
struct my_lst {gen_lst * next; int val;};
void
adder1(gen_lst * lst, void * extra) {
int * aux = extra;
struct my_lst * ml = (struct my_lst *)lst;
aux += ml->val;
}
int
my_sum(my_lst * lst) {
int aux = 0;
gen_map(adder1, (gen_lst *) lst, (void *)&aux);
return aux;
}
or you can use gcc extention like:
void
gen_map2(void (*f)(gen_lst * lst), gen_lst * lst) {
for(; lst; lst = lst->next) {
f(lst);
}
}
int
my_sum2(my_lst * lst) {
int aux = 0;
void
adder(gen_lst * lst) {
struct my_lst * ml = (struct my_lst *)lst;
aux += ml->val;
}
gen_map2(adder, (gen_lst *) lst);
return aux;
}
Note that argument 'extra' is now gone.
Of course, you my ask why I do computations in this way? If all what
needs to be done is computing sum of elements on a list, than a single function would do. However, if there are more operations, then
using several looping functions spead knowledge how 'gen_lst' is
build and how to traverse it. Which may be not that bad if you
are sure that it will always be a list and all you need is summing.
But you may need more complex aggregate or multiple kinds of aggregates. Typically there are several operations one wants to do for each
kind of aggregate. And general part may be much larger than type
specific part. With approach as above you only need to write
type specific operations for each kind of aggregate, general parts
may be reused. And depending on details it may be possible to
share operations for different kinds of aggregates.
I do not know if you believe that this kind of coding is useful,
but I hope that it is clear that variant using nested functions
will always be simpler than variant which passes extra arguments.
So under WSL it's 500 times faster. I've no idea why.
I see. It is up to you to decide if you want to investigate deeper.
Linux version generates small pieces of executable code on the
stack. Currently security folks hate such code and want to ban
it. In Linux gcc specially marks executable needing executable
stack and if executable is marked kernel allow executing code
from the stack. Otherwise attempts to execute code from the
stack lead to exception and kernel signals error in such case.
There was some talk to use different method, and code generated
on Windows may be different. One possible implementation would
be to attempt execution on the stack, but also install signal
handler. Signal handler can then effectively interpret was in
on the stack. Clearly this would lead to much worse performance.
However, I work under Windows! So do lots of people. And there, the fact
that these exotic features might make programs significantly slower has
to be taken into consideration.
I tried a Python version too. There, 200 iterations took 5.4 seconds on
both OSes. PyPy on Windows was twice as fast.
Once you have interpreted language you get slowdown from the
interpreter, but no extra trouble due executable stack: CPU treats
bytecode as data, so in this way one can bypass security restrictions.
IIUC PyPy may generate machine code, but probably can defer anything
it can not handle to interpreter.
On 11/10/2025 01:05, Waldek Hebisch wrote:
[...]
I haven't implemented closures in a static language. (I got partway in
the dynamic one, but decided it wasn't worth the effort or extra
complexity just to run silly examples like manboy() or twice().)
[...]
On 11/10/2025 01:05, Waldek Hebisch wrote:
bart <bc@freeuk.com> wrote:
On 10/10/2025 05:18, Waldek Hebisch wrote:<snip>
bart <bc@freeuk.com> wrote:
Last week I posted a link to a 'man or boy' test; this a version in C: >>>>>
https://rosettacode.org/wiki/Man_or_boy_test#C
This test requires local functions with full closures to implement
natively. The first C version emulates what is needed.
I tweaked that version so that is used N=13 instead of N=10, and
evaluates the function 10,000 times. The second version uses C
extensions to provide the higher level functionality.
These are timings I got for the two versions:
First C version (gcc -O2: 0.8 seconds
(Tiny C): 1.8 seconds
Second C version (gcc -O2): 590 seconds (extrapolated from 1K loops) >>>>> (gcc -O0): 940 seconds
So avoiding those complex functional features makes even TCC's poor code >>>>> 300 times that optimised C. And nearly 800:1 better comparing -O2 with -O2.
It is not clear to me what you measure. I used:
#include <stdio.h>
#define INT(body) ({ int lambda(){ body; }; lambda; })
int
main(){
int a(int k, int xl(), int x2(), int x3(), int x4(), int x5()){
int b(){
return a(--k, b, xl, x2, x3, x4);
}
return k<=0 ? x4() + x5() : b();
}
int res = 0;
int i;
for(i = 0; i < 10000; i++) {
res += a(13, INT(return 1), INT(return -1), INT(return -1), INT(return
1), INT(return 0));
}
printf("res = %d\n", res);
}
Compiling this using:
gcc -O -Wall man1.c -o man1
I get 0.895s. Using -O2 I get 0.693s. Version without loop, but using >>>> 13 as an argument (as above) takes 0.002s. This is on a fast machine, >>>> on really slow mini-PC I get 10.117s at -O2. I suspect that your time is >>>> due to operationg system, the results above are for Linux on x86_64.
On Linux on 1GHz Risc-V at -O2 I get:
real 0m28.910s
user 0m14.512s
sys 0m14.381s
which means that about half of execution time is in system calls.
Looking at assembly shows that the machine code is calling
'__riscv_flush_icache', that is it flushing cache to make sure
that processor will execute freshly generated instructions
(not needed on PC-s). System calls are expensive, and flushing
cache also adds slowdown, so this adds substantial cost. Still,
the Risc-V machine is quite slow, and can do this much faster
than your estimate of execution time.
For me on fast machine first (emulated) version takes 0.331s when
compiled by gcc at -O2 and 0.767s when compiled by tcc. On Risc-V
emulated version (compiled by gcc -O2) needs 3.553s.
So emulated version is clearly faster, especially if version
using nested functions need support from operating system.
But the point of using nested functions/closures is that code
is simpler.
Well, the code is smaller; that doesn't mean it is simpler! It can be
harder to know what's going on with a cryptic bit of code like this.
(I believe it was Knuth who failed to correctly predict the result of
N=10 (?), at some point in history when computing it was not practical.)
As I wrote below, this test is doing nonsense operation. Most real
uses of higher order functions are simpler (some higher order may
even disregard them as too simple). For example, you want to
perform some operation on all elements of an aggregate. For a list
one can have mostly standard C code like:
typedef struct gen_lst gen_lst;
struct gen_lst {gen_lst * next;};
void
gen_map(void (*f)(gen_lst * lst, void * extra), gen_lst * lst, void * extra) {
for(; lst; lst = lst->next) {
f(lst, extra);
}
}
typedef struct my_lst my_lst;
struct my_lst {gen_lst * next; int val;};
void
adder1(gen_lst * lst, void * extra) {
int * aux = extra;
struct my_lst * ml = (struct my_lst *)lst;
aux += ml->val;
}
int
my_sum(my_lst * lst) {
int aux = 0;
gen_map(adder1, (gen_lst *) lst, (void *)&aux);
return aux;
}
or you can use gcc extention like:
void
gen_map2(void (*f)(gen_lst * lst), gen_lst * lst) {
for(; lst; lst = lst->next) {
f(lst);
}
}
int
my_sum2(my_lst * lst) {
int aux = 0;
void
adder(gen_lst * lst) {
struct my_lst * ml = (struct my_lst *)lst;
aux += ml->val;
}
gen_map2(adder, (gen_lst *) lst);
return aux;
}
Note that argument 'extra' is now gone.
Of course, you my ask why I do computations in this way? If all what
needs to be done is computing sum of elements on a list, than a single
function would do. However, if there are more operations, then
using several looping functions spead knowledge how 'gen_lst' is
build and how to traverse it. Which may be not that bad if you
are sure that it will always be a list and all you need is summing.
But you may need more complex aggregate or multiple kinds of aggregates.
Typically there are several operations one wants to do for each
kind of aggregate. And general part may be much larger than type
specific part. With approach as above you only need to write
type specific operations for each kind of aggregate, general parts
may be reused. And depending on details it may be possible to
share operations for different kinds of aggregates.
I do not know if you believe that this kind of coding is useful,
but I hope that it is clear that variant using nested functions
will always be simpler than variant which passes extra arguments.
TBH I found both the C and extended versions hard-going. (The spacing
either side of "*" didn't help!)
I think here you genuinely need better language features, but intuitive
ones that everyone can get their head around.
For this sort of stuff I'd prefer using a higher level language, not a
lower level one with hairy-looking synyax that has higher order
functions bolted on. For example in mine (which is both dynamic and intepreted) I can sum lists like this:
a := (10,20,30,40)
println reduce(+, a) # shows 100
I think this is typical of scripting languages. In mine however,
'reduce' isn't built-in, I have to implement it in the language itself
like this:
export func reduce(op, a)=
x := head(a)
for y in tail(a) do
x := mapss(op, x, y)
od
x
end
Only 'mapss' is built-in. There is some functional stuff going on here,
but there are no closures in the language.
So under WSL it's 500 times faster. I've no idea why.
I see. It is up to you to decide if you want to investigate deeper.
Linux version generates small pieces of executable code on the
stack. Currently security folks hate such code and want to ban
it. In Linux gcc specially marks executable needing executable
stack and if executable is marked kernel allow executing code
from the stack. Otherwise attempts to execute code from the
stack lead to exception and kernel signals error in such case.
There was some talk to use different method, and code generated
on Windows may be different. One possible implementation would
be to attempt execution on the stack, but also install signal
handler. Signal handler can then effectively interpret was in
on the stack. Clearly this would lead to much worse performance.
I haven't implemented closures in a static language. (I got partway in
the dynamic one, but decided it wasn't worth the effort or extra
complexity just to run silly examples like manboy() or twice().)
Anyway, I wouldn't have expected to have to use executable memory.
I *might* have expected to do whatever the emulated version does, which doesn't need executable memory either!
--However, I work under Windows! So do lots of people. And there, the fact >>> that these exotic features might make programs significantly slower has
to be taken into consideration.
I tried a Python version too. There, 200 iterations took 5.4 seconds on
both OSes. PyPy on Windows was twice as fast.
Once you have interpreted language you get slowdown from the
interpreter, but no extra trouble due executable stack: CPU treats
bytecode as data, so in this way one can bypass security restrictions.
IIUC PyPy may generate machine code, but probably can defer anything
it can not handle to interpreter.
The interesting thing about the interpreted version, is that my
emulation of closures is so much faster than other interpreted languages where they are built-in. Because I have to implement them in bytecode,
but those others can use the underlying C or whatever.
Using LuaJIT, a well-regarded and usually well-performing tracing-JIT product, the 10K x manbody(13) test takes 32 seconds, twice as long as
my own pure interpreter.
(Because the workings are so obscure, I don't know Python/Lua enough to
do emulated versions to see if they are any faster.)
On 11.10.2025 12:25, bart wrote:
On 11/10/2025 01:05, Waldek Hebisch wrote:
[...]
I haven't implemented closures in a static language. (I got partway in
the dynamic one, but decided it wasn't worth the effort or extra
complexity just to run silly examples like manboy() or twice().)
You obviously missed the point.
The function twice() that someone
suggested was a simple example for a _functional composition_, a
principle that (in the more general case) is a fundamental "higher
order" functional principle. The somewhat more general case,
instead of f(f(x)), would be f(g(x)) - without parameter typically
written as f o g - or (f o g)(x) . It's certainly not "silly" (or
only to the folks who didn't understand its purpose here). So the
question was basically whether a language supports [features for]
functional composition or not. Say, with functions as first class
objects, for example something like funct h = f o g; print h(x)
And as that special case funct twice = f o f; print twice(x)
For mathematics - and for abstrated representation in programming
languages' context - it's a fundamental concept and quite useful.
On 08/10/2025 22:21, Waldek Hebisch wrote:<snip>
David Brown <david.brown@hesbynett.no> wrote:
On 05/10/2025 05:15, Lawrence DrCOOliveiro wrote:
On Sun, 5 Oct 2025 01:52:54 +0100, bart wrote:
On 04/10/2025 23:00, Lawrence DrCOOliveiro wrote:
On Sat, 4 Oct 2025 15:07:23 +0100, bart wrote:
In terms of programming, "zero-order functions" would be statements or
expressions. "First-order functions" are normal functions in a language >>> like C or Pascal (and presumably also Algol, though I have little
familiarity with that language). "Second-order functions" or "Higher
order functions" are functions that deal with functions. As Bart said,
these are very common in functional programming languages. But they are >>> also very common in more modern or advanced imperative languages.
Actually, in simplest form they appear quite early in developement
of programming languages. Namely, "deals" may mean various things.
Simplest thing is to call function passed as an argument, that is
very old thing.
That is usually handled by passing a pointer to a function. Pointers
(or references of some kind) to functions are a much simpler concept
than "dealing" with functions themselves.
Some people treat this as too simple and request
ability to create new functions and store them in variables.
In classic Pascal local functions cupture their environment, so
in this sense are new ones. They are limited, because they become
invalid after return from enclosing function. And classic Pascal
does not allow storing functions on variables.
I think that natural defintion of higher order function should
include also simple cases, so functions which just take a function
as an argument and call it should count as higher order functions.
OTOH higher order functions are used as propagande/advertising
term, so people who use this term want to exclude classic usage
and apply it only to more complicated cases. So I understand
why they give different definition, but IMO such definitions are
misleading.
Maybe. It is not unreasonable to say that functions taking parameters
of function pointer type are "higher order". But I would only say that
a language supports higher order functions if it can also return new functions from functions - and assign these to variables and pass them around.
It is not enough to simply support function pointers, such as
C. Under the hood, of course, these returned functions will, in a
compiled language, be handled with structs for closures and pointers to function code. But to be a language that supports higher order
functions, this needs to be hidden in the syntax of the language.
Somewhat contrary to certain other posters, higher order functions are
not "complicated stuff", and language features such as lambdas,
closures, and nested functions do not imply or require higher order
functions - though any language that supports higher order functions
will support these features.
I disagree. AFAICS one can create sensible language which supports
higher order functions but has no closures or nested functions.
For languages that support a limited subset of higher order functions,
that would be true - by the definitions that you have been using, C
would be such a language. But by the time you have a language that can return new functions and assign them to variables, then you have come
most of the way towards closures and nested functions - there is no real difference between a nested function and a lambda assigned to a local variable. So I don't see much point in a language that supports general higher order functions and does not support closures and nested
functions. But perhaps you know of some examples?
For example, C does not support higher order functions - and will still
not do so even if the proposal for introducing lambdas to the language
is accepted. Pascal has nested functions, but not higher order functions. >>>
C++, on the other hand, has lambdas and higher-order functions, but not
nested functions.
You probably can give some argument that C++ supports higher-order
functions, but it is quite unclear to me how you can do this without
acceptiong that C or Pascal support higher-order functions.
template <typename F>
auto do_twice(F f) {
return [f](auto x) {
return f(f(x));
};
}
int add_one(int x) { return x + 1; }
int test_add_two(int x) {
auto add_two = do_twice(add_one);
return add_two(x);
}
You can't write something equivalent to the higher order function
"do_twice" in C or Pascal, taking a function and returning a new one.
Of course C++'s support and syntax here is somewhat suboptimal, compared
to functional programming languages or other modern languages that have
had such features from the start.
To support higher order functions, a language has to be able to handle
functions as first-class objects - it has to be able to take them as
function parameters, and return them as functions. It is not sufficient >>> to take and return function pointers, like C - it has to be something
that is treated in the language as a function.
I strongly disagree with this statement about pointers. Using pointers
to represent functions is usual in C and pretty consistent with making
explicit things that other languages do behind the scenes. Rather,
main limitation in C is that one can only use existing functions, there
is no way to make a new one.
Yes, that is the main limitation of C (in this context) - and the
limitation is because C only supports functions through function
pointers. Technically, C only supports declaring and defining
functions, and taking their address - even when you "call" a function in
C, you are applying the call operator to a function pointer. In order
to support more advanced higher order functions - such as returning new functions - a language must be able to support larger underlying data structures while maintaining the syntax of function calls. It has to
handle captures, closures and/or other details. A "function" is not
merely a pointer to an address in code memory.
Classic Pascal can make new functions,
but one can not legaly return freshly created function, so in this
aspect is only marginally better than C.
It's a good while since I programmed in Pascal, but I was not aware of
any such facility in Pascal. It has nested functions, but that is not
the same thing.
Lambdas in C and C++ are
problematic as they are not usual way to represent functions in C or
C++.
It is correct that lambdas in C++ (C does not have lambdas, though there
is a proposal to add them to the language) are not as simple as normal functions. They are basically an alternative syntax for functors, which
are classes with an overridden call operator.
OTOH a tiny extention to C could give reasonable support. Namely,
while other languages may implicitly allocate heap memory, C requires
explicit calls to 'malloc' (and 'free').
Requiring memory allocation to be explicit is considered a strength of
C. Changing that is not going to go down well. (Lambdas in C++ do not require implicit memory allocation. C++ lambdas and functions that
return new functions have certain visibility limitations in C++ - they cannot cross boundaries of independently compiled units, as each lambda
has a unique type. Having separate compilation and independence while retaining full flexibility is supported in the language and standard library, with std::function<>, but it comes with a considerable overhead
- including heap allocations.)
Similarly, natural way
to create new function in C would be appropriate allocation function,
say called 'falloc' (an matching 'ffree'). 'falloc' should take
pointer to a function 'f', a value 'v' and probably number of arguments
'n' and return poiter 'g' so that call
(*g)(a_1,\dots, a_n)
is equvalent to
(*f)(v, a_1,\dots, a_n)
taking a 'v' a pointer this allows passing arbitrary amount of extra
data to 'f'.
With such extention C would allow reasonably natural expression of
things done in other languages claiming to support higher order functions. >> People using other languages may disregard this as too primitive,
but IMO it would very natural in C.
BTW: people doing functional programming sometimes claim that to
support it one needs garbage collection. I arrived at the idea
above thinking which parts of functional programming can work
well without garbage collection.
Garbage collection is not, AFAIK, necessary for common functional programming techniques. But it might make some of them more convenient
or efficient to use.
On 10/10/2025 16:40, David Brown wrote:
On 08/10/2025 22:21, Waldek Hebisch wrote:
I disagree.-a AFAICS one can create sensible language which supports
higher order functions but has no closures or nested functions.
For languages that support a limited subset of higher order functions,
that would be true - by the definitions that you have been using, C
would be such a language.-a But by the time you have a language that
can return new functions and assign them to variables, then you have
come most of the way towards closures and nested functions - there is
no real difference between a nested function and a lambda assigned to
a local variable.-a So I don't see much point in a language that
supports general higher order functions and does not support closures
and nested functions.-a But perhaps you know of some examples?
There are advantages to having local, nested functions when the
alternative is:
* Defining them as separate, ordinary functions at module level...
* Where their names are now part of the global name space which
-a can clash and can't be reused, or could accidentally be exported
-a from the module when you forget 'static'
* Where those functions could be called from places where they are
-a not intended to be called
* Where they may not be localised to what would be their enclosing
-a function, so could be located anywhere in the module, or even
-a elsewhere
* Losing the close relationship between them
None of that needs closures.
Further, the above applies to C; local
functions can implemented that can refer to useful entities defined in
the enclosing function such as static variables, types, structs and enums.
Closures are only needed for access to stack-frame entities of the
enclosing functions, including parameters.
Of course, you will need closures to run all the silly examples of
higher order functions that abound on the internet.
I've yet to see examples that have real uses and that don't make code
harder to grasp.
I don't however, use local functions; I've never gotten into the habit.
I couldn't even tell you if my languages support them and how well.
When
I tried to run the program below right now, it had trouble resolving
names 'a b vector' from the nested function. I had to put in qualifiers
to make it work.
--------------------------
proc main=
-a-a-a const a = 100
-a-a-a static int b
-a-a-a type vector = [4]int
-a-a-a proc fred(int c) =
-a-a-a-a-a-a-a println "Fred", main.a, main.b, c, main.vector.typestr
-a-a-a end
-a-a-a b := 200
-a-a-a fred(300)
end
Output is:
Fred 100 200 300 [4]i64
David Brown <david.brown@hesbynett.no> wrote:
On 08/10/2025 22:21, Waldek Hebisch wrote:
David Brown <david.brown@hesbynett.no> wrote:
For example, C does not support higher order functions - and will still >>>> not do so even if the proposal for introducing lambdas to the language >>>> is accepted. Pascal has nested functions, but not higher order functions. >>>>
C++, on the other hand, has lambdas and higher-order functions, but not >>>> nested functions.
You probably can give some argument that C++ supports higher-order
functions, but it is quite unclear to me how you can do this without
acceptiong that C or Pascal support higher-order functions.
template <typename F>
auto do_twice(F f) {
return [f](auto x) {
return f(f(x));
};
}
int add_one(int x) { return x + 1; }
int test_add_two(int x) {
auto add_two = do_twice(add_one);
return add_two(x);
}
You can't write something equivalent to the higher order function
"do_twice" in C or Pascal, taking a function and returning a new one.
But 'do_twice' returns a lambda, not a function. You can not pass
result of 'do_twice' to a function expecting normal C++ function
(pointer) as an argument. If what you return does not need to be
a function, than you can do equvalent in C: return approproate
structure and provide "call" operation. Only advantage of C++
here is that in C++ one can overload call, so syntatically this
looks like a function call. But semantially it is not a function
call (because called thing is not a function).
Of course C++'s support and syntax here is somewhat suboptimal, compared
to functional programming languages or other modern languages that have
had such features from the start.
To support higher order functions, a language has to be able to handle >>>> functions as first-class objects - it has to be able to take them as
function parameters, and return them as functions. It is not sufficient >>>> to take and return function pointers, like C - it has to be something
that is treated in the language as a function.
I strongly disagree with this statement about pointers. Using pointers
to represent functions is usual in C and pretty consistent with making
explicit things that other languages do behind the scenes. Rather,
main limitation in C is that one can only use existing functions, there
is no way to make a new one.
Yes, that is the main limitation of C (in this context) - and the
limitation is because C only supports functions through function
pointers. Technically, C only supports declaring and defining
functions, and taking their address - even when you "call" a function in
C, you are applying the call operator to a function pointer. In order
to support more advanced higher order functions - such as returning new
functions - a language must be able to support larger underlying data
structures while maintaining the syntax of function calls. It has to
handle captures, closures and/or other details. A "function" is not
merely a pointer to an address in code memory.
When using trampolines closure is a pointer to an address of
freshly generated code. Note that on some OS-es (IIUC AIX and
Alpha/IA64 VMS) C function is a pointer to descriptor. Descriptor
contains both pointer to code and pointer to environment. IIUC VMS
on x86_64 decided to use trampolines. IIUC on all architectures
you can pass closures to VMS C and VMS C will call them correctly.
So fact that typical C implementation uses address of code is
just implementation datail.
BTW: people doing functional programming sometimes claim that to
support it one needs garbage collection. I arrived at the idea
above thinking which parts of functional programming can work
well without garbage collection.
Garbage collection is not, AFAIK, necessary for common functional
programming techniques. But it might make some of them more convenient
or efficient to use.
Some functional techniques work well without garbage collection.
But some basically loose all advantates without garbage collection
or equivalent automatic technique. In particular, passing
complex dynamically allocated data structures trough chain of
calls becomes problematic due to need for deallocation.
On 10/10/2025 21:04, bart wrote:
There was a discussion elsewhere about gcc's handling of nested
functions sometimes needing an executable stack, or other complications.
-aThis occurs when you have a nested function that has a closure - thus needing a hidden struct pointer - but where it needs to be passed
somewhere as though it were a pointer to a function without this extra parameter.-a One way to handle this is with a small function created directly on the stack.
Let me try to describe a possible real-world use, using an imaginary language to avoid any bias or complicating things with real-world syntax details.
You have a marvellous string sorting function, taking an array of
strings and a comparison function as a parameter :
-a-a-a-asort_strings(string s[], function comp(string, string))
You can use this with comparison functions that are case-sensitive, reverse-order case-insensitive, or whatever you like.
-a-a-a-adefine make_sort_func(string country) :
-a-a-a-a-a-a-a sort_func = lambda(string s1, string s2) :
-a-a-a-a-a-a-a-a-a-a-a return compare_names(s1, s2, country)
-a-a-a-a-a-a-a return sort_func
Now you can do your sorting with :
-a-a-a-adefine sort_names(string s[], string country) :
-a-a-a-a-a-a-a sort_strings(s[], make_sort_func(country))
Somehow, you have got to create a function that compares two strings,
with that function depending on an independent input (the country). That
is a closure - you've made a new function that is based on the three- parameter compare_names function and a bit of captured data. Whether you want to do that with a lambda (an anonymous function), a named lambda, a function that returns a new function, or a local function, is just a
matter of style and taste (assuming the language supports the feature).
Of course there are always ways to avoid this - such as making your
original "sort_strings" function take a "void *" parameter that is
passed to the comparison function as an environment, or making an array
of sort functions indexed by country.-a But I hope that example shows
that closures and higher-order functions can be useful in simple
examples that are not purely theoretical, and that it is clear what the
code above is doing.
I couldn't even tell you if my languages support them and how well.
That is a rather interesting thing to say!-a People often don't know all
the details of languages they use, but /usually/ the authors of those languages have a fair idea of what they can do :-)-a (Yes, I know that
some of your languages were made a long time ago.)
On 13/10/2025 19:58, David Brown wrote:
On 10/10/2025 21:04, bart wrote:
There was a discussion elsewhere about gcc's handling of nested
functions sometimes needing an executable stack, or other
complications. -a-aThis occurs when you have a nested function that has
a closure - thus needing a hidden struct pointer - but where it needs
to be passed somewhere as though it were a pointer to a function
without this extra parameter.-a One way to handle this is with a small
function created directly on the stack.
Thanks for clearing that up. I think I would have simply disallowed such uses.
Why should a special function with a hidden extra parameter be
able to masquerade as an ordinary function? The same point comes up
later on.
Let the user make their own arrangements for this.
(This is actually a similar problem to what happens when an interpreted language has to pass a function reference via an FFI to be used as a callback function.
Obviously, native code functions can't directly call a function pointer
that refers to some byte code rather than machine instructions. There
needs to be something inbetween.
I've never fully solved that for the general case. But I had in mind a
fixed number of predefined, intermediate, native code functions that sit between the external library, and a special reentry point to the interpreter.
I /think/ it can be done with data rather than new executable code.)
Let me try to describe a possible real-world use, using an imaginary
language to avoid any bias or complicating things with real-world
syntax details.
You have a marvellous string sorting function, taking an array of
strings and a comparison function as a parameter :
-a-a-a-a-asort_strings(string s[], function comp(string, string))
You can use this with comparison functions that are case-sensitive,
reverse-order case-insensitive, or whatever you like.
-a-a-a-a-adefine make_sort_func(string country) :
-a-a-a-a-a-a-a-a sort_func = lambda(string s1, string s2) :
-a-a-a-a-a-a-a-a-a-a-a-a return compare_names(s1, s2, country)
-a-a-a-a-a-a-a-a return sort_func
Now you can do your sorting with :
-a-a-a-a-adefine sort_names(string s[], string country) :
-a-a-a-a-a-a-a-a sort_strings(s[], make_sort_func(country))
Somehow, you have got to create a function that compares two strings,
with that function depending on an independent input (the country).
That is a closure - you've made a new function that is based on the
three- parameter compare_names function and a bit of captured data.
Whether you want to do that with a lambda (an anonymous function), a
named lambda, a function that returns a new function, or a local
function, is just a matter of style and taste (assuming the language
supports the feature).
Of course there are always ways to avoid this - such as making your
original "sort_strings" function take a "void *" parameter that is
passed to the comparison function as an environment, or making an
array of sort functions indexed by country.-a But I hope that example
shows that closures and higher-order functions can be useful in simple
examples that are not purely theoretical, and that it is clear what
the code above is doing.
You've used static typing for your examples: the sort function clearly
takes two arguments, not optionally an extra argument!
So, is this an example where a 'trampoline' is needed, to pretend your
true sort function is a simple 2-operand one? I don't think this is
going to be help performance any, not on Windows (I understand Linux
does something special here).
But as you say, this example isn't that compelling, as there are any
number of workarounds.
I couldn't even tell you if my languages support them and how well.
That is a rather interesting thing to say!-a People often don't know
all the details of languages they use, but /usually/ the authors of
those languages have a fair idea of what they can do :-)-a (Yes, I know
that some of your languages were made a long time ago.)
This can happen with little-used features, or ones you've forgotten
about or never used. You tend to find them when overhauling code.
However I think I first had nested functions by accident, by neglecting
to disallow them.
On 12/10/2025 05:56, Waldek Hebisch wrote:
David Brown <david.brown@hesbynett.no> wrote:
On 08/10/2025 22:21, Waldek Hebisch wrote:
David Brown <david.brown@hesbynett.no> wrote:
To support higher order functions, a language has to be able to handle >>>>> functions as first-class objects - it has to be able to take them as >>>>> function parameters, and return them as functions. It is not sufficient >>>>> to take and return function pointers, like C - it has to be something >>>>> that is treated in the language as a function.
I strongly disagree with this statement about pointers. Using pointers >>>> to represent functions is usual in C and pretty consistent with making >>>> explicit things that other languages do behind the scenes. Rather,
main limitation in C is that one can only use existing functions, there >>>> is no way to make a new one.
Yes, that is the main limitation of C (in this context) - and the
limitation is because C only supports functions through function
pointers. Technically, C only supports declaring and defining
functions, and taking their address - even when you "call" a function in >>> C, you are applying the call operator to a function pointer. In order
to support more advanced higher order functions - such as returning new
functions - a language must be able to support larger underlying data
structures while maintaining the syntax of function calls. It has to
handle captures, closures and/or other details. A "function" is not
merely a pointer to an address in code memory.
When using trampolines closure is a pointer to an address of
freshly generated code. Note that on some OS-es (IIUC AIX and
Alpha/IA64 VMS) C function is a pointer to descriptor. Descriptor
contains both pointer to code and pointer to environment. IIUC VMS
on x86_64 decided to use trampolines. IIUC on all architectures
you can pass closures to VMS C and VMS C will call them correctly.
So fact that typical C implementation uses address of code is
just implementation datail.
It's a question of efficiency. If some of your "function pointers" are direct pointers to code, and some are pointers to a descriptor, then whenever you are calling through a "function pointer" you need code that checks what style of pointer you have, and acts accordingly. This may
be significant overhead compared to a straight indirect jump.
Trampolines on the stack avoid that overhead, but have the cost of
requiring an executable stack.
On 11/10/2025 02:42, Lawrence DrCOOliveiro wrote:
On Sat, 11 Oct 2025 01:59:28 +0100, bart wrote:
I'm not quite clear what you code does, or whether I would need to
do the same.
You said you couldnrCOt see a need for this sort of functionality.
This is an example of this sort of functionality.
You said you used some complicated method to wrap how you access
C-style struct info. I don't know what code would look like before
and after you did that. Whatever it does, I'm not convinced your
approach is the simplest.
It didn't do anything interesting beyond printing 'hello'.
| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 76 |
| Nodes: | 6 (0 / 6) |
| Uptime: | 53:58:29 |
| Calls: | 1,265 |
| Calls today: | 1 |
| Files: | 1,362 |
| D/L today: |
3 files (2,684K bytes) |
| Messages: | 285,448 |