• Re: Algol 68 / Genie - new release

    From Lawrence =?iso-8859-13?q?D=FFOliveiro?=@ldo@nz.invalid to comp.lang.misc on Fri Oct 10 21:45:09 2025
    From Newsgroup: comp.lang.misc

    On Fri, 10 Oct 2025 20:04:14 +0100, bart wrote:

    I've yet to see examples [of closures] that have real uses and that
    don't make code harder to grasp.

    I used a class factory in my Python wrapper for Cairo. It was
    convenient to define higher-level Python wrappers around lower-level C
    structs, where there was a lot of commonality across those different
    wrappers, to save having multiple copies of the same code in the
    source. Thus, defining a rCLFontExtentsrCY wrapper class around the rCLfont_extents_trCY struct became as simple as

    FontExtents = def_struct_class \
    (
    name = "FontExtents",
    ctname = "font_extents_t"
    )

    And rCLTextExtentsrCY as a wrapper around rCLtext_extents_trCY; but here it became convenient to add extra attributes, to return entire Rect and
    Vector components as single objects, where the C code had the coordinates
    as separate fields. So I define the derived attributes in an additional temporary helper class:

    class TextExtentsExtra :
    # extra members for TextExtents class.

    @property
    def bounds(self) :
    "returns the bounds of the text_extents as a Rect."
    return \
    Rect(self.x_bearing, self.y_bearing, self.width, self.height)
    #end bounds

    @property
    def advance(self) :
    "returns the x- and y-advance of the text_extents as a Vector."
    return \
    Vector(self.x_advance, self.y_advance)
    #end advance

    #end TextExtentsExtra

    and then merge that into the wrapper:

    TextExtents = def_struct_class \
    (
    name = "TextExtents",
    ctname = "text_extents_t",
    extra = TextExtentsExtra
    )
    del TextExtentsExtra

    The definition of rCLdef_struct_classrCY is 79 lines of code. For more
    details, see <https://gitlab.com/ldo/qahirah>.

    This is a comparatively simple example; I have more complex ones
    elsewhere, if you want them.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From bart@bc@freeuk.com to comp.lang.misc on Sat Oct 11 00:29:55 2025
    From Newsgroup: comp.lang.misc

    On 10/10/2025 22:45, Lawrence DrCOOliveiro wrote:
    On Fri, 10 Oct 2025 20:04:14 +0100, bart wrote:

    I've yet to see examples [of closures] that have real uses and that
    don't make code harder to grasp.

    I used a class factory in my Python wrapper for Cairo. It was
    convenient to define higher-level Python wrappers around lower-level C structs, where there was a lot of commonality across those different wrappers, to save having multiple copies of the same code in the
    source. Thus, defining a rCLFontExtentsrCY wrapper class around the rCLfont_extents_trCY struct became as simple as

    FontExtents = def_struct_class \
    (
    name = "FontExtents",
    ctname = "font_extents_t"
    )

    And rCLTextExtentsrCY as a wrapper around rCLtext_extents_trCY; but here it became convenient to add extra attributes, to return entire Rect and
    Vector components as single objects, where the C code had the coordinates
    as separate fields. So I define the derived attributes in an additional temporary helper class:

    class TextExtentsExtra :
    # extra members for TextExtents class.

    @property
    def bounds(self) :
    "returns the bounds of the text_extents as a Rect."
    return \
    Rect(self.x_bearing, self.y_bearing, self.width, self.height)
    #end bounds

    @property
    def advance(self) :
    "returns the x- and y-advance of the text_extents as a Vector."
    return \
    Vector(self.x_advance, self.y_advance)
    #end advance

    #end TextExtentsExtra

    and then merge that into the wrapper:

    TextExtents = def_struct_class \
    (
    name = "TextExtents",
    ctname = "text_extents_t",
    extra = TextExtentsExtra
    )
    del TextExtentsExtra

    The definition of rCLdef_struct_classrCY is 79 lines of code. For more details, see <https://gitlab.com/ldo/qahirah>.

    This is a comparatively simple example; I have more complex ones
    elsewhere, if you want them.

    I can't see any use of closures above, and I can't find those 79 lines
    of code in that link. Not that it would help.

    Wrapping functions is not hard and I can't see why higher order
    functions are needed.

    But I've found in the past with languages like Python and C++ which
    offer a plethora of advanced features, that people will use them because they're there. No matter if the code becomes more cryptic. Or
    considerbly less efficient with all those layers.

    Sometimes I want to port algorithms, or benchmarks, expressed in such language, but use of exotic features can make that impossible.


    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From antispam@antispam@fricas.org (Waldek Hebisch) to comp.lang.misc on Sat Oct 11 00:05:02 2025
    From Newsgroup: comp.lang.misc

    bart <bc@freeuk.com> wrote:
    On 10/10/2025 05:18, Waldek Hebisch wrote:
    bart <bc@freeuk.com> wrote:
    <snip>

    Last week I posted a link to a 'man or boy' test; this a version in C:

    https://rosettacode.org/wiki/Man_or_boy_test#C

    This test requires local functions with full closures to implement
    natively. The first C version emulates what is needed.

    I tweaked that version so that is used N=13 instead of N=10, and
    evaluates the function 10,000 times. The second version uses C
    extensions to provide the higher level functionality.

    These are timings I got for the two versions:

    First C version (gcc -O2: 0.8 seconds
    (Tiny C): 1.8 seconds
    Second C version (gcc -O2): 590 seconds (extrapolated from 1K loops) >>> (gcc -O0): 940 seconds

    So avoiding those complex functional features makes even TCC's poor code >>> 300 times that optimised C. And nearly 800:1 better comparing -O2 with -O2. >>
    It is not clear to me what you measure. I used:

    #include <stdio.h>
    #define INT(body) ({ int lambda(){ body; }; lambda; })
    int
    main(){
    int a(int k, int xl(), int x2(), int x3(), int x4(), int x5()){
    int b(){
    return a(--k, b, xl, x2, x3, x4);
    }
    return k<=0 ? x4() + x5() : b();
    }
    int res = 0;
    int i;
    for(i = 0; i < 10000; i++) {
    res += a(13, INT(return 1), INT(return -1), INT(return -1), INT(return
    1), INT(return 0));
    }
    printf("res = %d\n", res);
    }

    Compiling this using:

    gcc -O -Wall man1.c -o man1

    I get 0.895s. Using -O2 I get 0.693s. Version without loop, but using
    13 as an argument (as above) takes 0.002s. This is on a fast machine,
    on really slow mini-PC I get 10.117s at -O2. I suspect that your time is
    due to operationg system, the results above are for Linux on x86_64.
    On Linux on 1GHz Risc-V at -O2 I get:

    real 0m28.910s
    user 0m14.512s
    sys 0m14.381s

    which means that about half of execution time is in system calls.
    Looking at assembly shows that the machine code is calling
    '__riscv_flush_icache', that is it flushing cache to make sure
    that processor will execute freshly generated instructions
    (not needed on PC-s). System calls are expensive, and flushing
    cache also adds slowdown, so this adds substantial cost. Still,
    the Risc-V machine is quite slow, and can do this much faster
    than your estimate of execution time.

    For me on fast machine first (emulated) version takes 0.331s when
    compiled by gcc at -O2 and 0.767s when compiled by tcc. On Risc-V
    emulated version (compiled by gcc -O2) needs 3.553s.

    So emulated version is clearly faster, especially if version
    using nested functions need support from operating system.

    But the point of using nested functions/closures is that code
    is simpler.

    Well, the code is smaller; that doesn't mean it is simpler! It can be
    harder to know what's going on with a cryptic bit of code like this.

    (I believe it was Knuth who failed to correctly predict the result of
    N=10 (?), at some point in history when computing it was not practical.)

    As I wrote below, this test is doing nonsense operation. Most real
    uses of higher order functions are simpler (some higher order may
    even disregard them as too simple). For example, you want to
    perform some operation on all elements of an aggregate. For a list
    one can have mostly standard C code like:

    typedef struct gen_lst gen_lst;
    struct gen_lst {gen_lst * next;};

    void
    gen_map(void (*f)(gen_lst * lst, void * extra), gen_lst * lst, void * extra) {
    for(; lst; lst = lst->next) {
    f(lst, extra);
    }
    }

    typedef struct my_lst my_lst;
    struct my_lst {gen_lst * next; int val;};

    void
    adder1(gen_lst * lst, void * extra) {
    int * aux = extra;
    struct my_lst * ml = (struct my_lst *)lst;
    aux += ml->val;
    }

    int
    my_sum(my_lst * lst) {
    int aux = 0;
    gen_map(adder1, (gen_lst *) lst, (void *)&aux);
    return aux;
    }

    or you can use gcc extention like:

    void
    gen_map2(void (*f)(gen_lst * lst), gen_lst * lst) {
    for(; lst; lst = lst->next) {
    f(lst);
    }
    }

    int
    my_sum2(my_lst * lst) {
    int aux = 0;
    void
    adder(gen_lst * lst) {
    struct my_lst * ml = (struct my_lst *)lst;
    aux += ml->val;
    }
    gen_map2(adder, (gen_lst *) lst);
    return aux;
    }

    Note that argument 'extra' is now gone.

    Of course, you my ask why I do computations in this way? If all what
    needs to be done is computing sum of elements on a list, than a single
    function would do. However, if there are more operations, then
    using several looping functions spead knowledge how 'gen_lst' is
    build and how to traverse it. Which may be not that bad if you
    are sure that it will always be a list and all you need is summing.
    But you may need more complex aggregate or multiple kinds of aggregates. Typically there are several operations one wants to do for each
    kind of aggregate. And general part may be much larger than type
    specific part. With approach as above you only need to write
    type specific operations for each kind of aggregate, general parts
    may be reused. And depending on details it may be possible to
    share operations for different kinds of aggregates.

    I do not know if you believe that this kind of coding is useful,
    but I hope that it is clear that variant using nested functions
    will always be simpler than variant which passes extra arguments.

    The test is doing nonsense operation, so one
    can miss simplicty, but clearly emulated version is longer.
    More important, in bigger program all involved calls must be
    dane in nonstandard way, which adds work (potentially quite a lot).
    In version with nested functions only efort is in definition
    of nested functions, calls are standard calls and rest of program
    can be written as if there were no use of nested functions.

    As you can see, when properly supported extra cost of nested
    functions is moderate.

    BTW: Some language implementation use approach which is
    equivalent to emulated version. For heavy use of nested
    functions this may lead to faster code (like in this example).
    But when "emulation" is part of language implementation
    all calls would need to do extra work, slowing down normall
    calls. In language like C this is deemed unacceptable,
    nested function are rarely use so they must pay extra,
    while normal call work at maximal speed.


    I'm testing the version show below (I added the += to make it match
    yours). Results on Windows are:

    c:\c>gc d opt
    Invoking: gcc -s -O2 d.c -o d.exe -lm -ldl -fno-strict-aliasing
    Compiled d.c to d.exe

    c:\c>tm d
    -64200

    TM: 6.01

    So 6 seconds to do 100 iterations. If I try it in WSL, using 10000 iterations, then I get:

    c:\c>wsl
    root@DESKTOP-11:/mnt/c/c# gcc -O2 d.c -od
    root@DESKTOP-11:/mnt/c/c# time ./d
    -6420000

    real 0m1.184s
    user 0m1.180s
    sys 0m0.001s

    So under WSL it's 500 times faster. I've no idea why.

    I see. It is up to you to decide if you want to investigate deeper.
    Linux version generates small pieces of executable code on the
    stack. Currently security folks hate such code and want to ban
    it. In Linux gcc specially marks executable needing executable
    stack and if executable is marked kernel allow executing code
    from the stack. Otherwise attempts to execute code from the
    stack lead to exception and kernel signals error in such case.
    There was some talk to use different method, and code generated
    on Windows may be different. One possible implementation would
    be to attempt execution on the stack, but also install signal
    handler. Signal handler can then effectively interpret was in
    on the stack. Clearly this would lead to much worse performance.

    However, I work under Windows! So do lots of people. And there, the fact that these exotic features might make programs significantly slower has
    to be taken into consideration.

    I tried a Python version too. There, 200 iterations took 5.4 seconds on
    both OSes. PyPy on Windows was twice as fast.

    Once you have interpreted language you get slowdown from the
    interpreter, but no extra trouble due executable stack: CPU treats
    bytecode as data, so in this way one can bypass security restrictions.
    IIUC PyPy may generate machine code, but probably can defer anything
    it can not handle to interpreter.

    Similarly, if language have garbage collection, then one can
    create functions in special part of the heap. During function
    creation needed part of heap is marked as writable and non-executable,
    once the function is created corresponding part of the heap is marked
    as non-writeable and executable satisfaing security demands at
    cost of 2 system calls per function created. When no longer
    needed function is garbage collected in usual way. Note that
    stack normally must be writable all the time, so with code on
    the stack there is risk of executing malicious data.
    --
    Waldek Hebisch
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Lawrence =?iso-8859-13?q?D=FFOliveiro?=@ldo@nz.invalid to comp.lang.misc on Sat Oct 11 00:16:02 2025
    From Newsgroup: comp.lang.misc

    On Sat, 11 Oct 2025 00:29:55 +0100, bart wrote:

    On 10/10/2025 22:45, Lawrence DrCOOliveiro wrote:

    The definition of rCLdef_struct_classrCY is 79 lines of code. For more
    details, see <https://gitlab.com/ldo/qahirah>.

    I can't see any use of closures above, and I can't find those 79 lines
    of code in that link. Not that it would help.

    ThatrCOs where closures are used. I could copy and paste the whole thing, if you like.

    But I've found in the past with languages like Python and C++ which
    offer a plethora of advanced features, that people will use them because they're there.

    You could prove your point by doing what my code does more simply, in C
    say, without using that rCLplethora of advanced featuresrCY. But you canrCOt manage that, can you?
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From bart@bc@freeuk.com to comp.lang.misc on Sat Oct 11 01:59:28 2025
    From Newsgroup: comp.lang.misc

    On 11/10/2025 01:16, Lawrence DrCOOliveiro wrote:
    On Sat, 11 Oct 2025 00:29:55 +0100, bart wrote:

    On 10/10/2025 22:45, Lawrence DrCOOliveiro wrote:

    The definition of rCLdef_struct_classrCY is 79 lines of code. For more
    details, see <https://gitlab.com/ldo/qahirah>.

    I can't see any use of closures above, and I can't find those 79 lines
    of code in that link. Not that it would help.

    ThatrCOs where closures are used. I could copy and paste the whole thing, if you like.

    But I've found in the past with languages like Python and C++ which
    offer a plethora of advanced features, that people will use them because
    they're there.

    You could prove your point by doing what my code does more simply, in C
    say, without using that rCLplethora of advanced featuresrCY. But you canrCOt manage that, can you?

    I'm not quite clear what you code does, or whether I would need to do
    the same.

    Because I implement actual languages and can add any features I like, it
    could be that I don't need it.

    Whatever it is, I can't say it's something I've missed in quite a few
    decades of coding!

    I see that the stuff you posted decorators. That is a feature where,
    everytime I look up what it does, I've forgotten it a few minutes later.

    I need simple, solid, intuitive features that anyone can understand and
    use confidenly. Or if necessary, implement a different way.

    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Lawrence =?iso-8859-13?q?D=FFOliveiro?=@ldo@nz.invalid to comp.lang.misc on Sat Oct 11 01:42:33 2025
    From Newsgroup: comp.lang.misc

    On Sat, 11 Oct 2025 01:59:28 +0100, bart wrote:

    I'm not quite clear what you code does, or whether I would need to do
    the same.

    You said you couldnrCOt see a need for this sort of functionality. This
    is an example of this sort of functionality.

    I need simple, solid, intuitive features that anyone can understand
    and use confidenly. Or if necessary, implement a different way.

    Hint:

    def def_struct_class(name, ctname, extra = None) :
    # defines a class with attributes that are a straightforward mapping
    # of a ctypes struct. Optionally includes extra members from extra
    # if specified.

    ctstruct = getattr(CAIRO, ctname)

    class result_class :

    ...

    #end def_struct_class

    Can you see the closure yet?
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From bart@bc@freeuk.com to comp.lang.misc on Sat Oct 11 10:42:49 2025
    From Newsgroup: comp.lang.misc

    On 11/10/2025 02:42, Lawrence DrCOOliveiro wrote:
    On Sat, 11 Oct 2025 01:59:28 +0100, bart wrote:

    I'm not quite clear what you code does, or whether I would need to do
    the same.

    You said you couldnrCOt see a need for this sort of functionality. This
    is an example of this sort of functionality.

    You said you used some complicated method to wrap how you access C-style struct info. I don't know what code would look like before and after you
    did that. Whatever it does, I'm not convinced your approach is the simplest.


    I need simple, solid, intuitive features that anyone can understand
    and use confidenly. Or if necessary, implement a different way.

    Hint:

    def def_struct_class(name, ctname, extra = None) :
    # defines a class with attributes that are a straightforward mapping
    # of a ctypes struct. Optionally includes extra members from extra
    # if specified.

    ctstruct = getattr(CAIRO, ctname)

    class result_class :

    ...

    #end def_struct_class

    Can you see the closure yet?

    No. I ran this:

    class fred:
    xyz="hello"

    def def_struct_class(name, ctname, extra = None) :
    ctstruct = getattr(fred, ctname)

    print(ctstruct)

    def_struct_class("a", "xyz")

    It didn't do anything interesting beyond printing 'hello'.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From bart@bc@freeuk.com to comp.lang.misc on Sat Oct 11 11:25:01 2025
    From Newsgroup: comp.lang.misc

    On 11/10/2025 01:05, Waldek Hebisch wrote:
    bart <bc@freeuk.com> wrote:
    On 10/10/2025 05:18, Waldek Hebisch wrote:
    bart <bc@freeuk.com> wrote:
    <snip>

    Last week I posted a link to a 'man or boy' test; this a version in C: >>>>
    https://rosettacode.org/wiki/Man_or_boy_test#C

    This test requires local functions with full closures to implement
    natively. The first C version emulates what is needed.

    I tweaked that version so that is used N=13 instead of N=10, and
    evaluates the function 10,000 times. The second version uses C
    extensions to provide the higher level functionality.

    These are timings I got for the two versions:

    First C version (gcc -O2: 0.8 seconds
    (Tiny C): 1.8 seconds
    Second C version (gcc -O2): 590 seconds (extrapolated from 1K loops) >>>> (gcc -O0): 940 seconds

    So avoiding those complex functional features makes even TCC's poor code >>>> 300 times that optimised C. And nearly 800:1 better comparing -O2 with -O2.

    It is not clear to me what you measure. I used:

    #include <stdio.h>
    #define INT(body) ({ int lambda(){ body; }; lambda; })
    int
    main(){
    int a(int k, int xl(), int x2(), int x3(), int x4(), int x5()){
    int b(){
    return a(--k, b, xl, x2, x3, x4);
    }
    return k<=0 ? x4() + x5() : b();
    }
    int res = 0;
    int i;
    for(i = 0; i < 10000; i++) {
    res += a(13, INT(return 1), INT(return -1), INT(return -1), INT(return
    1), INT(return 0));
    }
    printf("res = %d\n", res);
    }

    Compiling this using:

    gcc -O -Wall man1.c -o man1

    I get 0.895s. Using -O2 I get 0.693s. Version without loop, but using
    13 as an argument (as above) takes 0.002s. This is on a fast machine,
    on really slow mini-PC I get 10.117s at -O2. I suspect that your time is >>> due to operationg system, the results above are for Linux on x86_64.
    On Linux on 1GHz Risc-V at -O2 I get:

    real 0m28.910s
    user 0m14.512s
    sys 0m14.381s

    which means that about half of execution time is in system calls.
    Looking at assembly shows that the machine code is calling
    '__riscv_flush_icache', that is it flushing cache to make sure
    that processor will execute freshly generated instructions
    (not needed on PC-s). System calls are expensive, and flushing
    cache also adds slowdown, so this adds substantial cost. Still,
    the Risc-V machine is quite slow, and can do this much faster
    than your estimate of execution time.

    For me on fast machine first (emulated) version takes 0.331s when
    compiled by gcc at -O2 and 0.767s when compiled by tcc. On Risc-V
    emulated version (compiled by gcc -O2) needs 3.553s.

    So emulated version is clearly faster, especially if version
    using nested functions need support from operating system.

    But the point of using nested functions/closures is that code
    is simpler.

    Well, the code is smaller; that doesn't mean it is simpler! It can be
    harder to know what's going on with a cryptic bit of code like this.

    (I believe it was Knuth who failed to correctly predict the result of
    N=10 (?), at some point in history when computing it was not practical.)

    As I wrote below, this test is doing nonsense operation. Most real
    uses of higher order functions are simpler (some higher order may
    even disregard them as too simple). For example, you want to
    perform some operation on all elements of an aggregate. For a list
    one can have mostly standard C code like:

    typedef struct gen_lst gen_lst;
    struct gen_lst {gen_lst * next;};

    void
    gen_map(void (*f)(gen_lst * lst, void * extra), gen_lst * lst, void * extra) {
    for(; lst; lst = lst->next) {
    f(lst, extra);
    }
    }

    typedef struct my_lst my_lst;
    struct my_lst {gen_lst * next; int val;};

    void
    adder1(gen_lst * lst, void * extra) {
    int * aux = extra;
    struct my_lst * ml = (struct my_lst *)lst;
    aux += ml->val;
    }

    int
    my_sum(my_lst * lst) {
    int aux = 0;
    gen_map(adder1, (gen_lst *) lst, (void *)&aux);
    return aux;
    }

    or you can use gcc extention like:

    void
    gen_map2(void (*f)(gen_lst * lst), gen_lst * lst) {
    for(; lst; lst = lst->next) {
    f(lst);
    }
    }

    int
    my_sum2(my_lst * lst) {
    int aux = 0;
    void
    adder(gen_lst * lst) {
    struct my_lst * ml = (struct my_lst *)lst;
    aux += ml->val;
    }
    gen_map2(adder, (gen_lst *) lst);
    return aux;
    }

    Note that argument 'extra' is now gone.

    Of course, you my ask why I do computations in this way? If all what
    needs to be done is computing sum of elements on a list, than a single function would do. However, if there are more operations, then
    using several looping functions spead knowledge how 'gen_lst' is
    build and how to traverse it. Which may be not that bad if you
    are sure that it will always be a list and all you need is summing.
    But you may need more complex aggregate or multiple kinds of aggregates. Typically there are several operations one wants to do for each
    kind of aggregate. And general part may be much larger than type
    specific part. With approach as above you only need to write
    type specific operations for each kind of aggregate, general parts
    may be reused. And depending on details it may be possible to
    share operations for different kinds of aggregates.

    I do not know if you believe that this kind of coding is useful,
    but I hope that it is clear that variant using nested functions
    will always be simpler than variant which passes extra arguments.

    TBH I found both the C and extended versions hard-going. (The spacing
    either side of "*" didn't help!)

    I think here you genuinely need better language features, but intuitive
    ones that everyone can get their head around.

    For this sort of stuff I'd prefer using a higher level language, not a
    lower level one with hairy-looking synyax that has higher order
    functions bolted on. For example in mine (which is both dynamic and intepreted) I can sum lists like this:

    a := (10,20,30,40)
    println reduce(+, a) # shows 100

    I think this is typical of scripting languages. In mine however,
    'reduce' isn't built-in, I have to implement it in the language itself
    like this:

    export func reduce(op, a)=
    x := head(a)
    for y in tail(a) do
    x := mapss(op, x, y)
    od
    x
    end

    Only 'mapss' is built-in. There is some functional stuff going on here,
    but there are no closures in the language.


    So under WSL it's 500 times faster. I've no idea why.

    I see. It is up to you to decide if you want to investigate deeper.
    Linux version generates small pieces of executable code on the
    stack. Currently security folks hate such code and want to ban
    it. In Linux gcc specially marks executable needing executable
    stack and if executable is marked kernel allow executing code
    from the stack. Otherwise attempts to execute code from the
    stack lead to exception and kernel signals error in such case.
    There was some talk to use different method, and code generated
    on Windows may be different. One possible implementation would
    be to attempt execution on the stack, but also install signal
    handler. Signal handler can then effectively interpret was in
    on the stack. Clearly this would lead to much worse performance.

    I haven't implemented closures in a static language. (I got partway in
    the dynamic one, but decided it wasn't worth the effort or extra
    complexity just to run silly examples like manboy() or twice().)

    Anyway, I wouldn't have expected to have to use executable memory.

    I *might* have expected to do whatever the emulated version does, which doesn't need executable memory either!


    However, I work under Windows! So do lots of people. And there, the fact
    that these exotic features might make programs significantly slower has
    to be taken into consideration.

    I tried a Python version too. There, 200 iterations took 5.4 seconds on
    both OSes. PyPy on Windows was twice as fast.

    Once you have interpreted language you get slowdown from the
    interpreter, but no extra trouble due executable stack: CPU treats
    bytecode as data, so in this way one can bypass security restrictions.
    IIUC PyPy may generate machine code, but probably can defer anything
    it can not handle to interpreter.

    The interesting thing about the interpreted version, is that my
    emulation of closures is so much faster than other interpreted languages
    where they are built-in. Because I have to implement them in bytecode,
    but those others can use the underlying C or whatever.

    Using LuaJIT, a well-regarded and usually well-performing tracing-JIT
    product, the 10K x manbody(13) test takes 32 seconds, twice as long as
    my own pure interpreter.

    (Because the workings are so obscure, I don't know Python/Lua enough to
    do emulated versions to see if they are any faster.)
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Janis Papanagnou@janis_papanagnou+ng@hotmail.com to comp.lang.misc on Sat Oct 11 13:08:49 2025
    From Newsgroup: comp.lang.misc

    On 11.10.2025 12:25, bart wrote:
    On 11/10/2025 01:05, Waldek Hebisch wrote:
    [...]

    I haven't implemented closures in a static language. (I got partway in
    the dynamic one, but decided it wasn't worth the effort or extra
    complexity just to run silly examples like manboy() or twice().)

    You obviously missed the point. The function twice() that someone
    suggested was a simple example for a _functional composition_, a
    principle that (in the more general case) is a fundamental "higher
    order" functional principle. The somewhat more general case,
    instead of f(f(x)), would be f(g(x)) - without parameter typically
    written as f o g - or (f o g)(x) . It's certainly not "silly" (or
    only to the folks who didn't understand its purpose here). So the
    question was basically whether a language supports [features for]
    functional composition or not. Say, with functions as first class
    objects, for example something like funct h = f o g; print h(x)
    And as that special case funct twice = f o f; print twice(x)
    For mathematics - and for abstrated representation in programming
    languages' context - it's a fundamental concept and quite useful.

    Janis

    [...]

    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From antispam@antispam@fricas.org (Waldek Hebisch) to comp.lang.misc on Sat Oct 11 12:39:11 2025
    From Newsgroup: comp.lang.misc

    bart <bc@freeuk.com> wrote:
    On 11/10/2025 01:05, Waldek Hebisch wrote:
    bart <bc@freeuk.com> wrote:
    On 10/10/2025 05:18, Waldek Hebisch wrote:
    bart <bc@freeuk.com> wrote:
    <snip>

    Last week I posted a link to a 'man or boy' test; this a version in C: >>>>>
    https://rosettacode.org/wiki/Man_or_boy_test#C

    This test requires local functions with full closures to implement
    natively. The first C version emulates what is needed.

    I tweaked that version so that is used N=13 instead of N=10, and
    evaluates the function 10,000 times. The second version uses C
    extensions to provide the higher level functionality.

    These are timings I got for the two versions:

    First C version (gcc -O2: 0.8 seconds
    (Tiny C): 1.8 seconds
    Second C version (gcc -O2): 590 seconds (extrapolated from 1K loops) >>>>> (gcc -O0): 940 seconds

    So avoiding those complex functional features makes even TCC's poor code >>>>> 300 times that optimised C. And nearly 800:1 better comparing -O2 with -O2.

    It is not clear to me what you measure. I used:

    #include <stdio.h>
    #define INT(body) ({ int lambda(){ body; }; lambda; })
    int
    main(){
    int a(int k, int xl(), int x2(), int x3(), int x4(), int x5()){
    int b(){
    return a(--k, b, xl, x2, x3, x4);
    }
    return k<=0 ? x4() + x5() : b();
    }
    int res = 0;
    int i;
    for(i = 0; i < 10000; i++) {
    res += a(13, INT(return 1), INT(return -1), INT(return -1), INT(return
    1), INT(return 0));
    }
    printf("res = %d\n", res);
    }

    Compiling this using:

    gcc -O -Wall man1.c -o man1

    I get 0.895s. Using -O2 I get 0.693s. Version without loop, but using >>>> 13 as an argument (as above) takes 0.002s. This is on a fast machine, >>>> on really slow mini-PC I get 10.117s at -O2. I suspect that your time is >>>> due to operationg system, the results above are for Linux on x86_64.
    On Linux on 1GHz Risc-V at -O2 I get:

    real 0m28.910s
    user 0m14.512s
    sys 0m14.381s

    which means that about half of execution time is in system calls.
    Looking at assembly shows that the machine code is calling
    '__riscv_flush_icache', that is it flushing cache to make sure
    that processor will execute freshly generated instructions
    (not needed on PC-s). System calls are expensive, and flushing
    cache also adds slowdown, so this adds substantial cost. Still,
    the Risc-V machine is quite slow, and can do this much faster
    than your estimate of execution time.

    For me on fast machine first (emulated) version takes 0.331s when
    compiled by gcc at -O2 and 0.767s when compiled by tcc. On Risc-V
    emulated version (compiled by gcc -O2) needs 3.553s.

    So emulated version is clearly faster, especially if version
    using nested functions need support from operating system.

    But the point of using nested functions/closures is that code
    is simpler.

    Well, the code is smaller; that doesn't mean it is simpler! It can be
    harder to know what's going on with a cryptic bit of code like this.

    (I believe it was Knuth who failed to correctly predict the result of
    N=10 (?), at some point in history when computing it was not practical.)

    As I wrote below, this test is doing nonsense operation. Most real
    uses of higher order functions are simpler (some higher order may
    even disregard them as too simple). For example, you want to
    perform some operation on all elements of an aggregate. For a list
    one can have mostly standard C code like:

    typedef struct gen_lst gen_lst;
    struct gen_lst {gen_lst * next;};

    void
    gen_map(void (*f)(gen_lst * lst, void * extra), gen_lst * lst, void * extra) {
    for(; lst; lst = lst->next) {
    f(lst, extra);
    }
    }

    typedef struct my_lst my_lst;
    struct my_lst {gen_lst * next; int val;};

    void
    adder1(gen_lst * lst, void * extra) {
    int * aux = extra;
    struct my_lst * ml = (struct my_lst *)lst;
    aux += ml->val;
    }

    int
    my_sum(my_lst * lst) {
    int aux = 0;
    gen_map(adder1, (gen_lst *) lst, (void *)&aux);
    return aux;
    }

    or you can use gcc extention like:

    void
    gen_map2(void (*f)(gen_lst * lst), gen_lst * lst) {
    for(; lst; lst = lst->next) {
    f(lst);
    }
    }

    int
    my_sum2(my_lst * lst) {
    int aux = 0;
    void
    adder(gen_lst * lst) {
    struct my_lst * ml = (struct my_lst *)lst;
    aux += ml->val;
    }
    gen_map2(adder, (gen_lst *) lst);
    return aux;
    }

    Note that argument 'extra' is now gone.

    Of course, you my ask why I do computations in this way? If all what
    needs to be done is computing sum of elements on a list, than a single
    function would do. However, if there are more operations, then
    using several looping functions spead knowledge how 'gen_lst' is
    build and how to traverse it. Which may be not that bad if you
    are sure that it will always be a list and all you need is summing.
    But you may need more complex aggregate or multiple kinds of aggregates.
    Typically there are several operations one wants to do for each
    kind of aggregate. And general part may be much larger than type
    specific part. With approach as above you only need to write
    type specific operations for each kind of aggregate, general parts
    may be reused. And depending on details it may be possible to
    share operations for different kinds of aggregates.

    I do not know if you believe that this kind of coding is useful,
    but I hope that it is clear that variant using nested functions
    will always be simpler than variant which passes extra arguments.

    TBH I found both the C and extended versions hard-going. (The spacing
    either side of "*" didn't help!)

    Well, this is a compromise. I can not change syntax of C but
    spaces are supposed to make '*' more visible.

    I think here you genuinely need better language features, but intuitive
    ones that everyone can get their head around.

    For this sort of stuff I'd prefer using a higher level language, not a
    lower level one with hairy-looking synyax that has higher order
    functions bolted on. For example in mine (which is both dynamic and intepreted) I can sum lists like this:

    a := (10,20,30,40)
    println reduce(+, a) # shows 100

    I think this is typical of scripting languages. In mine however,
    'reduce' isn't built-in, I have to implement it in the language itself
    like this:

    export func reduce(op, a)=
    x := head(a)
    for y in tail(a) do
    x := mapss(op, x, y)
    od
    x
    end

    Only 'mapss' is built-in. There is some functional stuff going on here,
    but there are no closures in the language.

    Good.

    So under WSL it's 500 times faster. I've no idea why.

    I see. It is up to you to decide if you want to investigate deeper.
    Linux version generates small pieces of executable code on the
    stack. Currently security folks hate such code and want to ban
    it. In Linux gcc specially marks executable needing executable
    stack and if executable is marked kernel allow executing code
    from the stack. Otherwise attempts to execute code from the
    stack lead to exception and kernel signals error in such case.
    There was some talk to use different method, and code generated
    on Windows may be different. One possible implementation would
    be to attempt execution on the stack, but also install signal
    handler. Signal handler can then effectively interpret was in
    on the stack. Clearly this would lead to much worse performance.

    I haven't implemented closures in a static language. (I got partway in
    the dynamic one, but decided it wasn't worth the effort or extra
    complexity just to run silly examples like manboy() or twice().)

    Anyway, I wouldn't have expected to have to use executable memory.

    I *might* have expected to do whatever the emulated version does, which doesn't need executable memory either!

    I think that language implementors want to have single kind of
    function call. Since closures need extra data popular alternatives
    are:
    - use normal pointers (address of code) to represent functions. For
    closures generate extra code that provides the data.
    - use "thick pointers" to represent functions. Thick pointer
    takes 2 machine words and provides both address of code and
    extra data.
    - use descriptors containing both address of code and extra data.
    Represent functions as pointer to the descriptor.

    Second way means that function pointers are bigger than normal
    pointers and do not fit into single machine word. That is
    problematic in languages like C and when interfacing C. Higher
    level languages frequently want to represent complex things via
    single pointer and thick pointers comflict with this. Given
    the above this approach is unattracive for most languages. In
    this version call may need extra instruction compared to first way.

    Third way is essentially "emulated" version, but done in more
    systematic way. Each call must fetch address of code and data from
    the descriptor, adding two memory accesses to each function call.

    Some OS-es went with third convention for C calls. But extra
    memory accesses have its cost, so C implementations typically
    use first way.

    IIUC second way was used in some implementations of languages like
    Pascal or Ada that is relatively low level languages with nested
    functions and restrictive type rules (when they did not care about
    detailed compatibility with C). Actually, in standard Pascal
    and IIRC in Ada too, one can not store functions in data structures
    so only place where representation of function matters is calls
    and both Pascal and Ada need to pass extra data for other
    types, so using thick pointers for functions is reasonably
    natural.

    Third way was used in original IBM PL/I and is frequently used
    by higher level languages which use reference sematintics and
    refer to any composite data via address. To allow smooth
    cooperation of code in various languages third way was used
    as system convention (so also for C) on some OS-ed. Otherwise,
    for interfacing with C they may also implement first way, but
    use it only for functions which may be callable from C.

    However, I work under Windows! So do lots of people. And there, the fact >>> that these exotic features might make programs significantly slower has
    to be taken into consideration.

    I tried a Python version too. There, 200 iterations took 5.4 seconds on
    both OSes. PyPy on Windows was twice as fast.

    Once you have interpreted language you get slowdown from the
    interpreter, but no extra trouble due executable stack: CPU treats
    bytecode as data, so in this way one can bypass security restrictions.
    IIUC PyPy may generate machine code, but probably can defer anything
    it can not handle to interpreter.

    The interesting thing about the interpreted version, is that my
    emulation of closures is so much faster than other interpreted languages where they are built-in. Because I have to implement them in bytecode,
    but those others can use the underlying C or whatever.

    Using LuaJIT, a well-regarded and usually well-performing tracing-JIT product, the 10K x manbody(13) test takes 32 seconds, twice as long as
    my own pure interpreter.

    (Because the workings are so obscure, I don't know Python/Lua enough to
    do emulated versions to see if they are any faster.)
    --
    Waldek Hebisch
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From bart@bc@freeuk.com to comp.lang.misc on Sat Oct 11 15:25:47 2025
    From Newsgroup: comp.lang.misc

    On 11/10/2025 12:08, Janis Papanagnou wrote:
    On 11.10.2025 12:25, bart wrote:
    On 11/10/2025 01:05, Waldek Hebisch wrote:
    [...]

    I haven't implemented closures in a static language. (I got partway in
    the dynamic one, but decided it wasn't worth the effort or extra
    complexity just to run silly examples like manboy() or twice().)

    You obviously missed the point.

    I think you're also missing mine. Obviously those are toy examples,
    which nevertheless need a LOT of work to implement.

    But I haven't seen many real-life examples where the benefits are clear.
    They just make the code more cryptic and less transparent. And
    near-impossible to port.


    The function twice() that someone
    suggested was a simple example for a _functional composition_, a

    Just having 'functional compositional' as a feature is not enough.

    principle that (in the more general case) is a fundamental "higher
    order" functional principle. The somewhat more general case,
    instead of f(f(x)), would be f(g(x)) - without parameter typically
    written as f o g - or (f o g)(x) . It's certainly not "silly" (or
    only to the folks who didn't understand its purpose here). So the
    question was basically whether a language supports [features for]
    functional composition or not. Say, with functions as first class
    objects, for example something like funct h = f o g; print h(x)
    And as that special case funct twice = f o f; print twice(x)
    For mathematics - and for abstrated representation in programming
    languages' context - it's a fundamental concept and quite useful.

    The only use I could see myself was for box-ticking, and in the end I
    decided that wasn't enough. My implementation would have been clunky,
    and this stuff needs to be 'native'.

    If my language had this feature, I could also write examples like this
    (not working code):

    fun twice(f) = {x: f(f(x))}
    fun plusthree(x) = x + 3

    g := twice(plusthree)
    println g(10) # displays 16

    My idea was to transform the code so it would be equivalent to emulating closures using existing language features, which would look like this
    (working code):

    fun af$1(x,f,g) = f(f(x))
    fun twice(f) = makecls(af$1, f)
    fun plusthree(x) = x + 3

    g := twice(plusthree)
    println callcls(g, 10)

    This uses a couple of helper functions:

    record cls = var fn, a, b end

    fun makecls(f, ?a, ?b) = cls(f, a, b)
    fun callcls(c, x) = (fn := c.fn; fn(x, c.a, c.b))


    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From antispam@antispam@fricas.org (Waldek Hebisch) to comp.lang.misc on Sun Oct 12 03:56:23 2025
    From Newsgroup: comp.lang.misc

    David Brown <david.brown@hesbynett.no> wrote:
    On 08/10/2025 22:21, Waldek Hebisch wrote:
    David Brown <david.brown@hesbynett.no> wrote:
    On 05/10/2025 05:15, Lawrence DrCOOliveiro wrote:
    On Sun, 5 Oct 2025 01:52:54 +0100, bart wrote:

    On 04/10/2025 23:00, Lawrence DrCOOliveiro wrote:

    On Sat, 4 Oct 2025 15:07:23 +0100, bart wrote:

    <snip>

    In terms of programming, "zero-order functions" would be statements or
    expressions. "First-order functions" are normal functions in a language >>> like C or Pascal (and presumably also Algol, though I have little
    familiarity with that language). "Second-order functions" or "Higher
    order functions" are functions that deal with functions. As Bart said,
    these are very common in functional programming languages. But they are >>> also very common in more modern or advanced imperative languages.

    Actually, in simplest form they appear quite early in developement
    of programming languages. Namely, "deals" may mean various things.
    Simplest thing is to call function passed as an argument, that is
    very old thing.

    That is usually handled by passing a pointer to a function. Pointers
    (or references of some kind) to functions are a much simpler concept
    than "dealing" with functions themselves.

    I would say that complex part is creating functions, including
    allocating memory, filling apropriate data structure and
    (if needed) generating machine code.

    Some people treat this as too simple and request
    ability to create new functions and store them in variables.
    In classic Pascal local functions cupture their environment, so
    in this sense are new ones. They are limited, because they become
    invalid after return from enclosing function. And classic Pascal
    does not allow storing functions on variables.

    I think that natural defintion of higher order function should
    include also simple cases, so functions which just take a function
    as an argument and call it should count as higher order functions.
    OTOH higher order functions are used as propagande/advertising
    term, so people who use this term want to exclude classic usage
    and apply it only to more complicated cases. So I understand
    why they give different definition, but IMO such definitions are
    misleading.

    Maybe. It is not unreasonable to say that functions taking parameters
    of function pointer type are "higher order". But I would only say that
    a language supports higher order functions if it can also return new functions from functions - and assign these to variables and pass them around.

    That is usually called "supporting first class functions".

    It is not enough to simply support function pointers, such as
    C. Under the hood, of course, these returned functions will, in a
    compiled language, be handled with structs for closures and pointers to function code. But to be a language that supports higher order
    functions, this needs to be hidden in the syntax of the language.



    Somewhat contrary to certain other posters, higher order functions are
    not "complicated stuff", and language features such as lambdas,
    closures, and nested functions do not imply or require higher order
    functions - though any language that supports higher order functions
    will support these features.

    I disagree. AFAICS one can create sensible language which supports
    higher order functions but has no closures or nested functions.


    For languages that support a limited subset of higher order functions,
    that would be true - by the definitions that you have been using, C
    would be such a language. But by the time you have a language that can return new functions and assign them to variables, then you have come
    most of the way towards closures and nested functions - there is no real difference between a nested function and a lambda assigned to a local variable. So I don't see much point in a language that supports general higher order functions and does not support closures and nested
    functions. But perhaps you know of some examples?

    I do not know of an example of language that can return created
    functions, but does support closures. However, in Pop11 there are
    two constructs. One, called "frozen values", takes existing function
    and a few values and creates new function, such that call to the
    new function is equivalent to call to the old one with given
    values as first arguments. Second one is closures. Closure are
    more complicated to implements but at the end reduce to frozen
    values. Looking at usage, frozen values are used quite a lot
    and when used code is slightly simpler than alternative code
    using closures. I did not count actual occurences, but my
    impression is that frozen values and closures are used with
    similar frequency or possible frozen values are more frequent.
    Based on this, I think that allowing only frozen values would
    give reasonable support for higher order functions. Adding
    support to C make sense when one wants to compile higher
    level language via C. Of course, this is adding a library
    function, so need not be in C standard.

    For example, C does not support higher order functions - and will still
    not do so even if the proposal for introducing lambdas to the language
    is accepted. Pascal has nested functions, but not higher order functions. >>>
    C++, on the other hand, has lambdas and higher-order functions, but not
    nested functions.

    You probably can give some argument that C++ supports higher-order
    functions, but it is quite unclear to me how you can do this without
    acceptiong that C or Pascal support higher-order functions.


    template <typename F>
    auto do_twice(F f) {
    return [f](auto x) {
    return f(f(x));
    };
    }

    int add_one(int x) { return x + 1; }
    int test_add_two(int x) {
    auto add_two = do_twice(add_one);

    return add_two(x);
    }

    You can't write something equivalent to the higher order function
    "do_twice" in C or Pascal, taking a function and returning a new one.

    But 'do_twice' returns a lambda, not a function. You can not pass
    result of 'do_twice' to a function expecting normal C++ function
    (pointer) as an argument. If what you return does not need to be
    a function, than you can do equvalent in C: return approproate
    structure and provide "call" operation. Only advantage of C++
    here is that in C++ one can overload call, so syntatically this
    looks like a function call. But semantially it is not a function
    call (because called thing is not a function).

    Of course C++'s support and syntax here is somewhat suboptimal, compared
    to functional programming languages or other modern languages that have
    had such features from the start.


    To support higher order functions, a language has to be able to handle
    functions as first-class objects - it has to be able to take them as
    function parameters, and return them as functions. It is not sufficient >>> to take and return function pointers, like C - it has to be something
    that is treated in the language as a function.

    I strongly disagree with this statement about pointers. Using pointers
    to represent functions is usual in C and pretty consistent with making
    explicit things that other languages do behind the scenes. Rather,
    main limitation in C is that one can only use existing functions, there
    is no way to make a new one.

    Yes, that is the main limitation of C (in this context) - and the
    limitation is because C only supports functions through function
    pointers. Technically, C only supports declaring and defining
    functions, and taking their address - even when you "call" a function in
    C, you are applying the call operator to a function pointer. In order
    to support more advanced higher order functions - such as returning new functions - a language must be able to support larger underlying data structures while maintaining the syntax of function calls. It has to
    handle captures, closures and/or other details. A "function" is not
    merely a pointer to an address in code memory.

    When using trampolines closure is a pointer to an address of
    freshly generated code. Note that on some OS-es (IIUC AIX and
    Alpha/IA64 VMS) C function is a pointer to descriptor. Descriptor
    contains both pointer to code and pointer to environment. IIUC VMS
    on x86_64 decided to use trampolines. IIUC on all architectures
    you can pass closures to VMS C and VMS C will call them correctly.
    So fact that typical C implementation uses address of code is
    just implementation datail.

    Classic Pascal can make new functions,
    but one can not legaly return freshly created function, so in this
    aspect is only marginally better than C.

    It's a good while since I programmed in Pascal, but I was not aware of
    any such facility in Pascal. It has nested functions, but that is not
    the same thing.

    I explained that in answer to Janis.

    Lambdas in C and C++ are
    problematic as they are not usual way to represent functions in C or
    C++.

    It is correct that lambdas in C++ (C does not have lambdas, though there
    is a proposal to add them to the language) are not as simple as normal functions. They are basically an alternative syntax for functors, which
    are classes with an overridden call operator.


    OTOH a tiny extention to C could give reasonable support. Namely,
    while other languages may implicitly allocate heap memory, C requires
    explicit calls to 'malloc' (and 'free').

    Requiring memory allocation to be explicit is considered a strength of
    C. Changing that is not going to go down well. (Lambdas in C++ do not require implicit memory allocation. C++ lambdas and functions that
    return new functions have certain visibility limitations in C++ - they cannot cross boundaries of independently compiled units, as each lambda
    has a unique type. Having separate compilation and independence while retaining full flexibility is supported in the language and standard library, with std::function<>, but it comes with a considerable overhead
    - including heap allocations.)

    Similarly, natural way
    to create new function in C would be appropriate allocation function,
    say called 'falloc' (an matching 'ffree'). 'falloc' should take
    pointer to a function 'f', a value 'v' and probably number of arguments
    'n' and return poiter 'g' so that call

    (*g)(a_1,\dots, a_n)


    is equvalent to

    (*f)(v, a_1,\dots, a_n)

    taking a 'v' a pointer this allows passing arbitrary amount of extra
    data to 'f'.

    With such extention C would allow reasonably natural expression of
    things done in other languages claiming to support higher order functions. >> People using other languages may disregard this as too primitive,
    but IMO it would very natural in C.

    BTW: people doing functional programming sometimes claim that to
    support it one needs garbage collection. I arrived at the idea
    above thinking which parts of functional programming can work
    well without garbage collection.


    Garbage collection is not, AFAIK, necessary for common functional programming techniques. But it might make some of them more convenient
    or efficient to use.

    Some functional techniques work well without garbage collection.
    But some basically loose all advantates without garbage collection
    or equivalent automatic technique. In particular, passing
    complex dynamically allocated data structures trough chain of
    calls becomes problematic due to need for deallocation.
    --
    Waldek Hebisch
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From David Brown@david.brown@hesbynett.no to comp.lang.misc on Mon Oct 13 20:58:15 2025
    From Newsgroup: comp.lang.misc

    On 10/10/2025 21:04, bart wrote:
    On 10/10/2025 16:40, David Brown wrote:
    On 08/10/2025 22:21, Waldek Hebisch wrote:

    I disagree.-a AFAICS one can create sensible language which supports
    higher order functions but has no closures or nested functions.


    For languages that support a limited subset of higher order functions,
    that would be true - by the definitions that you have been using, C
    would be such a language.-a But by the time you have a language that
    can return new functions and assign them to variables, then you have
    come most of the way towards closures and nested functions - there is
    no real difference between a nested function and a lambda assigned to
    a local variable.-a So I don't see much point in a language that
    supports general higher order functions and does not support closures
    and nested functions.-a But perhaps you know of some examples?


    There are advantages to having local, nested functions when the
    alternative is:

    * Defining them as separate, ordinary functions at module level...

    * Where their names are now part of the global name space which
    -a can clash and can't be reused, or could accidentally be exported
    -a from the module when you forget 'static'

    * Where those functions could be called from places where they are
    -a not intended to be called

    * Where they may not be localised to what would be their enclosing
    -a function, so could be located anywhere in the module, or even
    -a elsewhere

    * Losing the close relationship between them

    None of that needs closures.

    I agree with all of those points. That's why languages like Pascal
    support local functions. Local functions can be useful without
    "advanced" higher order functions (those that return new functions, or
    create functions, rather than simply accepting function pointer
    parameters). I merely said that I think closures and nested functions
    of some sort are so useful for advanced high order functions that a
    language is unlikely to support advanced high order functions without
    also supporting closures and nested functions. But languages can
    certainly support nested functions without being able to write functions
    that return new functions.

    Further, the above applies to C; local
    functions can implemented that can refer to useful entities defined in
    the enclosing function such as static variables, types, structs and enums.


    C does not support local functions. But it could conceivably be added
    to the language (as demonstrated by the gcc extension).

    Closures are only needed for access to stack-frame entities of the
    enclosing functions, including parameters.

    Yes. But those are /really/ useful. Nested functions can have their
    use without closures, but closures make them much more powerful.

    Of course you can emulate closures manually. Basically, you make a
    struct that has all the capture-by-value data and pointers for all the capture-by-reference data, and you add a pointer to that struct to your function parameters. A language that supports closures handles this automatically for you, so that you don't all the boilerplate and manual updates whenever the captured data changes. And the extra struct
    pointer is hidden.

    There was a discussion elsewhere about gcc's handling of nested
    functions sometimes needing an executable stack, or other complications.
    This occurs when you have a nested function that has a closure - thus needing a hidden struct pointer - but where it needs to be passed
    somewhere as though it were a pointer to a function without this extra parameter. One way to handle this is with a small function created
    directly on the stack.


    Of course, you will need closures to run all the silly examples of
    higher order functions that abound on the internet.

    I've yet to see examples that have real uses and that don't make code
    harder to grasp.


    What is hard or easy to grasp is a very subjective matter.

    Let me try to describe a possible real-world use, using an imaginary
    language to avoid any bias or complicating things with real-world syntax details.

    You have a marvellous string sorting function, taking an array of
    strings and a comparison function as a parameter :

    sort_strings(string s[], function comp(string, string))

    You can use this with comparison functions that are case-sensitive, reverse-order case-insensitive, or whatever you like.

    You have a library that has a comparison function for names, taking a
    country code as a parameter because different countries use different
    criteria for ordering names (some treat "o" and "||" as different letters
    some treat them as the same letter for ordering, some treat "||" like
    "oe", and so on) :

    compare_names(string name1, string name2, string country)


    The country is chosen by the user. How do you sort your strings using
    these functions? How do you write the function :

    sort_names(string s[], string country)

    ?

    You could do it by :

    define sort_names(string s[], string country) :

    sort_func = lambda (string name1, string name2) :
    return compare_names(name1, name2, country)

    sort_strings(s[], sort_func)

    Perhaps you are using this kind of country-specific sorter function a
    lot, and you want to refactor this into its own function to aid reuse
    and modularisation :

    define make_sort_func(string country) :
    sort_func = lambda(string s1, string s2) :
    return compare_names(s1, s2, country)
    return sort_func

    Now you can do your sorting with :

    define sort_names(string s[], string country) :
    sort_strings(s[], make_sort_func(country))



    Somehow, you have got to create a function that compares two strings,
    with that function depending on an independent input (the country).
    That is a closure - you've made a new function that is based on the three-parameter compare_names function and a bit of captured data.
    Whether you want to do that with a lambda (an anonymous function), a
    named lambda, a function that returns a new function, or a local
    function, is just a matter of style and taste (assuming the language
    supports the feature).

    Of course there are always ways to avoid this - such as making your
    original "sort_strings" function take a "void *" parameter that is
    passed to the comparison function as an environment, or making an array
    of sort functions indexed by country. But I hope that example shows
    that closures and higher-order functions can be useful in simple
    examples that are not purely theoretical, and that it is clear what the
    code above is doing.


    I don't however, use local functions; I've never gotten into the habit.


    Fair enough. I don't think many people use them a lot - especially with languages that have other ways of handling scoping, such as classes or namespaces.

    I couldn't even tell you if my languages support them and how well.

    That is a rather interesting thing to say! People often don't know all
    the details of languages they use, but /usually/ the authors of those languages have a fair idea of what they can do :-) (Yes, I know that
    some of your languages were made a long time ago.)

    When
    I tried to run the program below right now, it had trouble resolving
    names 'a b vector' from the nested function. I had to put in qualifiers
    to make it work.

    --------------------------
    proc main=
    -a-a-a const a = 100
    -a-a-a static int b
    -a-a-a type vector = [4]int

    -a-a-a proc fred(int c) =
    -a-a-a-a-a-a-a println "Fred", main.a, main.b, c, main.vector.typestr
    -a-a-a end

    -a-a-a b := 200
    -a-a-a fred(300)
    end

    Output is:

    Fred 100 200 300 [4]i64



    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From David Brown@david.brown@hesbynett.no to comp.lang.misc on Mon Oct 13 22:38:59 2025
    From Newsgroup: comp.lang.misc

    On 12/10/2025 05:56, Waldek Hebisch wrote:
    David Brown <david.brown@hesbynett.no> wrote:
    On 08/10/2025 22:21, Waldek Hebisch wrote:
    David Brown <david.brown@hesbynett.no> wrote:

    For example, C does not support higher order functions - and will still >>>> not do so even if the proposal for introducing lambdas to the language >>>> is accepted. Pascal has nested functions, but not higher order functions. >>>>
    C++, on the other hand, has lambdas and higher-order functions, but not >>>> nested functions.

    You probably can give some argument that C++ supports higher-order
    functions, but it is quite unclear to me how you can do this without
    acceptiong that C or Pascal support higher-order functions.


    template <typename F>
    auto do_twice(F f) {
    return [f](auto x) {
    return f(f(x));
    };
    }

    int add_one(int x) { return x + 1; }
    int test_add_two(int x) {
    auto add_two = do_twice(add_one);

    return add_two(x);
    }

    You can't write something equivalent to the higher order function
    "do_twice" in C or Pascal, taking a function and returning a new one.

    But 'do_twice' returns a lambda, not a function. You can not pass
    result of 'do_twice' to a function expecting normal C++ function
    (pointer) as an argument. If what you return does not need to be
    a function, than you can do equvalent in C: return approproate
    structure and provide "call" operation. Only advantage of C++
    here is that in C++ one can overload call, so syntatically this
    looks like a function call. But semantially it is not a function
    call (because called thing is not a function).


    Syntactically, it looks like a function - it is "callable" or
    "invokable" in C++ terms. Yes, it is a somewhat different beast from a
    plain function pointer - every lambda is its own unique type, and can
    wrap a closure as well as a function pointer. If the lambda does not
    need a closure, it can be converted to a function pointer of the
    appropriate type, but it is still not a normal function. C++ supports a number of things that can be used syntactically and semantically like a function in many situations - normal functions, lambdas, function
    templates, methods, functors, std::function objects, etc. They can't
    all be used in the same way in all the same circumstances. In
    particular, if some kind of closure or extra information is needed, you
    can't use a function-like object when a plain function pointer is
    required. But often you use them in connection with templates, allowing
    the syntax and convenience of using them like functions while generating efficient object code.

    If you need a general "pointer to a function or function-like object", however, in C++ you have std::function<> with a good deal of run-time
    overhead to support its generality.

    Of course C++'s support and syntax here is somewhat suboptimal, compared
    to functional programming languages or other modern languages that have
    had such features from the start.


    To support higher order functions, a language has to be able to handle >>>> functions as first-class objects - it has to be able to take them as
    function parameters, and return them as functions. It is not sufficient >>>> to take and return function pointers, like C - it has to be something
    that is treated in the language as a function.

    I strongly disagree with this statement about pointers. Using pointers
    to represent functions is usual in C and pretty consistent with making
    explicit things that other languages do behind the scenes. Rather,
    main limitation in C is that one can only use existing functions, there
    is no way to make a new one.

    Yes, that is the main limitation of C (in this context) - and the
    limitation is because C only supports functions through function
    pointers. Technically, C only supports declaring and defining
    functions, and taking their address - even when you "call" a function in
    C, you are applying the call operator to a function pointer. In order
    to support more advanced higher order functions - such as returning new
    functions - a language must be able to support larger underlying data
    structures while maintaining the syntax of function calls. It has to
    handle captures, closures and/or other details. A "function" is not
    merely a pointer to an address in code memory.

    When using trampolines closure is a pointer to an address of
    freshly generated code. Note that on some OS-es (IIUC AIX and
    Alpha/IA64 VMS) C function is a pointer to descriptor. Descriptor
    contains both pointer to code and pointer to environment. IIUC VMS
    on x86_64 decided to use trampolines. IIUC on all architectures
    you can pass closures to VMS C and VMS C will call them correctly.
    So fact that typical C implementation uses address of code is
    just implementation datail.

    It's a question of efficiency. If some of your "function pointers" are
    direct pointers to code, and some are pointers to a descriptor, then
    whenever you are calling through a "function pointer" you need code that checks what style of pointer you have, and acts accordingly. This may
    be significant overhead compared to a straight indirect jump.
    Trampolines on the stack avoid that overhead, but have the cost of
    requiring an executable stack.

    BTW: people doing functional programming sometimes claim that to
    support it one needs garbage collection. I arrived at the idea
    above thinking which parts of functional programming can work
    well without garbage collection.


    Garbage collection is not, AFAIK, necessary for common functional
    programming techniques. But it might make some of them more convenient
    or efficient to use.

    Some functional techniques work well without garbage collection.
    But some basically loose all advantates without garbage collection
    or equivalent automatic technique. In particular, passing
    complex dynamically allocated data structures trough chain of
    calls becomes problematic due to need for deallocation.


    Yes, I can see that being complicated.


    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From bart@bc@freeuk.com to comp.lang.misc on Mon Oct 13 23:13:41 2025
    From Newsgroup: comp.lang.misc

    On 13/10/2025 19:58, David Brown wrote:
    On 10/10/2025 21:04, bart wrote:

    There was a discussion elsewhere about gcc's handling of nested
    functions sometimes needing an executable stack, or other complications.
    -aThis occurs when you have a nested function that has a closure - thus needing a hidden struct pointer - but where it needs to be passed
    somewhere as though it were a pointer to a function without this extra parameter.-a One way to handle this is with a small function created directly on the stack.

    Thanks for clearing that up. I think I would have simply disallowed such
    uses. Why should a special function with a hidden extra parameter be
    able to masquerade as an ordinary function? The same point comes up
    later on.

    Let the user make their own arrangements for this.

    (This is actually a similar problem to what happens when an interpreted language has to pass a function reference via an FFI to be used as a
    callback function.

    Obviously, native code functions can't directly call a function pointer
    that refers to some byte code rather than machine instructions. There
    needs to be something inbetween.

    I've never fully solved that for the general case. But I had in mind a
    fixed number of predefined, intermediate, native code functions that sit between the external library, and a special reentry point to the
    interpreter.

    I /think/ it can be done with data rather than new executable code.)


    Let me try to describe a possible real-world use, using an imaginary language to avoid any bias or complicating things with real-world syntax details.

    You have a marvellous string sorting function, taking an array of
    strings and a comparison function as a parameter :

    -a-a-a-asort_strings(string s[], function comp(string, string))

    You can use this with comparison functions that are case-sensitive, reverse-order case-insensitive, or whatever you like.

    -a-a-a-adefine make_sort_func(string country) :
    -a-a-a-a-a-a-a sort_func = lambda(string s1, string s2) :
    -a-a-a-a-a-a-a-a-a-a-a return compare_names(s1, s2, country)
    -a-a-a-a-a-a-a return sort_func

    Now you can do your sorting with :

    -a-a-a-adefine sort_names(string s[], string country) :
    -a-a-a-a-a-a-a sort_strings(s[], make_sort_func(country))



    Somehow, you have got to create a function that compares two strings,
    with that function depending on an independent input (the country). That
    is a closure - you've made a new function that is based on the three- parameter compare_names function and a bit of captured data. Whether you want to do that with a lambda (an anonymous function), a named lambda, a function that returns a new function, or a local function, is just a
    matter of style and taste (assuming the language supports the feature).

    Of course there are always ways to avoid this - such as making your
    original "sort_strings" function take a "void *" parameter that is
    passed to the comparison function as an environment, or making an array
    of sort functions indexed by country.-a But I hope that example shows
    that closures and higher-order functions can be useful in simple
    examples that are not purely theoretical, and that it is clear what the
    code above is doing.

    You've used static typing for your examples: the sort function clearly
    takes two arguments, not optionally an extra argument!

    So, is this an example where a 'trampoline' is needed, to pretend your
    true sort function is a simple 2-operand one? I don't think this is
    going to be help performance any, not on Windows (I understand Linux
    does something special here).

    But as you say, this example isn't that compelling, as there are any
    number of workarounds.

    I couldn't even tell you if my languages support them and how well.

    That is a rather interesting thing to say!-a People often don't know all
    the details of languages they use, but /usually/ the authors of those languages have a fair idea of what they can do :-)-a (Yes, I know that
    some of your languages were made a long time ago.)

    This can happen with little-used features, or ones you've forgotten
    about or never used. You tend to find them when overhauling code.

    However I think I first had nested functions by accident, by neglecting
    to disallow them.

    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From David Brown@david.brown@hesbynett.no to comp.lang.misc on Tue Oct 14 00:39:37 2025
    From Newsgroup: comp.lang.misc

    On 14/10/2025 00:13, bart wrote:
    On 13/10/2025 19:58, David Brown wrote:
    On 10/10/2025 21:04, bart wrote:

    There was a discussion elsewhere about gcc's handling of nested
    functions sometimes needing an executable stack, or other
    complications. -a-aThis occurs when you have a nested function that has
    a closure - thus needing a hidden struct pointer - but where it needs
    to be passed somewhere as though it were a pointer to a function
    without this extra parameter.-a One way to handle this is with a small
    function created directly on the stack.

    Thanks for clearing that up. I think I would have simply disallowed such uses.

    That is not an unreasonable choice. There are tradeoffs in allowing
    such functions in C (or any other language where you can't pass around
    more information than just the address for a function pointer).

    Why should a special function with a hidden extra parameter be
    able to masquerade as an ordinary function? The same point comes up
    later on.

    Let the user make their own arrangements for this.

    (This is actually a similar problem to what happens when an interpreted language has to pass a function reference via an FFI to be used as a callback function.

    Obviously, native code functions can't directly call a function pointer
    that refers to some byte code rather than machine instructions. There
    needs to be something inbetween.

    I've never fully solved that for the general case. But I had in mind a
    fixed number of predefined, intermediate, native code functions that sit between the external library, and a special reentry point to the interpreter.

    I /think/ it can be done with data rather than new executable code.)


    It can always be done with data - but that might be less efficient than
    using executable code. The overhead is probably worth it, however.


    Let me try to describe a possible real-world use, using an imaginary
    language to avoid any bias or complicating things with real-world
    syntax details.

    You have a marvellous string sorting function, taking an array of
    strings and a comparison function as a parameter :

    -a-a-a-a-asort_strings(string s[], function comp(string, string))

    You can use this with comparison functions that are case-sensitive,
    reverse-order case-insensitive, or whatever you like.

    -a-a-a-a-adefine make_sort_func(string country) :
    -a-a-a-a-a-a-a-a sort_func = lambda(string s1, string s2) :
    -a-a-a-a-a-a-a-a-a-a-a-a return compare_names(s1, s2, country)
    -a-a-a-a-a-a-a-a return sort_func

    Now you can do your sorting with :

    -a-a-a-a-adefine sort_names(string s[], string country) :
    -a-a-a-a-a-a-a-a sort_strings(s[], make_sort_func(country))



    Somehow, you have got to create a function that compares two strings,
    with that function depending on an independent input (the country).
    That is a closure - you've made a new function that is based on the
    three- parameter compare_names function and a bit of captured data.
    Whether you want to do that with a lambda (an anonymous function), a
    named lambda, a function that returns a new function, or a local
    function, is just a matter of style and taste (assuming the language
    supports the feature).

    Of course there are always ways to avoid this - such as making your
    original "sort_strings" function take a "void *" parameter that is
    passed to the comparison function as an environment, or making an
    array of sort functions indexed by country.-a But I hope that example
    shows that closures and higher-order functions can be useful in simple
    examples that are not purely theoretical, and that it is clear what
    the code above is doing.

    You've used static typing for your examples: the sort function clearly
    takes two arguments, not optionally an extra argument!


    Yes.

    So, is this an example where a 'trampoline' is needed, to pretend your
    true sort function is a simple 2-operand one? I don't think this is
    going to be help performance any, not on Windows (I understand Linux
    does something special here).


    Trampolines are fast, but require an executable stack. Function
    descriptor pointers are slower (even when the function pointer is just a normal function - all indirect function calls get an overhead) but
    avoids the executable stack. I don't know what happens for this kind of
    thing in Windows, sorry.

    But as you say, this example isn't that compelling, as there are any
    number of workarounds.


    You can always find workarounds for all kinds of things. A language
    doesn't have to have any kind of for or while loop, if it has "goto". A language doesn't need functions to take more than one parameter, if you
    can pass a pointer to a struct. This is not about "can you make
    flexible sort functions without closures?", it is about showing how real
    code can use closures to give neat and clear code.

    I couldn't even tell you if my languages support them and how well.

    That is a rather interesting thing to say!-a People often don't know
    all the details of languages they use, but /usually/ the authors of
    those languages have a fair idea of what they can do :-)-a (Yes, I know
    that some of your languages were made a long time ago.)

    This can happen with little-used features, or ones you've forgotten
    about or never used. You tend to find them when overhauling code.

    However I think I first had nested functions by accident, by neglecting
    to disallow them.


    OK.


    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From antispam@antispam@fricas.org (Waldek Hebisch) to comp.lang.misc on Fri Oct 24 04:33:37 2025
    From Newsgroup: comp.lang.misc

    David Brown <david.brown@hesbynett.no> wrote:
    On 12/10/2025 05:56, Waldek Hebisch wrote:
    David Brown <david.brown@hesbynett.no> wrote:
    On 08/10/2025 22:21, Waldek Hebisch wrote:
    David Brown <david.brown@hesbynett.no> wrote:

    To support higher order functions, a language has to be able to handle >>>>> functions as first-class objects - it has to be able to take them as >>>>> function parameters, and return them as functions. It is not sufficient >>>>> to take and return function pointers, like C - it has to be something >>>>> that is treated in the language as a function.

    I strongly disagree with this statement about pointers. Using pointers >>>> to represent functions is usual in C and pretty consistent with making >>>> explicit things that other languages do behind the scenes. Rather,
    main limitation in C is that one can only use existing functions, there >>>> is no way to make a new one.

    Yes, that is the main limitation of C (in this context) - and the
    limitation is because C only supports functions through function
    pointers. Technically, C only supports declaring and defining
    functions, and taking their address - even when you "call" a function in >>> C, you are applying the call operator to a function pointer. In order
    to support more advanced higher order functions - such as returning new
    functions - a language must be able to support larger underlying data
    structures while maintaining the syntax of function calls. It has to
    handle captures, closures and/or other details. A "function" is not
    merely a pointer to an address in code memory.

    When using trampolines closure is a pointer to an address of
    freshly generated code. Note that on some OS-es (IIUC AIX and
    Alpha/IA64 VMS) C function is a pointer to descriptor. Descriptor
    contains both pointer to code and pointer to environment. IIUC VMS
    on x86_64 decided to use trampolines. IIUC on all architectures
    you can pass closures to VMS C and VMS C will call them correctly.
    So fact that typical C implementation uses address of code is
    just implementation datail.

    It's a question of efficiency. If some of your "function pointers" are direct pointers to code, and some are pointers to a descriptor, then whenever you are calling through a "function pointer" you need code that checks what style of pointer you have, and acts accordingly. This may
    be significant overhead compared to a straight indirect jump.
    Trampolines on the stack avoid that overhead, but have the cost of
    requiring an executable stack.

    On Alpha VMS and AFAIK on AIX _all_ function pointers are pointers
    to a descriptor. There is an overhead, something like two extra
    machine instructions per function call. Trying to mix pointers to
    descriptors with pointers to code makes no sense for C: you end up
    with bigger and slower code. It can work if pointers to desctiptors
    and pointers to code are sepatate types and decision which one
    to use is made at compile time. But then various kinds of functions
    must jumps some hops to cooperate.
    --
    Waldek Hebisch
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Lawrence =?iso-8859-13?q?D=FFOliveiro?=@ldo@nz.invalid to comp.lang.misc on Fri Oct 31 06:51:30 2025
    From Newsgroup: comp.lang.misc

    On Sat, 11 Oct 2025 10:42:49 +0100, bart wrote:

    On 11/10/2025 02:42, Lawrence DrCOOliveiro wrote:

    On Sat, 11 Oct 2025 01:59:28 +0100, bart wrote:

    I'm not quite clear what you code does, or whether I would need to
    do the same.

    You said you couldnrCOt see a need for this sort of functionality.
    This is an example of this sort of functionality.

    You said you used some complicated method to wrap how you access
    C-style struct info. I don't know what code would look like before
    and after you did that. Whatever it does, I'm not convinced your
    approach is the simplest.

    You keep saying that. But you have no idea how to write the code.

    It didn't do anything interesting beyond printing 'hello'.

    I rest my case.
    --- Synchronet 3.21a-Linux NewsLink 1.2