• Question regarding behaviour of the sh(1) "-c" option

    From Lew Pitcher@lew.pitcher@digitalfreehold.ca to comp.unix.shell on Fri Aug 29 15:09:25 2025
    From Newsgroup: comp.unix.shell

    Long ago, I played with Minix (v1.5), and wrote a SysV-style init(8) for it.
    At that time, I couldn't see a proper way to parse the "process" field of /etc/inittab, and simply crafted a naive parser from strtok() calls and
    special handling for redirect characters. I knew that this wasn't the way
    to do it; I really needed to use the commandline parsing logic of sh(1).

    Fast forward to now, and I'm looking at recycling my old Minix init(8)
    for use in a Linux "homebrew" container environment, and I want to fix
    this commandline parsing clumsiness. My current thinking is that there
    may be some way to invoke sh(1) to perform the parsing and execution for
    me. My only oddball requirement is that sh(1) "get out of the way" (so
    to speak), and directly /exec()/ the resulting parsed commandline, rather
    than fork() and exec().

    I noticed the "-c" sh(1) commandline option, and have puzzled about it.
    From tests, it /appears/ to do what I want: parse the given string into
    argv[] arguments, then exec() that list, but the documentation doesn't
    really say that this behaviour is to be expected.

    From the OpenGroup's description of sh(1)[1], it /looks/ like sh(1) is
    supposed to fork() and exec(), and the Linux sh(1) manpage[2] is even less helpful about the behaviour of the -c option. However, limited tests on my Linux boxen show that bash(1) (at least version 4.3.48) appears to exec()
    only.

    So, can anyone advise me; does the sh(1) "-c" option cause shell to
    a) parse the given string into a commandline argument list, and then
    b) exec(2) that argument list, replacing sh(1) with the specified
    binary within the process?


    Thanks for any advice you can give me.

    [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html [2] https://linux.die.net/man/1/sh
    --
    Lew Pitcher
    "In Skills We Trust"
    Not LLM output - I'm just like this.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Adam Sampson@ats@offog.org to comp.unix.shell on Fri Aug 29 18:09:19 2025
    From Newsgroup: comp.unix.shell

    Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:

    So, can anyone advise me; does the sh(1) "-c" option cause shell to
    a) parse the given string into a commandline argument list, and then
    b) exec(2) that argument list, replacing sh(1) with the specified
    binary within the process?

    Not necessarily, because you can specify an arbitrary sequence of
    commands with -c -- e.g. sh -c "date; uname -a" -- including more
    complex things like loops and function definitions. Or you might give it
    a command that only uses builtins, so it doesn't need to run any
    external binaries.

    However, it's a reasonable optimisation to exec the command when
    possible. The latest versions of bash, mksh, busybox sh, dash and ksh93
    all do this when given a single command with -c, and all except mksh
    will exec the last command when given a sequence of simple commands.

    Looking at the source for sysvinit-3.14's implementation of init, it
    looks at the command given in inittab to see whether it contains any
    quotes or shell metacharacters. If none are present, it splits up the
    command on whitespace and calls execvp directly; if special characters
    are found, it constructs a "sh -c" command and execvps that. busybox
    init does exactly the same. So it sounds like your approach is
    reasonable.

    Neither sysvinit or busybox init prepends "exec" to the given command
    (although there's commented-out code in sysvinit suggesting that it did
    do this at some point). This has the advantage that you can write
    arbitrary complex shell commands in inittab, e.g. to set up the
    environment for a process before starting it.
    --
    Adam Sampson <ats@offog.org> <http://offog.org/>
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Helmut Waitzmann@nn.throttle@xoxy.net to comp.unix.shell on Fri Aug 29 19:58:03 2025
    From Newsgroup: comp.unix.shell

    Lew Pitcher <lew.pitcher@digitalfreehold.ca>:
    Long ago, I played with Minix (v1.5), and wrote a SysV-style
    init(8) for it. At that time, I couldn't see a proper way to
    parse the "process" field of /etc/inittab, and simply crafted a
    naive parser from strtok() calls and special handling for
    redirect characters. I knew that this wasn't the way to do it; I
    really needed to use the commandline parsing logic of sh(1).


    Fast forward to now, and I'm looking at recycling my old Minix
    init(8) for use in a Linux "homebrew" container environment, and
    I want to fix this commandline parsing clumsiness. My current
    thinking is that there may be some way to invoke sh(1) to
    perform the parsing and execution for me. My only oddball
    requirement is that sh(1) "get out of the way" (so to speak),
    and directly /exec()/ the resulting parsed commandline, rather
    than fork() and exec().


    I noticed the "-c" sh(1) commandline option, and have puzzled
    about it. From tests, it /appears/ to do what I want: parse the
    given string into argv[] arguments, then exec() that list, but
    the documentation doesn't really say that this behaviour is to
    be expected.

    By the way:-a This is not a property of the "-c" sh option.-a This
    behavior might occur whenever the shell is to execute a last
    command before exiting, for example with a shell script given in
    the shell's invocation parameter list or commands fed into the
    shell's stdin.
    From the OpenGroup's description of sh(1)[1], it /looks/ like
    sh(1) is supposed to fork() and exec(), and the Linux sh(1)
    manpage[2] is even less helpful about the behaviour of the -c
    option. However, limited tests on my Linux boxen show that
    bash(1) (at least version 4.3.48) appears to exec() only.

    If Bash knows that after having (forked a new process and) execed
    the binary with its argv[] arguments there is nothing left to do
    other than receiving the exit status of that forked process and
    then exiting using that exit status then it might well dispense
    with forking a new process and just exec the binary with its
    argv[] list.
    Try and compare the following shell command lines:
    (1)
    bash -c '
    ps -o ppid -o pgid -o pid -o stat -o args
    es="$?"
    printf '\''ps exited returning %s\n'\'' "$es"
    exit "$es"
    ' sh
    In this first example obviously the shell has to stay alive after
    having execed the "ps" binary in order to be able to execute the
    variable assignment and the "printf" command.
    (2)
    bash -c '
    ps -o ppid -o pgid -o pid -o stat -o args
    ' sh
    In this second example "bash" doesn't have to do more than fork
    and exec "ps", then wait for "ps" to exit, receive the exit
    status and exit returning that exit status.-a So it might very
    well dispense with forking a new process but just replace itself
    with the "ps" binary.
    (3)
    bash -c '
    exec ps -o ppid -o pgid -o pid -o stat -o args
    ' sh
    Finally in this third example (see below) the "exec" shell
    builtin command tells "bash" to dispense with forking a new
    process but just replace itself with the "ps" binary.
    Repeat these command lines, but now use "sh" rather than "bash".-a
    I expect the same behavior as before in (1) and (3) respectively,
    but (2) may be different than with "bash".
    So, can anyone advise me; does the sh(1) "-c" option cause shell to
    a) parse the given string into a commandline argument list, and then
    b) exec(2) that argument list, replacing sh(1) with the specified
    binary within the process?

    Not the "-c" option but the "exec" shell builtin command will do
    what you want to be done:-a The "exec" shell builtin command tells
    the shell not to fork a new process but rather exec the binary
    replacing itself.
    [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
    By the way:-a There is new version of the POSIX standard:
    <http://pubs.opengroup.org/onlinepubs/9799919799/mindex.html>.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Christian Weisgerber@naddy@mips.inka.de to comp.unix.shell on Fri Aug 29 17:45:36 2025
    From Newsgroup: comp.unix.shell

    On 2025-08-29, Christian Weisgerber <naddy@mips.inka.de> wrote:

    Some sh implementations may exec() a final command as an optimization. Comparing sh -c '/bin/sleep 100', I see

    FreeBSD sh (Almquist derivate): fork + exec
    OpenBSD sh (PD ksh derivate): fork + exec
    GNU bash: exec

    Actually, I confused myself there. Allow me to revise:

    FreeBSD sh (Almquist derivate): exec
    OpenBSD sh (PD ksh derivate): fork + exec
    GNU bash: exec

    But notice that all three will switch to fork+exec if there is more
    than a single command, e.g. sh -c '/bin/sleep 100; /bin/sleep 200'.
    --
    Christian "naddy" Weisgerber naddy@mips.inka.de
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Christian Weisgerber@naddy@mips.inka.de to comp.unix.shell on Fri Aug 29 17:33:24 2025
    From Newsgroup: comp.unix.shell

    On 2025-08-29, Lew Pitcher <lew.pitcher@digitalfreehold.ca> wrote:

    I noticed the "-c" sh(1) commandline option, and have puzzled about it.
    From tests, it /appears/ to do what I want: parse the given string into argv[] arguments, then exec() that list, but the documentation doesn't
    really say that this behaviour is to be expected.

    From the OpenGroup's description of sh(1)[1], it /looks/ like sh(1) is supposed to fork() and exec(),

    That's what it does.

    Some sh implementations may exec() a final command as an optimization. Comparing sh -c '/bin/sleep 100', I see

    FreeBSD sh (Almquist derivate): fork + exec
    OpenBSD sh (PD ksh derivate): fork + exec
    GNU bash: exec
    --
    Christian "naddy" Weisgerber naddy@mips.inka.de
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Lew Pitcher@lew.pitcher@digitalfreehold.ca to comp.unix.shell on Fri Aug 29 18:51:20 2025
    From Newsgroup: comp.unix.shell

    On Fri, 29 Aug 2025 18:09:19 +0100, Adam Sampson wrote:

    Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:

    So, can anyone advise me; does the sh(1) "-c" option cause shell to
    a) parse the given string into a commandline argument list, and then
    b) exec(2) that argument list, replacing sh(1) with the specified
    binary within the process?

    Not necessarily, because you can specify an arbitrary sequence of
    commands with -c -- e.g. sh -c "date; uname -a" -- including more
    complex things like loops and function definitions. Or you might give it
    a command that only uses builtins, so it doesn't need to run any
    external binaries.

    This is what I suspected.

    However, it's a reasonable optimisation to exec the command when
    possible. The latest versions of bash, mksh, busybox sh, dash and ksh93
    all do this when given a single command with -c, and all except mksh
    will exec the last command when given a sequence of simple commands.

    And, this behaviour is new to me :-)
    I wonder if this is a documented POSIX requirement for sh(1), or an ad-hoc optimization that a variety of shells found useful. In other words, can I depend on /every/ sh(1) implementation acting this way?

    Looking at the source for sysvinit-3.14's implementation of init, it
    looks at the command given in inittab to see whether it contains any
    quotes or shell metacharacters. If none are present, it splits up the
    command on whitespace and calls execvp directly; if special characters
    are found, it constructs a "sh -c" command and execvps that. busybox
    init does exactly the same. So it sounds like your approach is
    reasonable.

    So, it seems that my original attempt at hand-parsing the (inittab-limited) commandline was not as far off-base as I had thought. I still don't like
    it, but at least it doesn't bother me as much, knowing that other init(8) implementations did essentially the same.

    Neither sysvinit or busybox init prepends "exec" to the given command (although there's commented-out code in sysvinit suggesting that it did
    do this at some point). This has the advantage that you can write
    arbitrary complex shell commands in inittab, e.g. to set up the
    environment for a process before starting it.

    That was my next stop, should the "-c" option not act the way I thought
    it was. It appears safer to prepend an "exec" onto the otherwise
    unedited inittab "process" string, then to depend on that string holding
    a single, well-formed command.


    Thanks for your help. Now I have direction.
    --
    Lew Pitcher
    "In Skills We Trust"
    Not LLM output - I'm just like this.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Lew Pitcher@lew.pitcher@digitalfreehold.ca to comp.unix.shell on Fri Aug 29 18:55:28 2025
    From Newsgroup: comp.unix.shell

    On Fri, 29 Aug 2025 17:45:36 +0000, Christian Weisgerber wrote:

    On 2025-08-29, Christian Weisgerber <naddy@mips.inka.de> wrote:

    Some sh implementations may exec() a final command as an optimization.
    Comparing sh -c '/bin/sleep 100', I see

    FreeBSD sh (Almquist derivate): fork + exec
    OpenBSD sh (PD ksh derivate): fork + exec
    GNU bash: exec

    Actually, I confused myself there. Allow me to revise:

    FreeBSD sh (Almquist derivate): exec
    OpenBSD sh (PD ksh derivate): fork + exec
    GNU bash: exec

    But notice that all three will switch to fork+exec if there is more
    than a single command, e.g. sh -c '/bin/sleep 100; /bin/sleep 200'.

    I /thought/ that my understanding of the actions of "-c" were off. Thanks
    for giving me the help.

    It looks like I can't guarantee that the "-c" option will exec() only. It appears that this is not a standard behaviour, and I don't want to pin
    my program to a specific shell.

    Thanks for the advice.
    --
    Lew Pitcher
    "In Skills We Trust"
    Not LLM output - I'm just like this.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Lew Pitcher@lew.pitcher@digitalfreehold.ca to comp.unix.shell on Fri Aug 29 19:01:53 2025
    From Newsgroup: comp.unix.shell

    Hi, Helmut


    On Fri, 29 Aug 2025 19:58:03 +0200, Helmut Waitzmann wrote:

    [snip]

    Please pardon my snippage of your excellent and detailed reply.
    Its going to take me a bit to go through your examples and
    process the alternatives.

    As I've replied to others, it appears that the behaviour I'm
    seeing in GNU bash isn't mandated by standards, and I won't
    depend on this behaviour for my refactoring efforts. But, I
    wanted to reply to a specific point that you bring up...


    Not the "-c" option but the "exec" shell builtin command will do
    what you want to be done:-a The "exec" shell builtin command tells
    the shell not to fork a new process but rather exec the binary
    replacing itself.

    Yes, I knew about the 'exec' builtin, and had considered using it.
    It looks like that might be the way to work my change, and I'll
    give it a try.


    Thanks for the advice and help.
    --
    Lew Pitcher
    "In Skills We Trust"
    Not LLM output - I'm just like this.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From John McCue@jmclnx@SPAMisBADgmail.com to comp.unix.shell on Fri Aug 29 19:55:10 2025
    From Newsgroup: comp.unix.shell

    Lew Pitcher <lew.pitcher@digitalfreehold.ca> wrote:
    Long ago, I played with Minix (v1.5), and wrote a SysV-style init(8) for it.
    <snip>

    I noticed the "-c" sh(1) commandline option, and have puzzled about it.
    From tests, it /appears/ to do what I want: parse the given string into argv[] arguments, then exec() that list, but the documentation doesn't
    really say that this behaviour is to be expected.

    From the OpenGroup's description of sh(1)[1], it /looks/ like sh(1) is supposed to fork() and exec(), and the Linux sh(1) manpage[2] is even less helpful about the behaviour of the -c option. However, limited tests on my Linux boxen show that bash(1) (at least version 4.3.48) appears to exec() only.

    So, can anyone advise me; does the sh(1) "-c" option cause shell to
    a) parse the given string into a commandline argument list, and then
    b) exec(2) that argument list, replacing sh(1) with the specified
    binary within the process?

    Assuming I understand what you are looking for,
    I issued this command:
    ksh -c "/bin/sleep 3600" &

    and did a "ps and got this result:

    PPID PID USER %CPU %MEM MAXRSS COMMAND
    2293 13544 jmccue 0.0 0.0 5060 -tcsh
    13544 19490 jmccue 0.0 0.0 1744 /bin/ps -wwwxo ...
    13544 25271 jmccue 0.0 0.0 1624 egrep 15489|13544
    13544 15489 jmccue 0.0 0.0 5060 ksh -c ksh <<<<<<
    15489 29050 jmccue 0.0 0.0 5060 /bin/sleep 3600 <<<<<<

    So at least in this case, sleep did not replace ksh

    <snip>

    But, I then issued this command
    ksh -c "exec /bin/sleep 3600" &

    and I got this from ps:

    PPID PID USER %CPU %MEM MAXRSS COMMAND
    2293 13544 jmccue 0.0 0.0 5060 -tcsh
    13544 8280 jmccue 0.0 0.0 1760 /bin/ps -wwwxo ...
    13544 12078 jmccue 0.0 0.0 1584 egrep 13544
    13544 10578 jmccue 0.0 0.0 5060 /bin/sleep 3600 <<<<<<

    looks like if you do a 'exec', you get what I think you want.
    --
    [t]csh(1) - "An elegant shell, for a more... civilized age."
    - Paraphrasing Star Wars
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Kaz Kylheku@643-408-1753@kylheku.com to comp.unix.shell on Fri Aug 29 20:29:01 2025
    From Newsgroup: comp.unix.shell

    On 2025-08-29, Lew Pitcher <lew.pitcher@digitalfreehold.ca> wrote:
    I noticed the "-c" sh(1) commandline option, and have puzzled about it.
    From tests, it /appears/ to do what I want: parse the given string into argv[] arguments, then exec() that list, but the documentation doesn't
    really say that this behaviour is to be expected.

    No the -c command line option takes an argument which is a string.

    That string is a shell script syntax.

    It can contain multiple commands, e.g.:

    sh -c 'for x in *; do echo "<$x>"; done; ls -ld .'

    If you're generating this argument in a shell script, you have
    to use proper quoting.

    The syntax can be single command with arguments, if you quote everything
    right, of course.

    If you want that command to replace the shell, you have to stick
    an exec on i

    sh -c 'exec cp foo.txt bar.txt'

    It's possible (and I suspect permissible, too) that shell
    implementations do this conversion to exec automatically.
    It's a form of tail call elimination, at the process level.

    If we know that the shell will execute a sequence of commands
    and then terminate, then if the last command (i.e. the one in the
    tail position) is an external command, it can be exec'd.
    --
    TXR Programming Language: http://nongnu.org/txr
    Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
    Mastodon: @Kazinator@mstdn.ca
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Lawrence =?iso-8859-13?q?D=FFOliveiro?=@ldo@nz.invalid to comp.unix.shell on Sat Aug 30 00:35:37 2025
    From Newsgroup: comp.unix.shell

    On Fri, 29 Aug 2025 15:09:25 -0000 (UTC), Lew Pitcher wrote:

    Fast forward to now, and I'm looking at recycling my old Minix
    init(8) for use in a Linux "homebrew" container environment, and I
    want to fix this commandline parsing clumsiness. My current thinking
    is that there may be some way to invoke sh(1) to perform the parsing
    and execution for me. My only oddball requirement is that sh(1) "get
    out of the way" (so to speak), and directly /exec()/ the resulting
    parsed commandline, rather than fork() and exec().

    I would say, donrCOt do it that way.

    If you look at other apps that let you specify commands that are
    slightly more complex than program-name-plus-arguments, they usually
    implement a very strict subset of shell syntax themselves. The idea is
    to avoid constructs that can lead to surprising behaviour (e.g. I/O redirections, pipelines and command substitution when you werenrCOt
    expecting it), which in turn could be a source of security
    vulnerabilities.

    For example, this <https://www.freedesktop.org/software/systemd/man/latest/systemd.service.html#Command%20lines>
    is how systemd does it. Good advice to follow:

    This syntax is inspired by shell syntax, but only the
    meta-characters and expansions described in the following
    paragraphs are understood, and the expansion of variables is
    different. Specifically, redirection using "<", "<<", ">", and
    ">>", pipes using "|", running programs in the background using
    "&", and other elements of shell syntax are not supported.
    --- Synchronet 3.21a-Linux NewsLink 1.2
  • From Lawrence =?iso-8859-13?q?D=FFOliveiro?=@ldo@nz.invalid to comp.unix.shell on Sat Aug 30 03:51:19 2025
    From Newsgroup: comp.unix.shell

    On Fri, 29 Aug 2025 18:09:19 +0100, Adam Sampson wrote:

    Neither sysvinit or busybox init prepends "exec" to the given command (although there's commented-out code in sysvinit suggesting that it did
    do this at some point). This has the advantage that you can write
    arbitrary complex shell commands in inittab, e.g. to set up the
    environment for a process before starting it.

    systemd breaks all that out into separate directives. Not only does that
    make it simpler to manage whatrCOs going on, it also allows things like the addition of directives to control the security environment in very fine- grained ways, e.g. namespace isolation, private temporary directory,
    making filesystems readonly, running under a non-root user with custom privilege setup, process quota limits.
    --- Synchronet 3.21a-Linux NewsLink 1.2