On 16.01.2025 00:26, Dan Cross wrote:

In article <vm9err$35gfs$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

[snip]
Since you're referring to me, the OP, please note that most arguments
here have quickly made a relation to a straw man (a performance theme)
or made other deviations from the basic question(s) that concerned me.

Essentially there were two questions I had that I can reformulate in a
more compact form as

"Why, in the first place, are all these path components
part of the default PATH for ordinary users? - Is there
any [functional] rationale or necessity for that?"

Not particularly.

The system has, probably for no really principled reason,
evolved over time such that that's simply the set of things that
are in $PATH by default on that particular machine. Another
machine might be different.

Yes, that is true and important. That's why it feels bad to change
it; the OS/distribution should provide a sensible definition.

If I had to hazard a guess, I imagine some of it comes from the
folks who put together the distribution, some upgrades, and the
choices of e.g. the window manager you're using (recalling that
at least one $PATH component appeared to come from that).

"_If_ many of the default PATH components are unnecessary,
where and how to best reduce these settings to a sensible
subset? - Without spoiling the system, of course."

There are many ways you could do this. Probably the easiest is
just to explicitly set $PATH in your shell's startup files to
those directories you care about; that's what I usually do. If
you make a mistake with it, you won't affect the rest of the
system.

Yes, I agree. That's what I had done in past days (but at home I
got sloppy).

If you want to set it globally for all users, there's likely
some file in /etc or similar that sets the defaults; on my Linux
machine I see a number of things in /etc/profile and
/etc/profile.d/* that seem relevant and there's /etc/login.defs;

I had inspected all the relevant system files on that and didn't
find anything, but maybe I missed it (I'll search once again).

PAM has its own way to set up $PATH. I'm not sure I'd bother,
though, if setting it up for your own account is sufficient.

Yeah, you're probably right that this is the simplest thing to do.

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 1/14/2025 8:55 AM, Dan Cross wrote:

... It seems to me that, as a normal user, the PATH

(and with it the path-search) could be drastically reduced. Is there
a method to only have them in the PATH when 'sudo'ing any programs
that require root privileges and the privileged programs in 'sbin'?

Yes, `sudo` can be configured to set $PATH for the programs that
it invokes; see sudoers(5) and look for `secure_path`. If you
don't invoke those from your normal shell, I don't see a problem
removing them from the default.

As for non-sudo PATH, you can always run:
export PATH=$(getconf PATH)
To get a minimal PATH setting guaranteed to include all POSIX
utilities.

--
Wayne

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 20.01.2025 09:05, Wayne wrote:

On 1/14/2025 8:55 AM, Dan Cross wrote:

... It seems to me that, as a normal user, the PATH

(and with it the path-search) could be drastically reduced. Is there
a method to only have them in the PATH when 'sudo'ing any programs
that require root privileges and the privileged programs in 'sbin'?

Yes, `sudo` can be configured to set $PATH for the programs that
it invokes; see sudoers(5) and look for `secure_path`. If you
don't invoke those from your normal shell, I don't see a problem
removing them from the default.

As for non-sudo PATH, you can always run:
export PATH=$(getconf PATH)
To get a minimal PATH setting guaranteed to include all POSIX
utilities.

Yes, this comes very close to what I was actually aiming at. In
my environment that's just /bin and /usr/bin and this is what I
typically need in my scripts (plus sometimes /usr/local/bin).
(For interactive use I'd add that too, and also my ~/bin, e.g.)
But all the sbin directories (and other clutter) would be gone.
It practically shows that these directories are unnecessary for
a normal (non-root) user. - Good hint. Thanks.

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 25.01.2025 16:35, Dan Cross wrote:

[...]

I don't usually use bash, but if you do and you really don't
like this behavior you can turn it off.

Neither do I use Bash. (Or rather; only rarely.)

[...]

Please don't put words in my mouth. [...]

Please accept my apology.

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 26.01.2025 02:36, Keith Thompson wrote:

[...]
As far as I can tell (I haven't investigated), all Bourne-based
shells other than Bash treat $PATH in the same way when executing
commands. Bash is the only shell I'm aware of that expands literal
'~' in $PATH.

The fact that it's not POSIX compliant behavior is at worst mildly
annoying. Bash correctly disables it in POSIX mode (enabled by the
--posix option, by running "set -o posix", by setting $POSIXLY_CORRECT
before starting bash, or by invoking it sas "sh").

The problem I see with that is that you also disable other features,
sensible features, that are common in the prominent shells and that
you don't want to miss. - So Bash users, if they don't want to get
bitten in a subtle way, are probably advised to be pointed out to
that behavior, or, as suggested by others already, not to use tilde
at all with PATH in Bash.

I dislike the behavior because (a) it can quietly cause unexpected
problems (it's easy to miss the fact that $HOME is expanded within
double quotes and ~ is not), and (b) it's not documented. I certainly consider it a misfeature. [...]

The consequences are yet more fundamental; if you observe the behavior
of other quotings as well...

Whether PATH='~' or PATH="~", both are stored as literal value in all
shells (including Bash!). - That's fine.

Where PATH='$HOME' is (as expected) stored as a literal value in all
shells (also in Bash) and PATH="$HOME" is (as expected) expanded in
all shells (also in Bash) and store in expanded form. - That's also
fine.

All the variable storage and quoting is fine and consistent across
all shells (including Bash).

But in Bash a literal '$HOME' doesn't get expanded during path search.
And you see '~' in a PATH as being stored literally but nonetheless it
gets expanded in the path search. - In other words; the problem is not
the quoting.

That's IMO so stupid a design that I can only explain it with no one
having been courageous enough over the many decades to fix that old implementation. - Of course, the later you consider to fix such a
[mis-]design the harder you have it; development people are concerned
about software that is relying on that and thus reluctant to fix it.)

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 26.01.2025 03:07, Keith Thompson wrote:

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

On 24.01.2025 23:00, Keith Thompson wrote:
[...]

I mean that a shell should behave consistently. (I think Bash does
not in the given case.)

Consistently with what? Bash consistently expands literal '~'s in
$PATH, and consistently disables that expansion in POSIX mode.

I think I have (already repeatedly) expanded on that, so let me just
enumerate some obvious aspects sparsely...
- expanding tilde once at dedicated places vs. twice
- consistency with other expansions (not done twice), e.g. '$HOME'
- (and if you accept that) consistency with the other shells

(Any single inconsistency is IMO already worthwhile to be critically discussed.)

Janis

[...]

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 26.01.2025 06:26, Kaz Kylheku wrote:

It's a feature that (if used) leaks tildes into child processes via
the environment variable. Path resultion in child processes, if it
reaches a PATH element with a tilde, will somehow process that tilde.

I just tried this experiment. I made a directory named ~ and put ~:
as the leading element of PATH. I put a program called "foo" that
directory.

Surely enough, I can run "foo" from the parent directory above.

The exec functions treat ~ as an ordinary path component.

(I cannot do that out of Bash, which processes the tilde, but
the 'p' family of the exec functions will find it!)

This is a problem similar to "." being in PATH.

[ Above context preserved for integrity. ]

If someone has, say, "~/bin" in their PATH, ahead of /bin and /usr/bin,
I can put a malicious program in some directory called "~/bin"
somewhere in the filesystem, give that program the name of a common
external utility, and trick the user into changing into that location
where they will run this common command, resolving to my malicious
program.

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.
So there would, IMO, not be a security hole (i.e. not because of that).

If we regard this as a security hole, that atually raises the priority
and bolsters the argument that it ought to be removed even if it
breaks some users, perhaps through a process of noisy deprecation.

Furhermore, the case can be made that the exec stuff in the Linux kernel
or C libraries should be patched with a check against components with a leading tilde.

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-26, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.
So there would, IMO, not be a security hole (i.e. not because of that).

The / is not part of a name; it is acting as the path component
separator. PATH allows relative paths with multiple components.
This is real!

$ PATH='~/bin':$PATH
$ cat '~/bin/cat' # script I prepared
#!/bin/sh
echo BOO!
$ cat # Bash in stock mode finds /bin/cat
^C
!130!
$ bash --posix # Bash in --posix mode falls victim
bash-4.4$ cat
BOO!
bash-4.4$

Someone who has literal ~/bin in their PATH so that their shell finds
utilities in their personal bin is about just as susceptible to an
attack as someone who has . in their PATH.

They might be even more susceptible. Here is why. Someone who has
~/bin in their PATH probably has a personal bin containing
custom programs with names distinct from standard utilities.
I have such things myself, such as "tagify".

The attack can be perpetrated using the name "tagify" regardless
of where the user has placed '~/bin' into PATH: first or last,
because that name is not found in any of the other locations
listed in PATH.

If we happen to have access to the user's ~/bin directory, we can set a
trap for each of the custom program names we find there.

Whereas the . exploit typically requires . to be ahead of
the standrad locations rather than as a fallback, and the trap is set
using some common utilities (ones not built into the user's shell).

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vmispd$27pfq$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 16.01.2025 00:26, Dan Cross wrote:
[snip]

I'm not sure I'd bother,
though, if setting it up for your own account is sufficient.

Yeah, you're probably right that this is the simplest thing to do.

Sounds good. Two other benefits here are that you can easily
add other components should you want or need to, and it's easy
to roll back if you make a mistake.

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

On 20.01.2025 09:05, Wayne wrote:

As for non-sudo PATH, you can always run:
export PATH=$(getconf PATH)
To get a minimal PATH setting guaranteed to include all POSIX
utilities.

Yes, this comes very close to what I was actually aiming at. In
my environment that's just /bin and /usr/bin and this is what I
typically need in my scripts (plus sometimes /usr/local/bin).

I am a late-comer to an interesting thread ...

I think that, depending on your Linux distribution, /bin and /sbin are
symbolic links to their respective /usr counterparts (in Debian there is
a "usrmerge" package to help with this transition). Also, depending on
the adherence to file hierarchy standards, there are different policies
when it comes to /opt and /usr/local (the latter here has no files, only
empty directories).

So a tidy minimalist might opt for /bin only (plus, if needed,
$HOME/bin, which is better than ~/bin, because tilde expansion is not,
AFAIK, included in POSIX) as non-root and /bin:/sbin else.

But all the sbin directories (and other clutter) would be gone. It practically shows that these directories are unnecessary for a normal (non-root) user.

Well, /usr/games is populated here ... (-:

Best regards

Axel

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 21.01.2025 07:52, Axel Reichert wrote:

[...]

I am a late-comer to an interesting thread ...

I think that, depending on your Linux distribution, /bin and /sbin are symbolic links to their respective /usr counterparts [...]

In my environment these are not only distinct directories but also
contain different sets of executables; the "ls | wc -l" values are:
/bin - 164
/sbin - 248
/usr/bin - 3230
/usr/sbin - 307

So a tidy minimalist might opt for /bin only

Most "interesting" functions for the user are in /usr/bin. But note
that 'rm' for example will not be found there; the path-search will
(also according to strace) need to pass /usr/bin and access /bin to
find it. (That was actually something, maybe "[mostly] harmless",
that slightly unsettled me.)

(plus, if needed,
$HOME/bin, which is better than ~/bin, because tilde expansion is not,
AFAIK, included in POSIX) as non-root and /bin:/sbin else.

I used "~/bin" here just sort of informally due to my laziness. (In
my .profile I've used the variables $HOME/bin.)

In shell scripting or setups I generally use no '~' expressions at
all. (I use tilde-syntax only interactively, or in communication.)

But all the sbin directories (and other clutter) would be gone. It
practically shows that these directories are unnecessary for a normal
(non-root) user.

Well, /usr/games is populated here ... (-:

Well, the 50+ games I had installed there I invoke by the GUI menu
(so no PATH component would be necessary for that in my case). But,
to be honest, I've almost never played all these games. (The games
I regularly play are the Roguelikes; either separately compiled, or,
nowadays more common, played over 'ssh' on some public server. :-)

Janis

Best regards

Axel

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

cross@spitfire.i.gajendra.net (Dan Cross) writes:

[Meta note: This is more of a comp.unix.shell sort of post; not
so much comp.unix.programmer. Followup-To: set accordingly.]

In article <vm5dei$2c7to$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

When I recently inspected an 'strace' log and saw the huge amount
of system-calls done for a simple standard command (like 'rm') -
it's more than a dozen! and most lead just to ENOENT - I wondered
about the default PATH definition which is for my system
/usr/lib/lightdm/lightdm
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/usr/games
(here I'm omitting my own additions, '~/bin' and '.', and I separated
them, one on each line for a better visualization of the "problem" or, >>maybe better, for the "questions".)

On a single-user system, it's not a huge deal, but on a
multiuser system where you may `cd` into a directory writable by
anyone (such as /tmp), `.` in $PATH is a known security problem.
YMMV, but caveat emptor.

The above PATH components are for a terminal running under some
window manager, a plain console window will not show the 'lightdm'
entry (but I rarely work on plain consoles).

This raises a few questions, and someone may shed some light on the >>rationale for above default settings... (and how to "fix" it best)

Why do I need 'lightdm/lightdm' in the user's PATH variable defined?
(That directory contains just one special script and one executable.)
This entry is what annoys me most; it also reminds me of systems that
have every program vendor add an own PATH entry for their products.
Would it be safe to just remove that (in my '~/.profile') from PATH?
Or can I make it vanish by some other change, to not appear in the
in the PATH first place? (Of course without destabilizing the system
by that.)

If you don't feel like you need to run that executable, and the
window manager works ok without it, I don't see why it would be
a problem to remove it from $PATH.

There's no files in '/usr/local/sbin' (on my system); no admins with >>special tools desires.

I don't seem to use executables from all the 'sbin' directories; I'm >>positive I need /usr/bin, /bin, and I've also installed some things
in /usr/local/bin. It seems to me that, as a normal user, the PATH
(and with it the path-search) could be drastically reduced. Is there
a method to only have them in the PATH when 'sudo'ing any programs
that require root privileges and the privileged programs in 'sbin'?

Yes, `sudo` can be configured to set $PATH for the programs that
it invokes; see sudoers(5) and look for `secure_path`. If you
don't invoke those from your normal shell, I don't see a problem
removing them from the default.

I mean, if I 'sodo' a shell I get - and I think this is sensible! -
only /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
(no 'lightdm', no 'games', and no personal settings) anyway, and I
seem to have those entries available independent of any parent
process's setting; PATH=/usr/bin sudo ksh will still provide all
the 'sbin' directories in the privileged shell's own PATH setting.

So my thought is, for the moment as a workaround, to edit the PATH
in the .profile, and _remove_ all 'sbin' and the 'lightdm' entries,
or just explicitly _define_ PATH without the spurious parts). (Or
would it be advisable to do that change in all the shells' .rc
files?) Or is there yet a better place to "fix" things system-wide?

(Or better not touch a running system? - but it looks so messy!)

Personally, I'd let well enough alone, but I suppose this
alludes to a larger question: does having those entries in $PATH
affect the operation of the system in any materially negative
way? Is this just a preference for tidiness kind of thing?

I generally optimize the $PATH such that most of the common
commands will hit an early component, and while one might wish
to override a system command at times, if necessary, a shell
function can be used rather than relying on the position of
the target directory in $PATH.

Note that most shells (e.g. ksh) will cache any hits and leverage the cache
on subsequent invocations.

If you 'strace' any gnome or kde application, you'll find hundreds
of failed open calls looking for shared objects, locales, caches,
and other collateral. I'd not be too worried about the impact
of extra components in the $PATH variable.

There's no harm in cleaning up, but I suspect any marginal
resource savings has already been offset by thinking about it
at all. :-)

What is the desired end-state here?

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 14/01/2025 13:55, Dan Cross wrote:

[Meta note: This is more of a comp.unix.shell sort of post; not
so much comp.unix.programmer. Followup-To: set accordingly.]

In article <vm5dei$2c7to$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

When I recently inspected an 'strace' log and saw the huge amount
of system-calls done for a simple standard command (like 'rm') -
it's more than a dozen! and most lead just to ENOENT - I wondered
about the default PATH definition which is for my system
/usr/lib/lightdm/lightdm
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/usr/games
(here I'm omitting my own additions, '~/bin' and '.', and I separated
them, one on each line for a better visualization of the "problem" or,
maybe better, for the "questions".)

On a single-user system, it's not a huge deal, but on a
multiuser system where you may `cd` into a directory writable by
anyone (such as /tmp), `.` in $PATH is a known security problem.
YMMV, but caveat emptor.

The above PATH components are for a terminal running under some
window manager, a plain console window will not show the 'lightdm'
entry (but I rarely work on plain consoles).

This raises a few questions, and someone may shed some light on the
rationale for above default settings... (and how to "fix" it best)

Why do I need 'lightdm/lightdm' in the user's PATH variable defined?
(That directory contains just one special script and one executable.)
This entry is what annoys me most; it also reminds me of systems that
have every program vendor add an own PATH entry for their products.
Would it be safe to just remove that (in my '~/.profile') from PATH?
Or can I make it vanish by some other change, to not appear in the
in the PATH first place? (Of course without destabilizing the system
by that.)

If you don't feel like you need to run that executable, and the
window manager works ok without it, I don't see why it would be
a problem to remove it from $PATH.

There's no files in '/usr/local/sbin' (on my system); no admins with
special tools desires.

I don't seem to use executables from all the 'sbin' directories; I'm
positive I need /usr/bin, /bin, and I've also installed some things
in /usr/local/bin. It seems to me that, as a normal user, the PATH
(and with it the path-search) could be drastically reduced. Is there
a method to only have them in the PATH when 'sudo'ing any programs
that require root privileges and the privileged programs in 'sbin'?

I'd say that anything in 'sbin' should be considered here-be-dragons, if
you really want to run that, then you should be made to type the whole
path - just as a safety net.

Yes, `sudo` can be configured to set $PATH for the programs that
it invokes; see sudoers(5) and look for `secure_path`. If you
don't invoke those from your normal shell, I don't see a problem
removing them from the default.

I mean, if I 'sodo' a shell I get - and I think this is sensible! -
only /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
(no 'lightdm', no 'games', and no personal settings) anyway, and I
seem to have those entries available independent of any parent
process's setting; PATH=/usr/bin sudo ksh will still provide all
the 'sbin' directories in the privileged shell's own PATH setting.

So my thought is, for the moment as a workaround, to edit the PATH
in the .profile, and _remove_ all 'sbin' and the 'lightdm' entries,
or just explicitly _define_ PATH without the spurious parts). (Or
would it be advisable to do that change in all the shells' .rc
files?) Or is there yet a better place to "fix" things system-wide?

(Or better not touch a running system? - but it looks so messy!)

Personally, I'd let well enough alone, but I suppose this
alludes to a larger question: does having those entries in $PATH
affect the operation of the system in any materially negative
way? Is this just a preference for tidiness kind of thing?

There's no harm in cleaning up, but I suspect any marginal
resource savings has already been offset by thinking about it
at all. :-)

What is the desired end-state here?

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Axel Reichert wrote:

tilde expansion is not, AFAIK, included in POSIX

Incorrect. See XCU 2.6.1 Tilde Expansion, which includes the following:

In an assignment (see XBD Section 4.26), multiple tilde-prefixes
can be used: one at the beginning of the word (that is, following
the <equals-sign> of the assignment), or one following any unquoted
<colon>, or both.

The reason this is there is for things like PATH=~/bin:~/sbin:/bin:...

--
Geoff Clare <netnews@gclare.org.uk>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-22, Geoff Clare <geoff@clare.See-My-Signature.invalid> wrote:

Axel Reichert wrote:

tilde expansion is not, AFAIK, included in POSIX

Incorrect. See XCU 2.6.1 Tilde Expansion, which includes the following:

In an assignment (see XBD Section 4.26), multiple tilde-prefixes
can be used: one at the beginning of the word (that is, following
the <equals-sign> of the assignment), or one following any unquoted
<colon>, or both.

The reason this is there is for things like PATH=~/bin:~/sbin:/bin:...

But surely that must be expanded before PATH is assigned.
PATH is processed by the "p" exec functions like execlp. I don't
think these perform tilde expansion!

So the reason for using tilde in PATH assignments is the same reason
as using them in any other shell programming situation; to save
keystrokes over typing $HOME.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Kaz Kylheku wrote:

On 2025-01-22, Geoff Clare <geoff@clare.See-My-Signature.invalid> wrote:

Axel Reichert wrote:

tilde expansion is not, AFAIK, included in POSIX

Incorrect. See XCU 2.6.1 Tilde Expansion, which includes the following:

In an assignment (see XBD Section 4.26), multiple tilde-prefixes
can be used: one at the beginning of the word (that is, following
the <equals-sign> of the assignment), or one following any unquoted
<colon>, or both.

The reason this is there is for things like PATH=~/bin:~/sbin:/bin:...

But surely that must be expanded before PATH is assigned.

Yes, of course.

PATH is processed by the "p" exec functions like execlp. I don't
think these perform tilde expansion!

Correct.

So the reason for using tilde in PATH assignments is the same reason
as using them in any other shell programming situation; to save
keystrokes over typing $HOME.

Yes. Perhaps I trimmed too much. The post I was replying to said
"$HOME/bin [..] is better than ~/bin, because tilde expansion is not,
AFAIK, included in POSIX" and $HOME is also, of course, expanded before
PATH is assigned. So there is no reason to prefer $HOME/bin over ~/bin
since (when used in an assignment) they are equivalent in POSIX.

--
Geoff Clare <netnews@gclare.org.uk>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <ccr96l-eot.ln1@ID-313840.user.individual.net>,
Geoff Clare <netnews@gclare.org.uk> wrote:
...

Yes. Perhaps I trimmed too much. The post I was replying to said
"$HOME/bin [..] is better than ~/bin, because tilde expansion is not,
AFAIK, included in POSIX" and $HOME is also, of course, expanded before
PATH is assigned. So there is no reason to prefer $HOME/bin over ~/bin
since (when used in an assignment) they are equivalent in POSIX.

1) I have no idea what your beef with Kaz is. It seems silly at best.

2) I know this isn't going to sit well with you, but there absolutely are situations (in bash) where ~ doesn't work as a substitute for $HOME, when setting the various "path" variables. I know that I had lines in .profile and/or .bashrc like: PATH=~/bin:$PATH (and similar) that did not work (that
is, the value stored in the variable contained an explicit tilde rather
than the expanded value - and this, of course, doesn't work at runtime). Replacing the tilde with $HOME fixes this.

Sorry I don't have details, but it is true nonetheless.

--
A 70 year old man who watches 6 hours of TV a day, plays a lot of golf
and seems to always be in Florida is a retiree, not a president.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vmthmu$3bb88$1@news.xmission.com>,
Kenny McCormack <gazelle@shell.xmission.com> wrote:

In article <ccr96l-eot.ln1@ID-313840.user.individual.net>,
Geoff Clare <netnews@gclare.org.uk> wrote:
...

Yes. Perhaps I trimmed too much. The post I was replying to said >>"$HOME/bin [..] is better than ~/bin, because tilde expansion is not, >>AFAIK, included in POSIX" and $HOME is also, of course, expanded before >>PATH is assigned. So there is no reason to prefer $HOME/bin over ~/bin >>since (when used in an assignment) they are equivalent in POSIX.

1) I have no idea what your beef with Kaz is. It seems silly at best.

The response you're replying to seemed pretty collegial to me;
am I missing something?

2) I know this isn't going to sit well with you, but there absolutely are >situations (in bash) where ~ doesn't work as a substitute for $HOME, when >setting the various "path" variables. I know that I had lines in .profile >and/or .bashrc like: PATH=~/bin:$PATH (and similar) that did not work (that >is, the value stored in the variable contained an explicit tilde rather
than the expanded value - and this, of course, doesn't work at runtime). >Replacing the tilde with $HOME fixes this.

Sorry I don't have details, but it is true nonetheless.

I think the issue is that one wants to prevent a literal string
like `~/bin` from showing up in $PATH. While a given shell
_may_ interpret such $PATH components as being relative to the
user's home directory (`bash` does) others may not (`ksh` does
not, at least not by default).

Further, C library functions like `execlp`, `execvp` and
`execvpe` that refer to $PATH may not handle `~` expansion, and
are not mandated to where defined by e.g. POSIX. Similarly for
analogous functions in other programming languages. Given that
programs that make use of those functions inherit $PATH when
invoked from a shell, this can lead to unexpected behavior in a
way that's hard to diagnose ("it works fine when I run this
program from the shell, but my editor can't find it!").

Also, it's not the case that shells always expand `~` when
setting $PATH, even when $HOME would be expanded, as Kenny
points out. Here's an example:

: term; echo $HOME
/home/cross
: term; pwd
/home/cross
: term; echo echo xyzzy >bin/quux
: term; chmod +x bin/quux
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; exec ksh
: term; export PATH=~/bin:/bin:/usr/bin
: term; echo $PATH
/home/cross/bin:/bin:/usr/bin
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; export PATH="$HOME/bin:/bin:/usr/bin"
: term; echo $PATH
/home/cross/bin:/bin:/usr/bin
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; export PATH="~/bin:/bin:/usr/bin"
: term; echo $PATH
~/bin:/bin:/usr/bin
: term; which quux
: term; quux
ksh: quux: not found
: term; exec /usr/pkg/bin/bash
: term; echo $PATH
~/bin:/bin:/usr/bin
: term; which quux
: term; quux
xyzzy
: term;

I suppose the moral is, for maximal portability, prefer using
$HOME (or another similar variable) to `~` when setting $PATH
unless you're doing so in a context where you are sure that `~`
will be expanded during assignment.

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 23.01.2025 17:47, Dan Cross wrote:

In article <vmthmu$3bb88$1@news.xmission.com>,
Kenny McCormack <gazelle@shell.xmission.com> wrote:

In article <ccr96l-eot.ln1@ID-313840.user.individual.net>,
Geoff Clare <netnews@gclare.org.uk> wrote:
...

Yes. Perhaps I trimmed too much. The post I was replying to said
"$HOME/bin [..] is better than ~/bin, because tilde expansion is not,
AFAIK, included in POSIX" and $HOME is also, of course, expanded before
PATH is assigned. So there is no reason to prefer $HOME/bin over ~/bin
since (when used in an assignment) they are equivalent in POSIX.

1) I have no idea what your beef with Kaz is. It seems silly at best.

The response you're replying to seemed pretty collegial to me;
am I missing something?

2) I know this isn't going to sit well with you, but there absolutely are
situations (in bash) where ~ doesn't work as a substitute for $HOME, when
setting the various "path" variables. I know that I had lines in .profile >> and/or .bashrc like: PATH=~/bin:$PATH (and similar) that did not work (that >> is, the value stored in the variable contained an explicit tilde rather
than the expanded value - and this, of course, doesn't work at runtime).
Replacing the tilde with $HOME fixes this.

Sorry I don't have details, but it is true nonetheless.

I think the issue is that one wants to prevent a literal string
like `~/bin` from showing up in $PATH. While a given shell
_may_ interpret such $PATH components as being relative to the
user's home directory (`bash` does) others may not (`ksh` does
not, at least not by default).

Further, C library functions like `execlp`, `execvp` and
`execvpe` that refer to $PATH may not handle `~` expansion, and
are not mandated to where defined by e.g. POSIX. Similarly for
analogous functions in other programming languages. Given that
programs that make use of those functions inherit $PATH when
invoked from a shell, this can lead to unexpected behavior in a
way that's hard to diagnose ("it works fine when I run this
program from the shell, but my editor can't find it!").

Also, it's not the case that shells always expand `~` when
setting $PATH, even when $HOME would be expanded, as Kenny
points out. Here's an example:

: term; echo $HOME
/home/cross
: term; pwd
/home/cross
: term; echo echo xyzzy >bin/quux
: term; chmod +x bin/quux
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; exec ksh
: term; export PATH=~/bin:/bin:/usr/bin
: term; echo $PATH
/home/cross/bin:/bin:/usr/bin
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; export PATH="$HOME/bin:/bin:/usr/bin"
: term; echo $PATH
/home/cross/bin:/bin:/usr/bin
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; export PATH="~/bin:/bin:/usr/bin"
: term; echo $PATH
~/bin:/bin:/usr/bin
: term; which quux
: term; quux
ksh: quux: not found
: term; exec /usr/pkg/bin/bash
: term; echo $PATH
~/bin:/bin:/usr/bin
: term; which quux
: term; quux
xyzzy
: term;

Bash behaves strange here; 'which' doesn't find the executable but
nonetheless bash executes it, shows its output?

I've changed the above test frame to remove some unnecessary parts
to eliminate distracting complexity. The output for three shells:

Bash:
/home/jap/bin/quux
xyzzy
/home/jap/bin/quux
xyzzy
xyzzy
# nothing returned from 'which' (silent), *no error* on call [***]

Ksh:
/home/jap/bin/quux
xyzzy
/home/jap/bin/quux
xyzzy
ksh: quux: not found
# nothing returned from 'which' (silent), error on call

Zsh:
/home/jap/bin/quux
xyzzy
/home/jap/bin/quux
xyzzy
quux not found
zsh:1: command not found: quux
# diagnostic returned from 'which', error on call

I'm not sure whether that effect is (as was suspected) an effect of
a quoted '~'; it seems to be some different problem, in Bash.[***]

For all these shells a quoted '~' as in the PATH assignation will be
stored as a literal and will not get expanded; so far, so expected.

A quoted '~' will not get expanded, so the result is not surprising.
Tilde expansion comes before quote removal. (In Ksh, to be sure, and
obviously also in the other shells, and I'd suppose also according
to POSIX.)

I suppose the moral is, for maximal portability, prefer using
$HOME (or another similar variable) to `~` when setting $PATH
unless you're doing so in a context where you are sure that `~`
will be expanded during assignment.

IMO, the consequence is to not quote a tilde expression in the first
place if you want it expanded in a shell variable. *Violating* that
rule will _in Bash_ produce this strange effect (in the given setup).

To me it appears that not finding (by 'which') an executable but
executing it nonetheless qualifies as a bug [in Bash].

My 2 cents.

Janis

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-23, Kenny McCormack <gazelle@shell.xmission.com> wrote:

In article <ccr96l-eot.ln1@ID-313840.user.individual.net>,
Geoff Clare <netnews@gclare.org.uk> wrote:
...

Yes. Perhaps I trimmed too much. The post I was replying to said >>"$HOME/bin [..] is better than ~/bin, because tilde expansion is not, >>AFAIK, included in POSIX" and $HOME is also, of course, expanded before >>PATH is assigned. So there is no reason to prefer $HOME/bin over ~/bin >>since (when used in an assignment) they are equivalent in POSIX.

1) I have no idea what your beef with Kaz is. It seems silly at best.

Beef? I don't perceive any beef. If anything, Geoff seems to be setting
an example for me to be less of an asshole. :)

At this stage and age, it's unlikely to work, but you never know.

I know that I had lines in .profile
and/or .bashrc like: PATH=~/bin:$PATH (and similar) that did not work (that is, the value stored in the variable contained an explicit tilde rather
than the expanded value - and this, of course, doesn't work at runtime). Replacing the tilde with $HOME fixes this.

The obvious situation is double quotes. Inside double quotes, parameter expansion happens, but not tilde expansion (not to mention pathname
expansion (globbing) and perhaps some other things).

So this won't work:

PATH="$PATH:~/bin"

Sorry I don't have details, but it is true nonetheless.

There they are.

By the way, the fact of tilde expansion not happening in double
quotes is a strikes against the Bash ~N feature for
referring to directories in the directory stack (pushd, popd).

My "cdlog" module, consisting entirely of shell scriptology outside of
the Bash executable, is far better.

In "cdlog" we refer to the third directory in the log (for instance)
as $c3. Additionally, the first four entries $c1 to $c4 are
also available as $x, $y, $z and $w: same number of keystrokes
as ~1 ... ~4.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-23, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

Bash behaves strange here; 'which' doesn't find the executable but nonetheless bash executes it, shows its output?

which is a nonstandard command; the POSIX command is type.

In those systems where "which" exists at all, it is often
a locally brewed program that is not exactly the same like
the one in other systems.

In Debian and derivatives thereof, /usr/bin/which is a shell script.

I see that in MacOS, there is a /usr/bin/which whose --help
says to send mail to which-bugs<at>gnu.org; the man page
implicates a Carlo Wood as the culprit behind it.
It doesn't appear to be part of GNU Coreutils.

Which which is which? Burn the which!

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-23 at 21:34 Janis Papanagnou wrote:

Bash behaves strange here; 'which' doesn't find the executable but nonetheless bash executes it, shows its output?
[…]
To me it appears that not finding (by 'which') an executable but
executing it nonetheless qualifies as a bug [in Bash].

It's not a bug, it's just a bash feature you don't kno^H^H^H remember
until it bites you ;). Time to reread 'COMMAND EXECUTION' section
in bash manual…

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-24, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:

Here's my bug report:

https://lists.gnu.org/archive/html/bug-bash/2025-01/msg00114.html

The documentation's wording that is second most relevant to this
issue is:

"Each variable assignment is checked for unquoted tilde-prefixes
immediately following a ':' or the first '='. In these cases, tilde
expansion is also performed. Consequently, one may use filenames with
tildes in assignments to PATH, MAILPATH, and CDPATH, and the shell
assigns the expanded value."

In particular, the "Consequently," suggests that it is /as a
consequence/ of the above requirements (expansion of unescaped tildes in variable assignments), and no other requirements, that it is possible to
tildes in PATH!

I say second most relevant, because I the most relevant passage in
the GNU Info manual:

6.11 Bash POSIX Mode

[ ... ]

The following list is what's changed when 'POSIX mode' is in effect:

1. [ ... ]

[ ... ]

19. Literal tildes that appear as the first character in elements of
the PATH variable are not expanded as described above under
Tilde Expansion.

Aha! This is the very feature: leading literal tildes in the PATH
elements being expanded, disabled by POSIX mode (such as invocation
as #!/bin/sh).

But Tilde Expansion mentions no such thing!

That's where the documentation would have to be, to fix the
dangling reference above.

Still, the requirement itself can be deemed as being documented by point
19 above; it implies that when Bash is in POSIX mode, tildes are somehow expanded. Though the promised detailed description is missing, it
cannot reasonably be anything other than replacement by $HOME,
which is the actual behavior that can be inferred from experiments.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 23 Jan 2025 21:24:47 -0800, Keith Thompson wrote:

it cannot reasonably be anything other than replacement by $HOME,
which is the actual behavior that can be inferred from experiments.

According to my experiments, a “~” in a $PATH component not immediately followed by a username is expanded, not to $HOME, but to the home
directory associated with the current user ID.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 23.01.2025 22:50, Keith Thompson wrote:

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

[...]

Bash behaves strange here; 'which' doesn't find the executable but
nonetheless bash executes it, shows its output?

I've changed the above test frame to remove some unnecessary parts
to eliminate distracting complexity. The output for three shells:

[snip]

I'm not sure whether that effect is (as was suspected) an effect of
a quoted '~'; it seems to be some different problem, in Bash.[***]

For all these shells a quoted '~' as in the PATH assignation will be
stored as a literal and will not get expanded; so far, so expected.

A quoted '~' will not get expanded, so the result is not surprising.
Tilde expansion comes before quote removal. (In Ksh, to be sure, and
obviously also in the other shells, and I'd suppose also according
to POSIX.)

I suppose the moral is, for maximal portability, prefer using
$HOME (or another similar variable) to `~` when setting $PATH
unless you're doing so in a context where you are sure that `~`
will be expanded during assignment.

IMO, the consequence is to not quote a tilde expression in the first
place if you want it expanded in a shell variable. *Violating* that
rule will _in Bash_ produce this strange effect (in the given setup).

To me it appears that not finding (by 'which') an executable but
executing it nonetheless qualifies as a bug [in Bash].

I wouldn't call it a bug in bash. Rather, it's a mismatch between bash
and which.

I'm not sure I can follow you what you mean by "a mismatch". Kornshell,
for example, also accesses /usr/bin/which (but it behaves correctly).

"which" is an external command, not a shell builtin. [...]

Same in Ksh. Not so in Zsh. - But both of these shells behave correctly.

"type" is a bash builtin that mostly does what "which" does, but it also
has access to all of bash's internal information (functions, aliases, keywords, builtin functions). "type" is specified by POSIX for sh, with
a description that implies it must be a shell builtin command. The bash "type" command has several options as well.

One reason to prefer 'type'.

I'm not sure why bash users use "which" rather than "type". I find
"type" far more useful.

Indeed. - For an explanation of the preference of 'which'; years long experienced users may be used to 'which' back from 'csh' times. - I'm,
for example, primarily a Ksh user; nonetheless, instead of using Ksh's
'whence' or the standard 'type', my quick finger memory (often) types
'which'.

The weirdness here is that, [...]

Another thing to remember is that variable references are expanded
within double quotes, but ~ is not:

Yes, that's what I had explicitly pointed out in my post. - But Bash
as well does this specific thing correctly.

Janis

$ echo "$HOME ~"
/home/kst ~

Going off on a tangent, there's a recent trend to make /bin a symlink to /usr/bin -- but the default $PATH on most systems still includes both /usr/bin and /bin. I wonder why that wasn't cleaned up.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 23.01.2025 23:46, Keith Thompson wrote:

Kaz Kylheku <643-408-1753@kylheku.com> writes:
[...]

The obvious situation is double quotes. Inside double quotes, parameter
expansion happens, but not tilde expansion (not to mention pathname
expansion (globbing) and perhaps some other things).

So this won't work:

PATH="$PATH:~/bin"

Sorry I don't have details, but it is true nonetheless.

There they are.

[...]

In fact it probably will *partially* work. As I mentioned elsethread,
bash expands a leading ~ (or even ~username) in an element of $PATH when executing a command.

Interpretation of any _leading_ '~' is the documented behavior for
tilde expansion.

But if you run a program from the command line that invokes another
program (say, a C program that calls system()), it won't treat that
element of $PATH the same way.

I think the point is that the _shell_ expands the expression.

Whether you do PATH="~" or PATH='~' or PATH=\~ , in all cases you have
defined a variable value that contains '~', a tilde. Not an expanded tilde-expression. So if you 'exec' a command that PATH variable (with
the literal tilde) will get passed to the executable environment (and
no shell interpretation takes place).

Use PATH=~ instead if you want it expanded.

For this and other reasons, though you *can* have a literal ~ in $PATH
in bash, it's best to avoid it and use $HOME instead.

Or use it correctly, unquoted and unescaped.

Janis

A literal '$HOME'
won't work at all, but that's less likely to be a problem if your at all aware of how double quotes work in the shell.

I suggest that bash's undocumented behavior is less than helpful.
I'll probably submit a bug report.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 23.01.2025 23:06, Kaz Kylheku wrote:

Which which is which? Burn the which!

:-)

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Kenny McCormack wrote:

In article <ccr96l-eot.ln1@ID-313840.user.individual.net>,
Geoff Clare <netnews@gclare.org.uk> wrote:
...

Yes. Perhaps I trimmed too much. The post I was replying to said >>"$HOME/bin [..] is better than ~/bin, because tilde expansion is not, >>AFAIK, included in POSIX" and $HOME is also, of course, expanded before >>PATH is assigned. So there is no reason to prefer $HOME/bin over ~/bin >>since (when used in an assignment) they are equivalent in POSIX.

1) I have no idea what your beef with Kaz is. It seems silly at best.

No beef. Quite the opposite in fact - I was attempting to be apologetic
for having written something that could be misinterpreted (because I
trimmed too much).

2) I know this isn't going to sit well with you, but there absolutely are situations (in bash) where ~ doesn't work as a substitute for $HOME, when setting the various "path" variables. I know that I had lines in .profile and/or .bashrc like: PATH=~/bin:$PATH (and similar) that did not work (that is, the value stored in the variable contained an explicit tilde rather
than the expanded value - and this, of course, doesn't work at runtime). Replacing the tilde with $HOME fixes this.

Sorry I don't have details, but it is true nonetheless.

As others have already pointed out, it's probably because you quoted the
tilde. Which means when I carefully put the caveat "when used in an assignment" into what I wrote about equivalence of $HOME/bin and ~/bin,
I wasn't careful enough - I should have written "when used in an
unquoted assignment".

--
Geoff Clare <netnews@gclare.org.uk>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vmu94j$1q2lp$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 23.01.2025 17:47, Dan Cross wrote:

[snip]

: term; echo $HOME
/home/cross
: term; pwd
/home/cross
: term; echo echo xyzzy >bin/quux
: term; chmod +x bin/quux
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; exec ksh
: term; export PATH=~/bin:/bin:/usr/bin
: term; echo $PATH
/home/cross/bin:/bin:/usr/bin
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; export PATH="$HOME/bin:/bin:/usr/bin"
: term; echo $PATH
/home/cross/bin:/bin:/usr/bin
: term; which quux
/home/cross/bin/quux
: term; quux
xyzzy
: term; export PATH="~/bin:/bin:/usr/bin"
: term; echo $PATH
~/bin:/bin:/usr/bin
: term; which quux
: term; quux
ksh: quux: not found
: term; exec /usr/pkg/bin/bash
: term; echo $PATH
~/bin:/bin:/usr/bin
: term; which quux
: term; quux
xyzzy
: term;

Bash behaves strange here; 'which' doesn't find the executable but >nonetheless bash executes it, shows its output?

Bash doesn't behave strangely here at all. Outside of its POSIX
mode, it's free to implement whatever behavior it likes with
respect to `~` expansion, and this is the behavior the author
has chosen.

But, critically, that does not mean that the programs _that it
invokes_ have to do the same. So while `bash` can perform `~`
expansion on the components of $PATH when it's searching for a
program to execute, one shouldn't be surprised if other programs
don't do the same thing.

As was pointed out, `which` is not built into the shell, and
therefore, not obligated to follow `bash`'s expansion rules.

I've changed the above test frame to remove some unnecessary parts
to eliminate distracting complexity. The output for three shells:

Sorry, I think this cuts too much: the point was to show how the
different shells handle `~` in different contexts; the actual
strings that are in `$PATH` are important to show.

[snip]

I suppose the moral is, for maximal portability, prefer using
$HOME (or another similar variable) to `~` when setting $PATH
unless you're doing so in a context where you are sure that `~`
will be expanded during assignment.

IMO, the consequence is to not quote a tilde expression in the first
place if you want it expanded in a shell variable. *Violating* that
rule will _in Bash_ produce this strange effect (in the given setup).

To me it appears that not finding (by 'which') an executable but
executing it nonetheless qualifies as a bug [in Bash].

Not a bug, but yes, surprising.

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vmvp3d$2671i$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 23.01.2025 23:46, Keith Thompson wrote:

[snip]
For this and other reasons, though you *can* have a literal ~ in $PATH
in bash, it's best to avoid it and use $HOME instead.

Or use it correctly, unquoted and unescaped.

Or just don't use it, and then you don't have to worry about it.

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <4chc6l-lju.ln1@ID-313840.user.individual.net>,
Geoff Clare <netnews@gclare.org.uk> wrote:
...

As others have already pointed out, it's probably because you quoted the >tilde.

No, I didn't.

--
Most Southerners interest in, knowledge of, and participation in politics begins with
and ends with: Screw the blacks. If a guy is onboard with that, he's our guy!

Get them back in chains where they belong!

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 24.01.2025 14:33, Dan Cross wrote:

In article <vmvp3d$2671i$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 23.01.2025 23:46, Keith Thompson wrote:

[snip]
For this and other reasons, though you *can* have a literal ~ in $PATH
in bash, it's best to avoid it and use $HOME instead.

Or use it correctly, unquoted and unescaped.

Or just don't use it, and then you don't have to worry about it.

But as a Ksh (or any non-Bash shell) user I don't have
to worry about it. (Why shall I see any issue with it?)
(But as I very early already said; I anyway don't use
tilde-expressions in scripts, even as a Ksh user.)

But more importantly; shell programmers shall be well aware
of what quotes mean in shells! They are not just fancy things
or accessories that one may or may not use as one likes. They
have clear semantics and are essential in shell programming.

If you want tilde-expressions expanded _don't quote them_.
It's not much different from file-globbing; don't escape or
quote a '*' (or other globbing meta-characters) if you want
it to become expanded.

The suggestion to "not use" this ~ or that * is misguiding.
Know the shell concepts! - Or know your shell, at least,
with all its inconsistencies and/or (where applicable) bugs.

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 24.01.2025 14:46, Dan Cross wrote:

In article <vmu94j$1q2lp$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

Bash behaves strange here; 'which' doesn't find the executable but
nonetheless bash executes it, shows its output?

Bash doesn't behave strangely here at all. Outside of its POSIX
mode, it's free to implement whatever behavior it likes with
respect to `~` expansion, and this is the behavior the author
has chosen.

But, critically, that does not mean that the programs _that it
invokes_ have to do the same. So while `bash` can perform `~`
expansion on the components of $PATH when it's searching for a
program to execute, one shouldn't be surprised if other programs
don't do the same thing.

As I see it, Bash does the same correct assignment to PATH in
case the tilde-expression is quoted or unquoted; here there's
no difference. Quoted or escaped it's a literal tilde in PATH,
unquoted it's an expanded tilde-expression. (As in all those
shells; bash, ksh, zsh, dash, sh.)

As was pointed out, `which` is not built into the shell, and
therefore, not obligated to follow `bash`'s expansion rules.

And this (at least) seems to be the source of an inconsistency;
'which' is also in other shells not a built-in. But all other
shells I tested (ksh, zsh, dash, sh) handle it consistently;
if 'which' ("/usr/bin/which") detects no program [in PATH] it
should not execute some program. Other shells do that correctly.

I've changed the above test frame to remove some unnecessary parts
to eliminate distracting complexity. The output for three shells:

Sorry, I think this cuts too much: the point was to show how the
different shells handle `~` in different contexts; the actual
strings that are in `$PATH` are important to show.

The actual strings and their handling WRT PATH is actually the
same in all these shells! Those test-cases have hidden the real
problem, so I reduced it to show that it's not (not per se) the
"tilde in PATH" that is the problem; Bash (as the other shells)
correctly expands an unquoted tilde-expression, and correctly
leaves it in PATH literally if it's quoted or escaped.

The settings PATH=~/bin and PATH="~/bin" respectively shall
result in the same behavior across shells when searching for
programs; in the first case looking into "/home/someuser/bin/"
and in the second case looking into "./~/bin/" (i.e. a path
component with a local directory named "~").

It appears as if Bash invokes a tilde-expansion twice(!); once
[if unquoted] when the assignment happens, and another time
when the path-search is actually done. Both of our test-cases
indicated that Bash seems to tilde-expand a PATH variable value
a second time; so while 'which' (as other shells and programs)
will not find any executable in "./~/bin/" (which would be the
correct interpretation of a quoted "~/bin") Bash does.

That inconsistency - and deviating from all the other shells -,
if not a bug, looks like a Bad Design Idea if it's been done
deliberately. - YMMV.

Janis

[...]

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Kaz Kylheku <643-408-1753@kylheku.com> wrote:

On 2025-01-23, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

Bash behaves strange here; 'which' doesn't find the executable but
nonetheless bash executes it, shows its output?

which is a nonstandard command; the POSIX command is type.

In those systems where "which" exists at all, it is often
a locally brewed program that is not exactly the same like
the one in other systems.

In Debian and derivatives thereof, /usr/bin/which is a shell script.

I see that in MacOS, there is a /usr/bin/which whose --help
says to send mail to which-bugs<at>gnu.org; the man page
implicates a Carlo Wood as the culprit behind it.
It doesn't appear to be part of GNU Coreutils.

Which which is which? Burn the which!

Slackware has the binary which and the manpage attributes
Carlo Wood at gnu.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vn0cno$29vrs$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 24.01.2025 14:33, Dan Cross wrote:

In article <vmvp3d$2671i$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 23.01.2025 23:46, Keith Thompson wrote:

[snip]
For this and other reasons, though you *can* have a literal ~ in $PATH >>>> in bash, it's best to avoid it and use $HOME instead.

Or use it correctly, unquoted and unescaped.

Or just don't use it, and then you don't have to worry about it.

But as a Ksh (or any non-Bash shell) user I don't have
to worry about it. (Why shall I see any issue with it?)

Because you might want to put whatever you assign to `PATH`
in quotes, for instance if their are spaces in one of the
component pathnames (people run `bash` on windows and all
kinds of weird places) and the behavior differs. $HOME is
pleasantly boring by comparison.

[snip]
But more importantly; shell programmers shall be well aware
of what quotes mean in shells! They are not just fancy things
or accessories that one may or may not use as one likes. They
have clear semantics and are essential in shell programming.

If you want tilde-expressions expanded _don't quote them_.

What if the expression refers to a file name with a space in
it? Of course, one can escape the whitespace characters in
filenames, but that gets tedious.

It's not much different from file-globbing; don't escape or
quote a '*' (or other globbing meta-characters) if you want
it to become expanded.

The suggestion to "not use" this ~ or that * is misguiding.
Know the shell concepts! - Or know your shell, at least,
with all its inconsistencies and/or (where applicable) bugs.

See above.

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vn0bpf$29qe6$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 24.01.2025 14:46, Dan Cross wrote:

In article <vmu94j$1q2lp$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

Bash behaves strange here; 'which' doesn't find the executable but
nonetheless bash executes it, shows its output?

Bash doesn't behave strangely here at all. Outside of its POSIX
mode, it's free to implement whatever behavior it likes with
respect to `~` expansion, and this is the behavior the author
has chosen.

But, critically, that does not mean that the programs _that it
invokes_ have to do the same. So while `bash` can perform `~`
expansion on the components of $PATH when it's searching for a
program to execute, one shouldn't be surprised if other programs
don't do the same thing.

As I see it, Bash does the same correct assignment to PATH in
case the tilde-expression is quoted or unquoted; here there's
no difference.

Bash demonstrably does not assign the same thing to $PATH when
the `~` is quoted or escaped versus unquoted/unescaped:

```
: term; exec bash
: term; PATH="~/bin:/bin:/usr/bin"
: term; echo $PATH
~/bin:/bin:/usr/bin
: term; PATH=~/bin:/bin:/usr/bin
: term; echo $PATH
/home/cross/bin:/bin:/usr/bin
: term;
```

Note that in the first form, the literal `~` character is in
$PATH, while in the second it has been expanded, and the result
is a pathname relative to my home directory.

Quoted or escaped it's a literal tilde in PATH,
unquoted it's an expanded tilde-expression. (As in all those
shells; bash, ksh, zsh, dash, sh.)

Yes, is what was shown.

As was pointed out, `which` is not built into the shell, and
therefore, not obligated to follow `bash`'s expansion rules.

And this (at least) seems to be the source of an inconsistency;
'which' is also in other shells not a built-in. But all other
shells I tested (ksh, zsh, dash, sh) handle it consistently;

If it's not built into those shells, then those shells don't
"handle" `which` in any meaningful way, other than exec'ing it
as they would any other program.

But `which` was just an example of a program that's not built
into the shell that inspects $PATH, which it inherited from the
shell that invoked it.

The interesting point of the example was to show that, as it
goes to run a program, `bash` will interpret each component of
`$PATH` and perform `~` expansion, but since it doesn't modify
`$PATH` when it does so, the programs that it invokes (and that
inherit `$PATH` from it) may not find programs that `bash` does
if they use execlp/execvp/etc, or even just inspect $PATH
directly.

if 'which' ("/usr/bin/which") detects no program [in PATH] it
should not execute some program. Other shells do that correctly.

Why do you say this? As I wrote earlier, nothing prevents
`bash` from treating $PATH however it likes outside of its POSIX
mode, and its behavior is entirely correct according to the
rules by which it does things. If other shells do things
differently, then that's their behavior; bash need not be bound
by it.

I've changed the above test frame to remove some unnecessary parts
to eliminate distracting complexity. The output for three shells:

Sorry, I think this cuts too much: the point was to show how the
different shells handle `~` in different contexts; the actual
strings that are in `$PATH` are important to show.

The actual strings and their handling WRT PATH is actually the
same in all these shells!

No it's not; that was the point.

Those test-cases have hidden the real problem,

That's the thing; there is no "problem." There are some
differences between how shells behave, but they're different
programs, so why is that bad?

The settings PATH=~/bin and PATH="~/bin" respectively shall
result in the same behavior across shells when searching for
programs; in the first case looking into "/home/someuser/bin/"
and in the second case looking into "./~/bin/" (i.e. a path
component with a local directory named "~").

I don't see any reason why that _must_ be true. Any given shell
is free to interpret $PATH any way it choses, or even use a
variable named something other than `PATH` for the search path.
For example, the `rc` shell used on 10th Edition Research Unix
and Plan 9 doesn't used `$path` (lower-case) to find programs.

It appears as if Bash invokes a tilde-expansion twice(!); once
[if unquoted] when the assignment happens, and another time
when the path-search is actually done.

Yes. That's precisely what I was showing.

Both of our test-cases
indicated that Bash seems to tilde-expand a PATH variable value
a second time;

I fail to see where this is being done a "second time": it seems
more likly that `bash` doesn't make any special note of the `~`
in `PATH="~/bin:whatever"` until it actuall goes to run a
program.

so while 'which' (as other shells and programs)
will not find any executable in "./~/bin/" (which would be the
correct interpretation of a quoted "~/bin") Bash does.

You seem to have invented a definition of "correct" here and are
pursuing it aggressively, but that is just one definition from
a large set of such definitions; there's no reason to assume the
behavior you expect here is any more correct than what `bash`
does.

That inconsistency - and deviating from all the other shells -,

Careful: "all other shells" is a pretty big net, and it wouldn't
surprise me at all if this broke down if you expanded the set of
things you were looking at. If you really want to twist your
nogging, have a look at what `csh` does, for example.

if not a bug, looks like a Bad Design Idea if it's been done
deliberately. - YMMV.

It's obviously being done deliberately, and is easy to find in
the source code:
https://git.savannah.gnu.org/cgit/bash.git/tree/findcmd.c#n533

You may consider it bad design, and you're well within your
rights to do so, but opinions on that vary, and it doesn't mean
yours is correct.

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 24.01.2025 22:34, Dan Cross wrote:

In article <vn0bpf$29qe6$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 24.01.2025 14:46, Dan Cross wrote:

In article <vmu94j$1q2lp$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

Bash behaves strange here; 'which' doesn't find the executable but
nonetheless bash executes it, shows its output?

Bash doesn't behave strangely here at all. Outside of its POSIX
mode, it's free to implement whatever behavior it likes with
respect to `~` expansion, and this is the behavior the author
has chosen.

But, critically, that does not mean that the programs _that it
invokes_ have to do the same. So while `bash` can perform `~`
expansion on the components of $PATH when it's searching for a
program to execute, one shouldn't be surprised if other programs
don't do the same thing.

As I see it, Bash does the same correct assignment to PATH in
case the tilde-expression is quoted or unquoted; here there's
no difference.

Bash demonstrably does not assign the same thing to $PATH when
the `~` is quoted or escaped versus unquoted/unescaped: [...]

What I tried to express was that Bash does the same assignment
as the other shells; in both cases. In the quoted case it takes
it literally, and in the unquoted case it expands it. There's no
difference here. The assignments are all handled _consistently_
across all the shells including Bash.

As was pointed out, `which` is not built into the shell, and
therefore, not obligated to follow `bash`'s expansion rules.

And this (at least) seems to be the source of an inconsistency;
'which' is also in other shells not a built-in. But all other
shells I tested (ksh, zsh, dash, sh) handle it consistently;

If it's not built into those shells, then those shells don't
"handle" `which` in any meaningful way, other than exec'ing it
as they would any other program.

The point is; 'which' should consider PATH, and Bash should
consider PATH. All shells (but Bash) do that in a consistent
way.

Depending on how the assignment had been made, literal tilde
or expanded tilde expression, the PATH value should be taken
as it is, IMO, and not tilde-expanded a second time.

This is nothing but a "creative" implementation in Bash; one
that is unnecessary, one that differs from the other shells,
and one that creates (IMO) inconsistent behavior, behavior
that lead to fancy suggestions to _not use_ tilde in PATH.

But `which` was just an example of a program that's not built
into the shell that inspects $PATH, which it inherited from the
shell that invoked it.

The interesting point of the example was to show that, [...]

I acknowledge and had acknowledged your test sample before.
(I just rewrite it for me to streamline the calls, isolate
and better see the source of the issue)

if 'which' ("/usr/bin/which") detects no program [in PATH] it
should not execute some program. Other shells do that correctly.

Why do you say this? As I wrote earlier, nothing prevents
`bash` from treating $PATH however it likes outside of its POSIX
mode, and its behavior is entirely correct according to the
rules by which it does things. If other shells do things
differently, then that's their behavior; bash need not be bound
by it.

Of course one can implement ones shell as he likes. As said
above, I like shells with a consistent implementation; where
a negative match of 'which' (or 'type' or 'whence') [for any
given program] should also result in the shells to not find
and execute it.

If the goal of your statement is just to say that implementers
can do what they like, I can agree. (But that's a rather weak
statement, IMO, and not very useful when valuating shells and
any [IMO] inconsistencies.) And if you think that my used term
"bug" for that is inappropriate, I can live with that as well.

[...]

The actual strings and their handling WRT PATH is actually the
same in all these shells!

No it's not; that was the point.

Above we agreed that the _variable_ assignment and expansion
of PATH is the same (in all shells). The difference [in Bash]
appears only when Bash does a PATH search to find appropriate
commands. My point was that this _path search_ does a second
expansion.

Those test-cases have hidden the real problem,

That's the thing; there is no "problem." There are some
differences between how shells behave, but they're different
programs, so why is that bad?

Because it's inconsistent, it behavior is unexpected, it does
in a very subtle way surprise the users, and folks that know
this inconsistency advise to thus better not use '~' in the
first place; to me that's all clear factors why that is "bad".

The settings PATH=~/bin and PATH="~/bin" respectively shall
result in the same behavior across shells when searching for
programs; in the first case looking into "/home/someuser/bin/"
and in the second case looking into "./~/bin/" (i.e. a path
component with a local directory named "~").

I don't see any reason why that _must_ be true.

Well, we see that shells handle escaping/quoting of variables
in a _consistent_ way. Why should I believe that a shell value
with a tilde should be handled inconsistently?

If you set PATH=$HOME or PATH=~ all works smoothly. If you use
PATH='$HOME' or PATH='~' then these path components are taken
literally. But [in Bash] only in case of '$HOME'.

Any given shell is free to interpret $PATH any way it choses, [...]

(We had covered that already.)

It appears as if Bash invokes a tilde-expansion twice(!); once
[if unquoted] when the assignment happens, and another time
when the path-search is actually done.

Yes. That's precisely what I was showing.

Both of our test-cases
indicated that Bash seems to tilde-expand a PATH variable value
a second time;

I fail to see where this is being done a "second time": it seems
more likly that `bash` doesn't make any special note of the `~`
in `PATH="~/bin:whatever"` until it actuall goes to run a
program.

$ PATH='~/bin:/usr/bin:/bin' bash -c 'echo $PATH'
~/bin:/usr/bin:/bin

$ PATH='~/bin:/usr/bin:/bin' bash -c 'echo $PATH; quux'
~/bin:/usr/bin:/bin
xyzzy - ~/bin:/usr/bin:/bin

$ cat /home/jap/bin/quux
echo xyzzy - "$PATH"

so while 'which' (as other shells and programs)
will not find any executable in "./~/bin/" (which would be the
correct interpretation of a quoted "~/bin") Bash does.

You seem to have invented a definition of "correct" here and are
pursuing it aggressively, but that is just one definition from
a large set of such definitions; there's no reason to assume the
behavior you expect here is any more correct than what `bash`
does.

The call it "consistent" vs. "inconsistent". (Modulo what we
have already said above.)

That inconsistency - and deviating from all the other shells -,

Careful: "all other shells" is a pretty big net, and it wouldn't
surprise me at all if this broke down if you expanded the set of
things you were looking at. If you really want to twist your
nogging, have a look at what `csh` does, for example.

No, I will not compare the shells sensibly used for programming
with Csh.

I was aware that if I write "all other shells" that someone will
be pricking. I tried with the common shells (Ksh, Bash, Zsh, Sh,
Dash); I think this is a sensible set of shells to discuss things.

if not a bug, looks like a Bad Design Idea if it's been done
deliberately. - YMMV.

It's obviously being done deliberately, and is easy to find in
the source code: https://git.savannah.gnu.org/cgit/bash.git/tree/findcmd.c#n533

You may consider it bad design, and you're well within your
rights to do so, but opinions on that vary, and it doesn't mean
yours is correct.

So you think that behavior of Bash is good design here? - Okay,
noted.

Janis

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 24.01.2025 22:42, Keith Thompson wrote:

~ is equivalent to $HOME in most contexts. I suggest that assuming that
~, like $HOME, is expanded in double quotes is a very easy mistake to
make. And the bash misfeature we're discussing can make it hard to
detect that mistake.

Yes, to both.

(It certainly helps to know the standard shell quoting mechanisms.
That's so basic that it cannot be overestimated and should always
be emphasized. - I've experienced that even regular, experienced
shell users don't seem to know the exact mechanism in all cases.)

Anyone reading the docs? :-)

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 24.01.2025 23:00, Keith Thompson wrote:

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

On 24.01.2025 14:46, Dan Cross wrote:

[...]

/usr/bin/which is limited in what it can do. It follows POSIX-specified behavior for $PATH; it doesn't recognize any shell-specific rules. [...]

Sure.

[...]

The settings PATH=~/bin and PATH="~/bin" respectively shall
result in the same behavior across shells when searching for
programs; in the first case looking into "/home/someuser/bin/"
and in the second case looking into "./~/bin/" (i.e. a path
component with a local directory named "~").

What do you mean by "shall result?

I mean that a shell should behave consistently. (I think Bash does
not in the given case.)

All shells that conform to POSIX behave as you describe. bash doesn't conform to POSIX unless you ask it to. Neither do csh, tcsh, and fish.

We were speaking about shell programming, so [seriously] I don't
consider Csh and Tcsh as sensible sample shells for the discussion.
(Thinking about it, I wonder whether Bash inherited tilde-handling
from Csh, maybe; that would at least explain something.)

(I don't know Fish, so I cannot comment on that.)

[...]

BTW, it hadn't occurred to me that you can have a relative path in a component of $PATH, but it does seem to work. I won't be taking
advantage of this information.

Yes. Some prefer to add '.' to PATH. (Though I have no intention
to discuss that habit.)

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 24.01.2025 22:38, Dan Cross wrote:

In article <vn0cno$29vrs$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 24.01.2025 14:33, Dan Cross wrote:

In article <vmvp3d$2671i$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

On 23.01.2025 23:46, Keith Thompson wrote:

[snip]
For this and other reasons, though you *can* have a literal ~ in $PATH >>>>> in bash, it's best to avoid it and use $HOME instead.

Or use it correctly, unquoted and unescaped.

Or just don't use it, and then you don't have to worry about it.

But as a Ksh (or any non-Bash shell) user I don't have
to worry about it. (Why shall I see any issue with it?)

Because you might want to put whatever you assign to `PATH`
in quotes, for instance if their are spaces in one of the
component pathnames (people run `bash` on windows and all
kinds of weird places) and the behavior differs. $HOME is
pleasantly boring by comparison.

Sure. That's why you can write, say,

PROJECT_PATH="Yet another space odyssey"
PATH=~/bin:"My Windows Bin":$PROJECT_PATH:'/tmp/$LITERAL':.

(The standard way. Clean unambiguous quoting. No issues.)

[snip]
But more importantly; shell programmers shall be well aware
of what quotes mean in shells! They are not just fancy things
or accessories that one may or may not use as one likes. They
have clear semantics and are essential in shell programming.

If you want tilde-expressions expanded _don't quote them_.

What if the expression refers to a file name with a space in
it? Of course, one can escape the whitespace characters in
filenames, but that gets tedious.

The (standard-)rules are simple and clean; see my sample above.

To say it explicitly; values in variables are *not* subject to
another word-splitting in assignments [in standard shells].

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vn2hsj$2pe96$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

[snip]

Most of this discussion seems to be talking at cross-purposes.
I don't see much point in responding to the specifics.

My point was simply to show that shell behavior varies with
respect to how $PATH is treated; caveat emptor.

I don't usually use bash, but if you do and you really don't
like this behavior you can turn it off.

You may consider it bad design, and you're well within your
rights to do so, but opinions on that vary, and it doesn't mean
yours is correct.

So you think that behavior of Bash is good design here? - Okay,
noted.

Please don't put words in my mouth. I don't really have much of
an opinion on whether what bash does here is good or not: it is
neither wrong nor right, it simply is what it is.

I do recognize that, for better or worse, the universe of useful
or even just interesting software is far larger than the small
sample set you seem to want to constrain it to. In that very
large space, there have been, and will continue to be, many
different design points and implementations with respect to how
these sorts of things are handled; indeed, just a couple of
years ago shells were an active area of research: https://nikos.vasilak.is/p/pash:hotos:2021.pdf

I have found that generally one would be wise not to make too
many assumptions about what is "correct" versus what one is just
most familiar/comfortable with.

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-26, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

On 24.01.2025 23:00, Keith Thompson wrote:

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:

On 24.01.2025 14:46, Dan Cross wrote:

[...]

/usr/bin/which is limited in what it can do. It follows POSIX-specified >>> behavior for $PATH; it doesn't recognize any shell-specific rules. [...]

Sure.

[...]

The settings PATH=~/bin and PATH="~/bin" respectively shall
result in the same behavior across shells when searching for
programs; in the first case looking into "/home/someuser/bin/"
and in the second case looking into "./~/bin/" (i.e. a path
component with a local directory named "~").

What do you mean by "shall result?

I mean that a shell should behave consistently. (I think Bash does
not in the given case.)

Consistently with what? Bash consistently expands literal '~'s in
$PATH, and consistently disables that expansion in POSIX mode.

All shells have shell-specific features. What's odd about this case is
that bash has a POSIX-violating feature that affects command name
resolution.

It's a feature that (if used) leaks tildes into child processes via
the environment variable. Path resultion in child processes, if it
reaches a PATH element with a tilde, will somehow process that tilde.

I just tried this experiment. I made a directory named ~ and put ~:
as the leading element of PATH. I put a program called "foo" that
directory.

Surely enough, I can run "foo" from the parent directory above.

The exec functions treat ~ as an ordinary path component.

(I cannot do that out of Bash, which processes the tilde, but
the 'p' family of the exec functions will find it!)

This is a problem similar to "." being in PATH.

If someone has, say, "~/bin" in their PATH, ahead of /bin and /usr/bin,
I can put a malicious program in some directory called "~/bin"
somewhere in the filesystem, give that program the name of a common
external utility, and trick the user into changing into that location
where they will run this common command, resolving to my malicious
program.

If we regard this as a security hole, that atually raises the priority
and bolsters the argument that it ought to be removed even if it
breaks some users, perhaps through a process of noisy deprecation.

Furhermore, the case can be made that the exec stuff in the Linux kernel
or C libraries should be patched with a check against components with a
leading tilde.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-26, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

you don't want to miss. - So Bash users, if they don't want to get
bitten in a subtle way, are probably advised to be pointed out to
that behavior, or, as suggested by others already, not to use tilde
at all with PATH in Bash.

That's a bizarre recommendation. Approximately all uses of tilde
in PATH are variants of PATH=~/bin:$PATH, which are expanded on
assignment in bash as in other sh-type shells.

--
Christian "naddy" Weisgerber naddy@mips.inka.de

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 26 Jan 2025 14:49:16 +0100, Janis Papanagnou wrote:

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.

But you can use “∕” in a file/directory name.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vn6ifi$8je0$3@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

On Sun, 26 Jan 2025 14:49:16 +0100, Janis Papanagnou wrote:

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.

But you can use in a file/directory name.

That's a relief.

--
Kenny, I'll ask you to stop using quotes of mine as taglines.

- Rick C Hodgin -

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 27 Jan 2025 15:48:21 +1100, Alexis wrote:

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

But you can use “∕” in a file/directory name.

Not in a POSIX-conforming way:

ldo@theon:trydir> mkdir f1
ldo@theon:trydir> touch f1/f2
ldo@theon:trydir> touch f1∕f2
ldo@theon:trydir> ls -lR
.:
total 4
drwxr-xr-x 2 ldo users 4096 Jan 27 19:04 f1
-rw-r--r-- 1 ldo users 0 Jan 27 19:04 f1∕f2

./f1:
total 0
-rw-r--r-- 1 ldo users 0 Jan 27 19:04 f2

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <87msfc7tgq.fsf@gmail.com>, Alexis <flexibeast@gmail.com> wrote: >Lawrence D'Oliveiro <ldo@nz.invalid> writes:

On Sun, 26 Jan 2025 14:49:16 +0100, Janis Papanagnou wrote:

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.

But you can use <line noise> in a file/directory name.

Not in a POSIX-conforming way:

Do you know what LDO meant by the above-quoted line noise?

--
I love the poorly educated.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <87ikq2bbf4.fsf@nosuchdomain.example.com>,
Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote: >cross@spitfire.i.gajendra.net (Dan Cross) writes:

In article <vn2hsj$2pe96$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

[snip]

Most of this discussion seems to be talking at cross-purposes.
I don't see much point in responding to the specifics.

My point was simply to show that shell behavior varies with
respect to how $PATH is treated; caveat emptor.

I don't usually use bash, but if you do and you really don't
like this behavior you can turn it off.

As far as I can tell, you can only turn off the behavior by
running in POSIX mode, which disables a lot of other Bash-specific >functionality. Personally, I'm unwilling to do that. I think I'd
like to see a "set -o" setting that disables just this feature.

That would be better, for sure.

You can *avoid* it by being careful not to put literal '~' characters
in $PATH (specifically at the beginning of any element of $PATH).
That's what I do (unintentionally before now, deliberately now that
I know about it.)

Indeed! I find it easiest just to not use `~` in $PATH. :-)

- Dan C.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 26.01.2025 20:01, Christian Weisgerber wrote:

On 2025-01-26, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

you don't want to miss. - So Bash users, if they don't want to get
bitten in a subtle way, are probably advised to be pointed out to
that behavior, or, as suggested by others already, not to use tilde
at all with PATH in Bash.

That's a bizarre recommendation. Approximately all uses of tilde
in PATH are variants of PATH=~/bin:$PATH, which are expanded on
assignment in bash as in other sh-type shells.

Yes, to both. (Only that "recommendation" should have been put in
quotes; as it should have got obvious what I think about that.)

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 26.01.2025 18:51, Kaz Kylheku wrote:

On 2025-01-26, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.
So there would, IMO, not be a security hole (i.e. not because of that).

The / is not part of a name; it is acting as the path component
separator. [...]

Oh, I took your wording (emphasis by me)

*directory* called "~/bin"

literally as being meant the name of the directory (as opposed to
the complete path to the bin directory). - Thanks for clarifying.

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 27.01.2025 01:02, Lawrence D'Oliveiro wrote:

On Sun, 26 Jan 2025 14:49:16 +0100, Janis Papanagnou wrote:

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.

But you can use “∕” in a file/directory name.

You can use it in file and directory _paths_ as separator.

But how would you create (or subsequently access) any file
where '/' is part of its actual _name_?

Janis

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-27, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

On Sun, 26 Jan 2025 14:49:16 +0100, Janis Papanagnou wrote:

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.

But you can use “∕” in a file/directory name.

Yes, you can use UTF-8 encoded characters that look like ASCII in
a directory name, such as U+201C (left double quote), U+2215 (division
slash) and U+201D (right double quote).

Try not to interrupt when grownups are talking, though.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-01-27, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

On Mon, 27 Jan 2025 15:48:21 +1100, Alexis wrote:

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

But you can use “∕” in a file/directory name.

Not in a POSIX-conforming way:

ldo@theon:trydir> mkdir f1
ldo@theon:trydir> touch f1/f2
ldo@theon:trydir> touch f1∕f2

[...]

Yes, yes, we all know what you're saying, and we all hope you've
enjoyed the attention.

He's making some irrelevant point about being able to use the
U+2215 "DIVISION SLASH" character in a path name, UTF-8 encoded.

Sure, Unicode character confusion can have security implications.

For instance, I'm guessing that if you call GetCommandLineA
on Windows on an input string containing U+2215, it will likely
map it to the ASCII slash. Which means that there is now a path
separator which wasn't there.

If a user is operating some program which calls another program, passing arguments to it, and that calling program validates against the use of
certain paths for security reasons, that user may be able to use this to
escape from the sandbox. For instance something like "../bar"
can be written with U+2215 slashes, so that it validates as a simple
file name; but upon mapping to ASCII, it becomes two components, one
of which escapes upward out of the working directory.

I don't suspect this problem intersects with the issue we are talking
about, but it's hard to be sure about a negative without doing a bunch
of work.

The ability to use a Unicode slash in a filename on POSIX systems,
thanks to UTF-8, is mostly a good thing, in my view.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 26 Jan 2025 23:51:33 -0800, Keith Thompson wrote:

Of course the '/' character can appear in a pathname. And of course it cannot appear in a pathname component, which POSIX also calls a
"filename".

But you can use “∕” in a file/directory name.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 27 Jan 2025 17:11:27 +0100, Janis Papanagnou wrote:

On 27.01.2025 01:02, Lawrence D'Oliveiro wrote:

On Sun, 26 Jan 2025 14:49:16 +0100, Janis Papanagnou wrote:

To my best knowledge using '/' as part of a file or directory name is
(as the '\0') prohibited by the operating system at a very low level.

But you can use “∕” in a file/directory name.

You can use it in file and directory _paths_ as separator.

No, the separator is the “/” character.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

In article <vn93nt$19o0q$7@dont-email.me>,
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

On Sun, 26 Jan 2025 23:51:33 -0800, Keith Thompson wrote:

Of course the '/' character can appear in a pathname. And of course it
cannot appear in a pathname component, which POSIX also calls a
"filename".

But you can use in a file/directory name.

But you can use Γ^@^\Γ^H^UΓ^@^] (line noise) in a file/directory name.


--
Politics is show business for ugly people.

Sports is politics for stupid people.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-02-03, Keith Thompson <Keith.S.Thompson+u@gmail.com> wrote:

Another interesting tidbit: the GNU `which` command has a "--skip-tilde" option that tells it to skip elements of $PATH that start with '~'
(and also directories anywhere under $HOME).

From the output of `which --help`:

--skip-tilde Skip directories in PATH that start with a tilde.

Le's get this straight. The programmer who wrote this (we need not
repeat his name again, since it came up here about twice recently)
noticed that he had unexpanded tildes leaking into his PATH environment variable, visible to child processes of his shell. Yet, instead of
realizing how wong that is and possibly reporting a bug against Bash,
he just ... hacked a feature into his pet utility to ignore them.

Everything to do with this utility is just whichcraft as far as I'm
concerned. (Okay, that is worn threadbare now, I will stop).

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)