Sysop: | Amessyroom |
---|---|
Location: | Fayetteville, NC |
Users: | 40 |
Nodes: | 6 (1 / 5) |
Uptime: | 21:28:08 |
Calls: | 291 |
Files: | 910 |
Messages: | 76,666 |
Posted today: | 1 |
(*) Summary[snip]
I wrote a sed script that makes a line replacement after it finds the
right spot. So far so good. Then I added quit command after the
change, but the quit does not seem to take effect---violating my
expectation. I'll appreciate any help on understanding what's going on.
So far so good. I decided to try it on longer files and I wanted to see[snip]
the change more quickly (without long files scrolling past my terminal),
so I decided to add a /q/ command right after the c commmand. I
thought---it will make sed quit right after making the change, so I can
see it works as desired and then I remove the /q/ and release it to production. But that did not happen.
I failed the exercise I gave myself. Can you help me to understand why
the q command isn't stopping sed as I thought it would? I'd like to get
a better intuition.
I failed the exercise I gave myself. Can you help me to understand why
the q command isn't stopping sed as I thought it would? I'd like to get
a better intuition.
Salvador Mirzo wrote:
I failed the exercise I gave myself. Can you help me to understand why
the q command isn't stopping sed as I thought it would? I'd like to get
a better intuition.
The specification in https://pubs.opengroup.org/onlinepubs/9799919799/
says at the end of the c command "Start the next cycle."
So any commands following it won't be executed.
The rationale does not mention the reason for the behavior specified.
Here, replacing the single-line c command with s/.*/$tag/ would do it.
If you needed a range of lines to be changed, that might become tricky.
Ralf Damaschke <rwspam@gmx.de> writes:
The specification in https://pubs.opengroup.org/onlinepubs/9799919799/
That's the home page. I believe you meant to link the sed page
directly. When copying URLs from the specification, we need to copy the framed URL, otherwise we always end up at the home page. The framed sed
page is at
https://pubs.opengroup.org/onlinepubs/9799919799/utilities/sed.html
I tried to change a line and then append another. In both
FreeBSD's and GNU's sed, I the append takes place (and I thought they
would not if they were to obey the specification).
$ sed '/^x/{c\
hello a\
hi }' x.txt a line hello hi more lines more lines $
$ sed '/^x/{c\
hello
a\
hi
}' x.txt
$ sed '/^x/{c\
hello a\
hi }' x.txt a line hello hi more lines more lines $
$ sed '/^x/{c\
hello
a\
hi
}' x.txt
(*) Summary
I wrote a sed script that makes a line replacement after it finds the
right spot. So far so good. Then I added quit command after the
change, but the quit does not seem to take effect---violating my
expectation. I'll appreciate any help on understanding what's going on.
(*) A detailed description
I wrote this program:
--8<-------------------------------------------------------->8---
%cat make-release
#!/bin/sh
usage()
{
printf '%s tag file\n' $0
exit 1
}
test $# '<' 2 && usage
tag="$1"
shift
sed "/<<Release>>=/ {
n;
c\\
$tag
}" $*
--8<-------------------------------------------------------->8---
Here's how I use it. My objective with it is to replace that
/something/ in the text file with a new argument.
--8<-------------------------------------------------------->8---
%cat sample.txt
Lorem ipsum dolor...
<<Release>>=
something
@
... sit a met [...]
%
--8<-------------------------------------------------------->8---
Here's how I invoke it:
--8<-------------------------------------------------------->8---
%sh make-release release1 sample.txt
Lorem ipsum dolor...
<<Release>>=
release1
@
... sit a met [...] --8<-------------------------------------------------------->8---
So far so good. I decided to try it on longer files and I wanted to see
the change more quickly (without long files scrolling past my terminal),
so I decided to add a /q/ command right after the c commmand. I
thought---it will make sed quit right after making the change, so I can
see it works as desired and then I remove the /q/ and release it to production. But that did not happen.
--8<-------------------------------------------------------->8---
%cat make-release
#!/bin/sh
usage()
{
printf '%s tag file\n' $0
exit 1
}
test $# '<' 2 && usage
tag="$1"
shift
sed "/<<Release>>=/ {
n;
c\\
$tag
q}" $*
--8<-------------------------------------------------------->8---
I still see the whole file:
--8<-------------------------------------------------------->8---
%sh make-release release1 sample.txt
Lorem ipsum dolor...
<<Release>>=
release1
@
... sit a met [...]
%
--8<-------------------------------------------------------->8---
I failed the exercise I gave myself. Can you help me to understand why
the q command isn't stopping sed as I thought it would? I'd like to get
a better intuition.
I've been reading Dale Dougherty and Arnold Robin's ``sed & awk'' book.
If you have any recommended sed-related bibliography, I'd appreciate it,
too.
Isn't this about the time where we give the caution about "Don't use sed
for anything beyond the s/foo/bar/g stage" ?
Kenny McCormack schrieb:
Isn't this about the time where we give the caution about "Don't use sed
for anything beyond the s/foo/bar/g stage" ?
Simply because you cannot imagine how to use it? No!
Seriously, the above looks like gobbledegook compared to the equivalent in AWK (or some other normal scripting langugae). "sed" looks like Intercal (once you get beyond s/foo/bar/g).
I'm sure that whatever it is that OP is trying to do, it could be easily translated to (say) AWK and look much nicer.
On 18.12.2024 20:46, Salvador Mirzo wrote:
(*) Summary
I wrote a sed script that makes a line replacement after it finds the
right spot. So far so good. Then I added quit command after the
change, but the quit does not seem to take effect---violating my
expectation. I'll appreciate any help on understanding what's going on.
First (before I forget it) change your string comparison '<' to the
numerical comparison operator '-lt' as in: test $# -lt 2 && usage Otherwise, if you get used to using the wrong operator, you may get
subtle errors in future if you continue that habit.
Also note that using $* may not work correctly (e.g. depending on
filenames [containing spaces] used). The safe form is a quoted "$@"
(Then I was tempted to make a similar comment as Kenny. But...)
WRT your question I'd be interested to understand more about the
intention of your original question...
I mean if you don't trust your 'sed' command just pipe it though
'less'; there's no need to change the 'sed' program just for that.
Personally I'd try whether it works (by adding "something" before
and also after the desired place in your sample.txt to be sure the
other occurrences were not changed), and then just call
sed -e '/<<Release>>=/,+1s/something/sth else/' sample.txt
to see it working.
On 2024-12-21, Ralf Damaschke <rwspam@gmx.de> wrote:
[...][...]
People looking for solutions in their production workflow do not want
"hey, look, this can be done in Sed!" type stuff. While people /can/
ramp up on it and easily become proficient, the only ones ever to do so are going to be engineers looking to waste time on puzzles.
[...]
I'm studying and I often go back to the past to see what life was I
like. I initially tried to solve the problem with /ed/, but did not
find a way to insert a string coming from the a shell script's cmdline.
Then I thought that /sed/ was there to make /ed/ more scriptable.
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
On 18.12.2024 20:46, Salvador Mirzo wrote:
(*) Summary
I wrote a sed script that makes a line replacement after it finds the
right spot. So far so good. Then I added quit command after the
change, but the quit does not seem to take effect---violating my
expectation. I'll appreciate any help on understanding what's going on.
First (before I forget it) change your string comparison '<' to the
numerical comparison operator '-lt' as in: test $# -lt 2 && usage
Otherwise, if you get used to using the wrong operator, you may get
subtle errors in future if you continue that habit.
Changed. Why is it the wrong operator? It seems it's not the standard one---checking just now on the POSIX.1 spec. I think I just tried it
out and given it worked as expected, I didn't think of checking it. I'm
new to the whole thing.
Also note that using $* may not work correctly (e.g. depending on
filenames [containing spaces] used). The safe form is a quoted "$@"
I learned something here.
[snip code]
(Then I was tempted to make a similar comment as Kenny. But...)
I'm studying and I often go back to the past to see what life was I
like. I initially tried to solve the problem with /ed/, but did not
find a way to insert a string coming from the a shell script's cmdline.
Then I thought that /sed/ was there to make /ed/ more scriptable.
WRT your question I'd be interested to understand more about the
intention of your original question...
The intention is mostly in the paragraph above, but the way I study is
to put things in real-world practice as much as possible. (When I
realize the solution is indeed too old to make sense, I replace it.
Otherwise I stick with it.) I have a literate programming file that
contains a chunk that's the version of the program I'm writing. So when
you ask the program its version, the information is included in the executable, an idea which I like. I get the version with a git command
such as
$ git log --oneline | head -1 | awk '{print $1}'
2566d31
So I wanted to include such string in the literate programming file. At first I wrote a solution in the programming language I'm using, but I remember seeing many older software using sed for something like that,
so I decided to study sed a bit. I read the sed part of Dale Dougherty
and Arnold Robbins's book "sed & awk", second edition, and I thought sed
was quite neat and sensible. What I'm noticing now is that there are
too many different sed behavior out there to make it sensible to use.
UNIX systems as a whole are like that, so I'm used to reminding myself
of using the common subset of everything. Perhaps the common subset of
sed is too small. If I have to use it just for search and replace, then perhaps it's not really worth it.
I mean if you don't trust your 'sed' command just pipe it though
'less'; there's no need to change the 'sed' program just for that.
Personally I'd try whether it works (by adding "something" before
and also after the desired place in your sample.txt to be sure the
other occurrences were not changed), and then just call
sed -e '/<<Release>>=/,+1s/something/sth else/' sample.txt
to see it working.
Thanks for the excellent instruction!
On 21/12/2024 12:17, Salvador Mirzo wrote:
I'm studying and I often go back to the past to see what life was I
like. I initially tried to solve the problem with /ed/, but did not
find a way to insert a string coming from the a shell script's cmdline.
Then I thought that /sed/ was there to make /ed/ more scriptable.
I think the other contributors are somewhat harsh on Sed. For
those who started on V6 Unix, there was just Ed, and, as you thought,
Sed was added in V7 as a scripting improvement to Ed.
Awk also came
in with V7. Some people adopted Awk with enthusiasm, but the early
versions were quite limited/buggy, partly thanks to the limitations of
the PDP-11; Sed was pretty reliable even in those days.
So at least some users tried and failed with Awk,
but found Sed usable with very
little to learn, thanks to the relationship with Ed. The arcana of
Sed are much easier to understand if you are/were a regular user of Ed, whereas Awk requires you to learn a whole new language [I'm not in any
way suggesting that that is unusually difficult].
So students will normally no longer learn or use Ed/Sed, apart
perhaps from "s/foo/bar/" and similar; it makes more sense to learn a
visual editor and Awk. But for older hands, and for those interested
in the history, there is still a use for Sed. Personally, I know my
way around Sed much better than Awk. So, again personally, Sed is my
stream editor of choice for tasks somewhat harder than "s ...", but not
so hard that I need to do some serious programming and checking of the documentation. [YMMV, and I'm certainly not trying to persuade any Awk
users that they should learn to use Sed instead.]
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
On 18.12.2024 20:46, Salvador Mirzo wrote:
First (before I forget it) change your string comparison '<' to the
numerical comparison operator '-lt' as in: test $# -lt 2 && usage
Otherwise, if you get used to using the wrong operator, you may get
subtle errors in future if you continue that habit.
Changed. Why is it the wrong operator?
It seems it's not the standard one---checking just now on the
POSIX.1 spec.
Also note that using $* may not work correctly (e.g. depending on
filenames [containing spaces] used). The safe form is a quoted "$@"
I learned something here.
--8<-------------------------------------------------------->8---
$ cat script.sh
#!/bin/sh
echo dollar-star $*
./script dollar-star $*
echo quoted-dollar-star "$*"
./script "$*"
echo dollar-at $@
./script $@
echo quoted-dollar-at "$@"
./script "$@"
$ cat script.c
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("\nfull cmdline: ");
for (int i = 0; i < argc; ++i) {
printf("%s ", argv[i]);
}
printf("\nargs: %d\n", argc);
for (int i = 0; i < argc; ++i) {
printf("arg %d: %s\n", i, argv[i]);
}
return 0;
}
--8<-------------------------------------------------------->8---
$ ./script.sh 1 2 "th ree"
dollar-star 1 2 th ree
full cmdline: ./script dollar-star 1 2 th ree
args: 6
arg 0: ./script
arg 1: dollar-star
arg 2: 1
arg 3: 2
arg 4: th
arg 5: ree
quoted-dollar-star 1 2 th ree
full cmdline: ./script 1 2 th ree
args: 2
arg 0: ./script
arg 1: 1 2 th ree
dollar-at 1 2 th ree
full cmdline: ./script 1 2 th ree
args: 5
arg 0: ./script
arg 1: 1
arg 2: 2
arg 3: th
arg 4: ree
quoted-dollar-at 1 2 th ree
full cmdline: ./script 1 2 th ree
args: 4
arg 0: ./script
arg 1: 1
arg 2: 2
arg 3: th ree
$
On 21/12/2024 12:17, Salvador Mirzo wrote:
I'm studying and I often go back to the past to see what life was I
like. I initially tried to solve the problem with /ed/, but did not
find a way to insert a string coming from the a shell script's cmdline.
Then I thought that /sed/ was there to make /ed/ more scriptable.
I think the other contributors are somewhat harsh on Sed. For
those who started on V6 Unix, there was just Ed, and, as you thought,
Sed was added in V7 as a scripting improvement to Ed. Awk also came
in with V7. Some people adopted Awk with enthusiasm, but the early
versions were quite limited/buggy, partly thanks to the limitations of
the PDP-11; Sed was pretty reliable even in those days. So at least
some users tried and failed with Awk, but found Sed usable with very
little to learn, thanks to the relationship with Ed. The arcana of
Sed are much easier to understand if you are/were a regular user of Ed, whereas Awk requires you to learn a whole new language [I'm not in any
way suggesting that that is unusually difficult].
So students will normally no longer learn or use Ed/Sed, apart
perhaps from "s/foo/bar/" and similar; it makes more sense to learn a
visual editor and Awk. But for older hands, and for those interested
in the history, there is still a use for Sed.
Personally, I know my way around Sed much better than Awk. So, again personally, Sed is my stream editor of choice for tasks somewhat
harder than "s ...", but not so hard that I need to do some serious programming and checking of the documentation. [YMMV, and I'm
certainly not trying to persuade any Awk users that they should learn
to use Sed instead.]
[...] I'm here on the USENET because I
discovered it in non-standard media. It turns out I find NNTP a much
better medium of discourse than any other. (I'm writing this from a GNU EMACS buffer and will be send out using Gnus. Gnus can be dramatically uninuitive, but there seems to be no real replacement for it when it
comes to USENET and perhaps mail.) [...]
I actually know awk a little bit. I've read the ``AWK Programming
Language'' by Aho, Kernighan and Weinberger, 1988. I loved it. [...]
for anything else you should just use awk ...
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
On 21.12.2024 13:17, Salvador Mirzo wrote:[...]
As previously mentioned, 'sed' might not be the best choice for[...]
developing such scripts; you might want to consider to learn 'awk'.
$ git log --oneline | head -1 | awk '{print $1}'
2566d31
With Awk you don't need 'head', it can be done like this
$ git log --oneline | awk 'NR==1 {print $1}'
(For long input files you may want an early exit
...| awk 'NR==1 { print $1 ; exit(0) }'
but that just as an aside.)
This raises another issue: it's often possible to replace a command in a pipeline that filters output with an option to the command that does the
same thing. There's no general rule for how to do this, since different commands do things differently, but for the example above:
git log --oneline -n 1 | awk '{print $1}'
or even:
git log -n 1 --format=%h
I haven't memorized the "--format" option, so I don't generally us it in ad-hoc one-liners, but I do use it in scripts. Note both of the above commands avoid generating the entire list of log entries, which could
save significant time on a large repo.
Using unnecessary commands in pipelines is Mostly Harmless, but IMHO
it's good to think about how to do things more efficiently. See also "Useless use of cat" (UUOC).
On Sat, 21 Dec 2024 08:13:52 -0600, Ed Morton wrote:
for anything else you should just use awk ...
If you want to suggest Awk, you might as well use Perl.
That does
everything Awk does, just as concisely, and plenty more besides.
On 2024-12-21, Ralf Damaschke <rwspam@gmx.de> wrote:
Simply because you cannot imagine how to use it? No!
He explained the "because", in the immediately following text that you snipped:
Seriously, the above looks like gobbledegook compared to theequivalent in
AWK (or some other normal scripting langugae). "sed" looks likeIntercal
(once you get beyond s/foo/bar/g).
I'm sure that whatever it is that OP is trying to do, it could beeasily
translated to (say) AWK and look much nicer.
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
[...]
Of course, if you can _reduce_ the amount of data at an early stage
(like in your 'git -n 1' sample) the earlier the better! (My 'git',
BTW, doesn't seem to support an option '-n'; which might be another
reason to let a standard tool like Awk do the task for which it has
been defined, text-processing.)
What version of git are you using?
Note that "-n 1" is an argument to "git log", not to "git".
[...]
On Sun, 22 Dec 2024 00:50:45 +0100, Janis Papanagnou wrote:
In practice it goes even farther; with Awk typical pipeline command
sequences that use utilities like cat, head, tail, grep, cut, sed, tr,
wc, seq, tee, etc. can typically all be represented and combined by Awk.
Another counterexample to the hoary old “do one thing and do it well” trope ...
On Sat, 21 Dec 2024 08:13:52 -0600, Ed Morton wrote:
for anything else you should just use awk ...
If you want to suggest Awk, you might as well use Perl. That does
everything Awk does, just as concisely, and plenty more besides.
In practice it goes even farther; with Awk typical pipeline command
sequences that use utilities like cat, head, tail, grep, cut, sed, tr,
wc, seq, tee, etc. can typically all be represented and combined by Awk.
The advantage of Awk is that it's standard on Unix systems ...
On Sun, 22 Dec 2024 01:02:03 +0100, Janis Papanagnou wrote:
The advantage of Awk is that it's standard on Unix systems ...
On my Debian system:
ldo@theon:~> apt-cache rdepends gawk | wc -l
89
ldo@theon:~> apt-cache rdepends perl | wc -l
1121
Let’s just say, you’re more likely to find Perl than Awk on a Linux system ...
I'd be surprised to
see *any* Linux distribution (other than small embedded systems) that
doesn't include awk (either gawk or mawk, or even busybox awk) by
default.
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 16:36:32 -0800, Keith Thompson wrote:
I'd be surprised to
see *any* Linux distribution (other than small embedded systems) that
doesn't include awk (either gawk or mawk, or even busybox awk) by
default.
The point being, it would likely be easier to remove Awk than to remove
Perl.
Which is quite different from what you wrote, which was that "you’re
more likely to find Perl than Awk on a Linux system".
On Fri, 20 Dec 2024 15:55:12 +0100, Janis Papanagnou wrote:
you may get subtle errors in future if you continue that habit.
For us newbies, do you have an example?
On Sat, 21 Dec 2024 08:13:52 -0600, Ed Morton wrote:
for anything else you should just use awk ...
If you want to suggest Awk, you might as well use Perl. That does
everything Awk does, just as concisely, and plenty more besides.
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 21:09:25 -0800, Keith Thompson wrote:
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 16:36:32 -0800, Keith Thompson wrote:
I can no longer keep up with the rapid movement of the goalposts. I
see no point in continuing this discussion.
with <87msgourzu.fsf@nosuchdomain.example.com> Keith Thompson wrote:
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 21:09:25 -0800, Keith Thompson wrote:
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 16:36:32 -0800, Keith Thompson wrote:
*SKIP* [ 8 lines 5 levels deep] # there isn't anything to see
I can no longer keep up with the rapid movement of the goalposts. I
see no point in continuing this discussion.
Point of this (and many others) discussion is to generate engagement.
Turns out, talking about lizard people doesn't ignite Usenet. Who knew.
In article <slrnvmidnd.9vr.apple.universe@freight.zombinet>,
Eric Pozharski <apple.universe@posteo.net> wrote:
with <87msgourzu.fsf@nosuchdomain.example.com> Keith Thompson wrote:
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 21:09:25 -0800, Keith Thompson wrote:
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 16:36:32 -0800, Keith Thompson wrote:
*SKIP* [ 8 lines 5 levels deep] # there isn't anything to see
I can no longer keep up with the rapid movement of the goalposts. I
see no point in continuing this discussion.
Point of this (and many others) discussion is to generate engagement.
Turns out, talking about lizard people doesn't ignite Usenet. Who knew.
Good one. You really got LDO's number to a T.
Awk is a mandatory POSIX tool (and so available on all POSIX-compliant
Unix-y systems) with a tiny but powerful language focused on just text processing, perl is none of that.
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Mon, 23 Dec 2024 07:26:15 -0600, Ed Morton wrote:
Awk is a mandatory POSIX tool (and so available on all POSIX-compliant
Unix-y systems) with a tiny but powerful language focused on just text
processing, perl is none of that.
Perl is all of that, and more. Text processing is very much the raison
d’être for Perl. Because it turns out it can get quite complicated.
It's not "all of that". Perl is not a mandatory POSIX tool.
In article <slrnvmidnd.9vr.apple.universe@freight.zombinet>,
Eric Pozharski <apple.universe@posteo.net> wrote:
with <87msgourzu.fsf@nosuchdomain.example.com> Keith Thompson wrote:
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 21:09:25 -0800, Keith Thompson wrote:
Lawrence D'Oliveiro <ldo@nz.invalid> writes:
On Sat, 21 Dec 2024 16:36:32 -0800, Keith Thompson wrote:
*SKIP* [ 8 lines 5 levels deep] # there isn't anything to seeGood one. You really got LDO's number to a T.
I can no longer keep up with the rapid movement of the goalposts. IPoint of this (and many others) discussion is to generate engagement.
see no point in continuing this discussion.
Turns out, talking about lizard people doesn't ignite Usenet. Who
knew.
The funny thing is: I was the one who first brought up: Wouldn't this
all be easier/better/more-maintainable in AWK. And I knew for a dead
cold fact that mentioning AWK would bring the Perl nuts out of the
woodwork to attack AWK and say how much better it would be to use
Perl.
I couldn't have been more prescient.
Is perl a mandatory POSIX tool? No.
On Tue, 24 Dec 2024 06:20:18 -0600, Ed Morton wrote:
Is perl a mandatory POSIX tool? No.
Nobody cares.
POSIX compliance is just a means to an end not an end in
itself. Linux pays attention to POSIX where it matters, and ignores it
where it doesn’t.
This is one case where it doesn’t.