I do like to have a summary of system state at hand (as in: df,
free, etc.), and I also do like to keep a history of such states,
so that I can track down why the system misbehaved at some point
or another. To record such states, I wrote sysreport.sh [1].
[1] http://am-1.org/~ivan/src/misc-utils-is-2026/sysreport.sh
What follows is the notes on usage and select details on
implementation. Unsurprisingly, re-reading the code made me
realize there're more bugs than I've intended. The bugs are
pointed out below, and some of them I intend to fix in 0.5.
Constructive criticism welcome.
I typically use sysreport more or less as follows. (I actually
have different invocation scripts on different machines; the
example below attempts to generalize them.)
#!/bin/sh
set -e
set -C -u
case "$PATH" in
(*/sbin | */sbin:*) ;;
(*) PATH=${PATH}:/sbin:/usr/sbin ;;
esac
set --
case "$(uname)" in
(Linux) set -- sensors ip_s_link ;;
(NetBSD) set -- envstat netstat_i netstat_ibd ;;
esac
set -- "$@" \
df_rw df_tmpfs \
" System swap usage" file:/proc/swaps
command -V free > /dev/null 2>&1 \
&& set -- "$@" free
test -s /proc/mdstat \
&& set -- "$@" file:/proc/mdstat
set -- "$@" \
ntp_drift ntpq_pn \
" System uptime" file:/proc/uptime \
" Load averages" file:/proc/loadavg
: "${BLOCKSIZE:=1024}"
: "${SYSREPORT_CSS:=sfn._uiJu17bUkphwwotAi3ttCKt5J3WZOj6tiMd8_vBru8.css}" export BLOCKSIZE SYSREPORT_CSS
sysreport "$@" \
> private/hist/"$(hostname)/sysreport/$(date -u +%F)".en.xhtml
That, of course, could be a daily Cron / snooze(1) job.
The tool is written in POSIX shell (though it uses external
commands outside POSIX as well) and produces HTML/XML (XHTML) [2].
The XML produced is intended to be at the same time valid and
conforming (non-XML) HTML.
[2] http://html.spec.whatwg.org/
HTML/XML is used purely for framing purposes, so that if I ever
need to extract the output of a particular command, I can do it
with XML processing tools. The tool makes no attempt to
"prettify" the output of individual commands it invokes, and
just escapes < and & (then wraps it all into <pre /> elements):
xhtml_escape () {
sed -e "s/&/\\&/g; s/</\\</g;"
}
There's a subtle bug: this escaping is sufficient for the
/content/ of XML elements, but xhtml_escape is used for
an attribute value once as well - and those require that
either ' or " is escaped, too, possibly both:
css=$( printf %s\\n "${SYSREPORT_CSS:-default.css}" \
| xhtml_escape)
Of course, there's a second bug in that ${css} is not used at all:
<link rel="stylesheet" href="${SYSREPORT_CSS:-default.css}" />
Besides, it probably needs URI %-escaping instead, which I know
no easy way how to implement in POSIX shell.
When a <section /> of the resulting output is just the output of
a particular command, the latter is invoked via xhtml_simple_cmd:
sysreport_envstat () {
xhtml_simple_cmd s-envstat "Sensor readings" \
envstat
}
xhtml_simple_cmd () {
xhtml_rep_header "$1" "$2"
shift 2
local x
x=
("$@" | xhtml_escape) || x=$?
xhtml_rep_footer
if test -n "$x" ; then
cat <<EOF
<p >Command terminated with non-zero exit status: $?</p>
EOF
fi
}
Note that as the section identifiers (like s-envstat above) are
fixed, "$ sysreport envstat envstat " will silently produce a
non-valid XML having duplicate "s-envstat" element identifiers.
With Bash, I'd have resolved that by using an array, so that
the first section would get id="s-envstat", the second,
"s-envstat-1", and so on. It is of course possible to use a
global counter instead (s-foo-1, s-bar-2, s-baz-3, etc.), but
that'd mean it won't be possible to extract given command's
output from the resulting file using a "static" XML identifier.
That said, I don't see much value in having the output of a
given command to appear multiple times in the resulting file.
As such, fixing this issue is not a priority.
A global counter variable is, however, used for identifiers
of "file:" sections (below), as I've figured that using the
basenames would somewhat likely result in duplicates (as above),
while using full filenames (with suitable substitutions, to
make the identifiers good for CSS selectors) would lead to
identifiers that are too verbose.
file_counter=0
sysreport_file () {
xhtml_rep_header s-file-${file_counter} \
"Contents of <code >$(printf "$1" | xhtml_escape)</code>"
file_counter=$((1 + file_counter))
xhtml_escape < "$1"
## .
xhtml_rep_footer
}
The missing %s\\n for printf is an obvious bug, which I don't
recall ever being triggered as I don't pass filenames containing
% or \ there.
The xhtml_rep_header and xhtml_rep_footer functions follow.
heading=
xhtml_rep_header () {
local x h
if test "${1:-XXX}" = "${1#*[!.0-9a-zA-Z-]}" ; then
x=" id=\"${1}\""
else
x=
fi
if test -n "$heading" ; then
h=$(printf %s\\n "$heading" | xhtml_escape)
fi
cat <<EOF
<section${x}>
<header>
<h2 >${h:-${2}}</h2>
</header>
<p >Started: <time >$(date -u +%F\ %T\ UTC | xhtml_escape)</time></p>
<pre
EOF
## .
printf \>
}
As could be seen, in lieu of escaping its first argument to
make it suitable for an id= attribute value, it checks it for
non-emptiness and non-presence of characters outside of the
[.0-9a-zA-Z-] set (a check that could perhaps be made clearer
by using "case" instead of "if"), and silently ignores it
otherwise.
The use of "date -u" is a matter of personal preference.
xhtml_rep_footer () {
cat <<EOF
</pre>
<p >Finished: <time >$(date -u +%F\ %T\ UTC | xhtml_escape)</time></p>
</section>
EOF
}
For some things, I've found no suitable command or option, so
more complex functions are used, e. g.:
sysreport_df_rw () {
xhtml_rep_header s-df-rw "Disk usage, writable, device-backed filesystems"
awk '! seen_p[$1] && /^\/dev/ && $4 ~ /(^|[ \t,])?rw([ \t,]|$)/ {
print $1; seen_p[$1] = 1; }' \
< /proc/mounts \
| tr \\n \\0 | xargs -r0 -- df -- \
| xhtml_escape
## .
xhtml_rep_footer
}
I prefer to have many smaller (up to 4480 MiB - a tad less than
the size of a DVD+R) filesystems. When one fills up, I turn it
into a read-only archive, and, if needed, create a replacement.
Thus, I often have dozens of filesystems mounted, only a handful
of which are writable and thus worth mentioning in a sysreport.
The code above only applies df(1) to filesystems that are
a. writable, and b. are mounted from /dev/* - which is to say,
are /not/ kernfs, overlay, procfs, tmpfs, etc.
I do not /expect/ the filenames of device specials to contain
', " or \, but I find it a good habit to inhibit the processing
of these characters in xargs anyway, to which end I use "-0",
and hence need "tr \\n \\0 " as well. (Former versions of this
code used GNU awk and printf("%s\0", $1).)
I also tend to use VM-backed filesystems often. For instance,
I might mount a new tmpfs, extract an archive there, use it for
a while, then umount - instead of extracting an archive to a
disk-based FS, followed by "$ rm -rf " when no longer needed.
To keep track of my tmpfs instances, I use a script like:
sysreport_df_tmpfs () {
xhtml_rep_header s-df-tmpfs "Disk usage, in-memory filesystems"
## FIXME: signal an error if both df invocations fail?
{ df -t mfs || : ; df -t tmpfs || : ; } 2> /dev/null \
| sed -e "1d; / 0 /d;" \
| LC_ALL=C sort -srnk3 \
| xhtml_escape
## .
xhtml_rep_footer
}
Note that there's a difference between GNU and NetBSD versions
of df(1): GNU allows multiple filesystem types to be specified
by repeated use of -t ($ df -t mfs -t tmpfs), while NetBSD
requires that multiple types are given as a single -t option
value, separated by commas ($ df -t mfs,tmpfs.) To cover both
variants above, I have to invoke df(1) twice.
I eliminate the header (1d) so that sort(1) doesn't put it after
all the entries. (Then again, now that I care about NetBSD mfs,
I can get /two/ headers there - the second of which won't be
eliminated, which is yet another bug.)
I also remove entries with " 0 " so that I can omit filesystems
that were mounted but are not actually used (have "0" in the
"Used" column), though it will omit filled-up instances ("0" in
the "Available" column) just as well. That rarely happens to
me, though, so fixing it is yet again not a priority.
XHTML needs its document header and footer markup, which I
implement thus:
xhtml_header () {
cat <<EOF
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title >${title}</title>
<link rel="stylesheet" href="${SYSREPORT_CSS:-default.css}" />
<!-- FIXME: should be superseded by @viewport in .css -->
<meta name="viewport" content="initial-scale=1.0" />
</head>
<body>
<article class="h-entry">
<h1 class="p-name" >${title}</h1>
EOF
}
xhtml_footer () {
cat <<EOF
</article>
</body>
</html>
EOF
}
I supply minimal http://microformats.org/ version 2 metadata -
give the entire <article /> the class of "entry", and set its
"name" property to the title - either generated, or supplied
via the SYSREPORT_TITLE environment variable. I don't publish
these reports, so I have no idea if it makes any difference to
outside reusers. I've got an impression that version 2 never
got implemented in popular search engines, and I can't be
bothered to use version 1 alongside or instead.
I realize that there might be cases when it makes sense to use
a "library card" <title /> different to the <h1 /> heading, but
I think those are mostly the same cases where one'd want to use
a more complex <header /> (than a mere heading element), which
is a tad too tough to implement in POSIX shell, IMO.
Finally, it's all brought together with the following "main loop"
over (non-option) arguments.
shown_header_p=
for sec ; do
com_arg=
case "$sec" in
(" "*) heading=${sec# } ; continue ;;
(file:*)
com=sysreport_${sec%%:*} ; com_arg=${sec#*:} ;;
(*) com=sysreport_${sec} ;;
esac
if ! command -v -- "$com" > /dev/null ; then
gerr 0 "Warning: %s: Unknown section; ignored" "$sec"
continue
fi
if test -z "$shown_header_p" ; then
shown_header_p=yes
xhtml_header
fi
## FIXME: only allowing a single argument for now
"$com" "$com_arg"
heading=
done
if test -n "$shown_header_p" ; then
xhtml_footer
fi
In principle, I can allow for a single argument to be passed to
other sysreport_* functions - by using *:* rather than file:* -
but at this point, none of them allows one, and signalling an
error when the argument passed is not used would needlessly
complicate the code.
I believe the above covers all the really interesting bits of
the code. Thoughts?
case "$(uname)" in
(Linux) set -- sensors ip_s_link ;;
(NetBSD) set -- envstat netstat_i netstat_ibd ;;
On 2026-09-11, Lawrence D'Oliveiro wrote:
On Thu, 10 Sep 2026 16:17:08 +0000, Ivan Shmakov wrote:
I typically use sysreport more or less as follows. (I actually
have different invocation scripts on different machines; the
example below attempts to generalize them.)
case "$(uname)" in
(Linux) set -- sensors ip_s_link ;;
Supports 300-odd Linux distros.
On 2026-09-10, Janis Papanagnou wrote:
On 2026-09-10 18:17, Ivan Shmakov wrote:
[1] http://am-1.org/~ivan/src/misc-utils-is-2026/sysreport.sh
What follows is the notes on usage and select details on
implementation. Unsurprisingly, re-reading the code made me
realize there're more bugs than I've intended. The bugs are
pointed out below, and some of them I intend to fix in 0.5.
Constructive criticism welcome.
It's quite cumbersome to wade through all that (as you say not yet finalized, buggy) code.
- I suggest to fix the code first before asking for criticism. Or,
if you have specific questions, just post the respective context
(instead of many hundreds of lines).
I admit that I consider it too cumbersome in this form.
xhtml_simple_cmd () {
xhtml_rep_header "$1" "$2"
shift 2
local x
Skimming over it I noticed a few things, for example you are using
commands like 'shift 2' without checking whether there's sufficient
numbers of arguments or handling it,
or that you (IMO unnecessarily) are specifying some shell-specific
'local' keywords (while at the same time you're defining just
/bin/sh as the interpreter and have a remark about it being operated
by a POSIX shell),
then there's also some style issues (e. g. unnecessary quotes and
escapes).
Unfortunately your code also doesn't have any supporting comments
which would have made the code for the reviewers easier to
understand; this I consider a most important prerequisite.
#!/bin/shIf PATH happens to be defined but empty (""), it will be set to
set -e
set -C -u
case "$PATH" in
(*/sbin | */sbin:*) ;;
(*) PATH=${PATH}:/sbin:/usr/sbin ;;
Ivan Shmakov <ivan@siamics.netREMOVE.invalid>:
#!/bin/sh
set -e
set -C -u
case "$PATH" in
(*/sbin | */sbin:*) ;;
(*) PATH=${PATH}:/sbin:/usr/sbin ;;
If PATH happens to be defined but empty (""), it will be set to
":/sbin:/usr/sbin" which is equivalent to ".:/sbin:/usr/sbin",
i.rC>e.-aat the first position there is a reference to the current
working directory which may be exploited by an attacker.
As a remedy either make the script fail if PATH is empty or[SNIP]
undefined, like
On 2026-09-10, Janis Papanagnou wrote:
On 2026-09-10 18:17, Ivan Shmakov wrote:
>> [1] http://am-1.org/~ivan/src/misc-utils-is-2026/sysreport.sh
>> What follows is the notes on usage and select details on
>> implementation. Unsurprisingly, re-reading the code made me
>> realize there're more bugs than I've intended. The bugs are
>> pointed out below, and some of them I intend to fix in 0.5.
>> Constructive criticism welcome.
> It's quite cumbersome to wade through all that (as you say not yet
> finalized, buggy) code.
Is finalized, bug-free code all that common out there? If
anything, my experience reading manual pages is that they will
all too often have a "BUGS" section with one or more items
yet to be resolved.
[...]
> Skimming over it I noticed a few things, for example you are using
> commands like 'shift 2' without checking whether there's sufficient
> numbers of arguments or handling it,
That's a valid concern, but not applicable to the code as a whole,
as the top of sysreport.sh sets -e and -u POSIX shell options:
set -e
set -C -u
[...]
> then there's also some style issues (e. g. unnecessary quotes and
> escapes).
I've long made certain code style choices, some of which might
disagree with those of others. If there're specific examples,
I'm willing to try my best to explain why I wrote the code that
way and not another.
The only two issues (i. e., inconsistencies) I can spot with my
style is that I often fail to indent "cat <<EOF", and that
I didn't quote "$file_counter", like (in "diff -u"-like syntax):
file_counter=0
sysreport_file () {
- xhtml_rep_header s-file-${file_counter} \
+ xhtml_rep_header s-file-"$file_counter" \
My coding style is to quote substitutions /if and only if/
both the language context allows word splitting /and/ such
word splitting is not desired. E. g.:
foo=$(bar) ; ## no word splitting possible
for f in $(ls -t) ; do : ; done ; ## word splitting desired
bar=baz ; qux "$bar" ; ## word splitting allowed, not desired
> Unfortunately your code also doesn't have any supporting comments
> which would have made the code for the reviewers easier to
> understand; this I consider a most important prerequisite.
I frankly hoped that my explanations in the original post would
be sufficient. Otherwise, being its author, I see that code as
mostly obvious. Case in point: after not touching it for four
years, I've had no trouble following what's going on.
Regardless, if there're specific questions, I'm willing to try
my best to answer them.
Ivan Shmakov <ivan@siamics.netREMOVE.invalid>:
#!/bin/sh
set -e
set -C -u
case "$PATH" in
(*/sbin | */sbin:*) ;;
(*) PATH=${PATH}:/sbin:/usr/sbin ;;
If PATH happens to be defined but empty (""), it will be set to
":/sbin:/usr/sbin" which is equivalent to ".:/sbin:/usr/sbin",
i.rC>e.-aat the first position there is a reference to the current
working directory which may be exploited by an attacker.
That's arguably a bug that should be fixed in the path searching
mechanisms under execvp and wherever not.
An empty PATH component should not be taken as dot, but simply skipped,
since such a situation can arise in an imperfect or buggy shell script
PATH manipuation.
On 2026-09-11, Helmut Waitzmann <nn.throttle@erine.email> wrote:
Ivan Shmakov <ivan@siamics.netREMOVE.invalid>:
#!/bin/sh
set -e
set -C -u
case "$PATH" in
(*/sbin | */sbin:*) ;;
(*) PATH=${PATH}:/sbin:/usr/sbin ;;
If PATH happens to be defined but empty (""), it will be set to
":/sbin:/usr/sbin" which is equivalent to ".:/sbin:/usr/sbin",
i.rC>e.-aat the first position there is a reference to the current
working directory which may be exploited by an attacker.
That's arguably a bug that should be fixed in the path searching
mechanisms under execvp and wherever not.
An empty PATH component should not be taken as dot, but simply skipped,
since such a situation can arise in an imperfect or buggy shell script
PATH manipuation.
PATH does not have to be empty for it to happen; just some variable
has to be empty/unset due to a typo or whatever:
PATH="$PATH:$foo_instalation" # typo, not set -u in effect.
One line of code could plug this.
On 2026-09-11, Helmut Waitzmann wrote:
Ivan Shmakov <ivan@siamics.netREMOVE.invalid>:
#!/bin/sh
set -e
set -C -u
case "$PATH" in
(*/sbin | */sbin:*) ;;
(*) PATH=${PATH}:/sbin:/usr/sbin ;;
If PATH happens to be defined but empty (""), it will be set to ":/sbin:/usr/sbin" which is equivalent to ".:/sbin:/usr/sbin",
i.rC>e.-aat the first position there is a reference to the current
working directory which may be exploited by an attacker.
As a remedy either make the script fail if PATH is empty or
undefined, like
PATH=${PATH:?}:/sbin:/usr/sbin
or don't prepend ":" if PATH is empty or undefined:
PATH=${PATH:+${PATH}:}/sbin:/usr/sbin
The latter remedy might cause searching for executables in a
nonrCEstandard but implementationrCEdefined way to fail, though, if
the system assigns implementationrCEdefined semantics to an unset
or empty PATH, which are unknown to this script (see <https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap08.html#tag_08_03>).
On 2026-09-12, Janis Papanagnou wrote:
On 2026-09-11 10:24, Ivan Shmakov wrote:
On 2026-09-10, Janis Papanagnou wrote:
"IS" == On 2026-09-10 18:17, Ivan Shmakov wrote:
[1] http://am-1.org/~ivan/src/misc-utils-is-2026/sysreport.sh
What follows is the notes on usage and select details on
implementation. Unsurprisingly, re-reading the code made me
realize there're more bugs than I've intended. The bugs are
pointed out below, and some of them I intend to fix in 0.5.
Is finalized, bug-free code all that common out there? If
anything, my experience reading manual pages is that they will
all too often have a "BUGS" section with one or more items
yet to be resolved.
Oh, I was just referring to the statement you made; that you know
already some bugs, and I imagined that you could handle those. (I
might have misread it anyway.)
Note that as the section identifiers (like s-envstat above) are
fixed, "$ sysreport envstat envstat " will silently produce a
non-valid XML having duplicate "s-envstat" element identifiers.
Skimming over it I noticed a few things, for example you are using
commands like 'shift 2' without checking whether there's sufficient
numbers of arguments or handling it,
That's a valid concern, but not applicable to the code as a whole,
as the top of sysreport.sh sets -e and -u POSIX shell options:
I had seen that. (For my own tools a rough exit is rarely the right
action. - That's why I mentioned it.)
Though some things would unsettle me, like in $(printf "$1" | ...)
relying on "$1" containing no '-'; I'd always write printf '%s' "$1". Especially strange I find that you made a comment about it being a
bug that not yet got "triggered" (with references to '%' and '\'
but not to '-').
- Why not just fix it; it's trivial.
My coding style is to quote substitutions /if and only if/ both the
language context allows word splitting /and/ such word splitting is
not desired. E. g.:
foo=$(bar) ; ## no word splitting possible
Similar in cases with 'case', like
case $PATH in
case $(uname) in
where no quoting is necessary (no word splitting).
| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 76 |
| Nodes: | 6 (0 / 6) |
| Uptime: | 54:14:36 |
| Calls: | 1,265 |
| Calls today: | 1 |
| Files: | 1,362 |
| D/L today: |
3 files (2,684K bytes) |
| Messages: | 285,449 |