• Re: Historical articles and longest retention. dot-stuff

    From Billy G.@42-contact-42@pugleaf.net to news.software.nntp on Wed Nov 11 09:19:50 2026
    From Newsgroup: news.software.nntp

    On 9/12/23, Russ Allbery wrote:
    "Billy G. (go-while)" <no-reply@no.spam> writes:
    On 12.09.23 20:49, Russ Allbery wrote:

    So it's allowed to have a header field name that starts with a period,
    as well as all sorts of other exotic and fascinating stuff that you
    don't see in practice.

    great thanks your help is priceless!

    I will never have time to write this personally, but if someone with a
    love of pedantic nit-picks ever felt like writing an NNTP and netnews compliance test suite that tried all sorts of edge conditions like this
    that could be run as a read/write NNTP client against a server (and presumably also the target of a feed), that would be a real service to everyone writing NNTP and netnews software.

    I have from time to time thought about rewriting some of the random tools
    I use, like tinyleaf, in Rust, but my thought process gets as far as
    header parsing and then I groan and find other hobbies.


    Two years late, but here is a start:

    https://github.com/go-while/nntp-testkit

    (MIT, Go, no dependencies)

    The part closest to what you described is nntp-conformance,
    a standalone black-box client:

    go install github.com/go-while/nntp-testkit/cmd/nntp-conformance@latest

    nntp-conformance -addr news.example:119 [-group test.group] [-tls]
    [-user u -pass p]

    It runs 57 checks against any server, each on its own connection:

    RFC 3977 reader commands
    (GROUP, LISTGROUP ranges, ARTICLE/HEAD/BODY/STAT,
    NEXT/LAST, OVER, HDR, LIST variants, NEWGROUPS,
    NEWNEWS, POST, the error codes for no group,
    no current article, out of range and unknown ids),
    AUTHINFO (RFC 4643),
    streaming and IHAVE (RFC 4644, RFC 3977 6.3.2),
    STARTTLS (RFC 4642,
    including the CVE-2011-0411 case of commands pipelined behind STARTTLS)
    and COMPRESS (RFC 8054, including a large dot-stuffed upload).

    A capability the server does not advertise is a skip, not a
    failure; nothing is posted unless you name a group with -group.

    Dot-stuffing, since that is where this thread went:
    the POST and COMPRESS upload checks send dot-stuffed body lines
    and read them back, and the client under the hood reports bare LF,
    bad stuffing and similar wire mistakes instead of tolerating them.

    So far it has been run against:
    INN, Diablo, NewsCache, go-pugleaf and OpenUsenetServer

    Latest runs (interim, 57 checks; a skip is a capability the
    server does not offer, or one the run did not set up):

    server pass fail skip
    ----------------------------- ---- ---- ----
    INN 2.8.0 prerelease (nnrpd) 41 0 16
    OpenUsenetServer 48 0 9
    NewsCache (over implicit TLS) 33 0 24

    INN's skips are the setup: no login, reader connection only,
    no implicit-TLS port; those runs are next. Diablo and go-pugleaf
    were run with earlier versions.

    It found real bugs in some of them, and a few bugs in its own checks,
    which is the point of running it against more than one server.

    For the "target of a feed" part there is a fake transit peer
    (streaming and IHAVE, with knobs for deferrals, rejects, stalls,
    drops and odd replies) that a server's outgoing feed can be pointed at,
    and a library for building test articles byte for byte, including
    deliberately broken ones: bare CR or LF, NUL, 8-bit bodies, folded
    headers, colon-less header lines, overlong lines.

    Those are Go packages for now, used by the e2e tests
    of INN, Diablo, NewsCache, go-pugleaf and OpenUsenetServer.

    For scale, just for info: the projects' own suites are much larger
    than the 57 checks, because most of their tests are about the
    server's insides, not about NNTP on the wire:

    project tests mostly about
    ----------- --------------------- ---------------------------------
    Diablo 569 tests, 211 files history/kpdb locking, expire,
    dnewslink/dspoolout feeds, spool,
    auth backends, control/PGP
    go-pugleaf 221 files, ~76k lines database, processor, web, mbox
    import, feed sender, server
    NewsCache 38 files, ~22k lines cache, upstream faults, spool,
    disk full, config, signals
    INN ~30 files storage methods, innfeed/innxmit,
    ctlinnd, filters, makehistory
    OpenUsenet- 43 files store, feed claims, admin,
    Server session

    What they share is NNTP and netnews behaviour,
    and that part is what moves into the kit, one check at a time.

    The packages are also the easy way to write your own tests.
    The pieces the checks are built from are there for any Go test:

    - a strict client (fails on wire mistakes instead of hiding them)
    - a fake server: scripted, handler-driven or backed by a small
    article store, with faults (delays, drops, overridden, endless
    or dripping replies), STARTTLS and COMPRESS
    - the fake transit peer
    - a runner that starts your server binary, stops it and dumps
    its logs when a test fails
    - test certificates (also expired and wrong-name ones)
    - the article builder

    A test against your own server is a few lines:

    c := client.Dial(t, "127.0.0.1:1119")
    a := article.Minimal(t, "local.test")
    a.Set("Subject", "two spaces, kept?")
    c.ExpectPost(240, a.Bytes())
    c.ExpectCmd(220, "ARTICLE %s", a.MessageID())

    and to check an outgoing feed, point it at a fake peer:

    peer := fakeserver.NewPeer(t, fakeserver.Options{},
    &fakeserver.PeerOptions{})
    // feed to peer.Addr(), post an article, then look at
    // peer.Articles() (or ArticlesRaw() for the exact bytes)


    Planned next, for the command line:

    a round-trip run:
    - inject a set of tricky articles by POST and IHAVE
    - read them back
    - diff what was served against what was sent

    and a relay run:
    - feed into one server
    - read from another
    - allow only specific headers (Path, Xref, ...) to differ

    We also probed how the servers answer commands with odd white
    space and control bytes, about 195 spellings ("ARTICLE <id>" with
    two spaces, TABs, VT, NUL, bare CR, ...), before deciding what a
    check may require. Interim result over seven servers
    (INN nnrpd and innd, Diablo reader and feed port, NewsCache, go-pugleaf, OpenUsenetServer):

    - all accept what RFC 3977 3.1 asks for: one or more SP or TAB
    between words, keywords in any case (now a new check on main)

    - all accept trailing white space and a bare LF line end, and
    all wait for more input after a bare CR

    - they differ on the rest: leading white space, an extra
    argument, VT/FF/NBSP or a bare CR as separator, lines over
    512 octets, CR CR LF (only INN says 501), and NUL: INN and
    Diablo cut the command at a NUL, so "ARTICLE <id>" followed by
    NUL and junk serves the article, while NewsCache answers
    "501 NUL in command"


    Edge cases you, or anyone, would like to see tried are very welcome,
    here or as GitHub issues.


    Code was written by AI, with my assistance XD
    --
    .......
    Billy G. (go-while)
    https://pugleaf.net
    @Newsgroup: rocksolid.nodes.help
    irc.pugleaf.net:6697 (SSL) #lounge
    TOR-IRC: ij7fmstcudrwyty5p6iloodeenur4wksovlhbyrfubat7eixplmrxvqd.onion:6667

    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Billy G.@42-contact-42@pugleaf.net to news.software.nntp on Wed Nov 11 10:09:29 2026
    From Newsgroup: news.software.nntp

    On 11/11/26 09:19, Billy G. wrote:
    Two years late, but here is a start:

    OBOE!
    --
    .......
    Billy G. (go-while)
    https://pugleaf.net
    @Newsgroup: rocksolid.nodes.help
    irc.pugleaf.net:6697 (SSL) #lounge
    TOR-IRC: ij7fmstcudrwyty5p6iloodeenur4wksovlhbyrfubat7eixplmrxvqd.onion:6667 --- Synchronet 3.22a-Linux NewsLink 1.2
  • From noreply@noreply@mixmin.net to news.software.nntp on Wed Nov 11 15:28:19 2026
    From Newsgroup: news.software.nntp

    On Wed, 11 Nov 2026 10:09:29 +0000, Billy G. wrote:
    On 11/11/26 09:19, Billy G. wrote:
    Two years late, but here is a start:

    OBOE!

    --
    .......
    Billy G. (go-while)
    https://pugleaf.net
    @Newsgroup: rocksolid.nodes.help
    irc.pugleaf.net:6697 (SSL) #lounge
    TOR-IRC: ij7fmstcudrwyty5p6iloodeenur4wksovlhbyrfubat7eixplmrxvqd.onion:6667

    just for fun . . .

    https://duckduckgo.com/?q=oboe+acronym&assist=true
    The acronym OBOE stands for "Observation Bombing of Enemy," a World War II military term for a bombing technique.
    Yamaha Corporation timbreandorchestration.org

    Definition of OBOE

    The acronym OBOE stands for "Observation Bombing of Enemy." This term originated during World War II and refers to a specific bombing technique used by military forces.

    Context of Use

    Military Application: The OBOE technique was employed to enhance the
    effectiveness of bombing missions by utilizing observation methods to
    identify enemy targets.
    Historical Significance: This method reflects the strategic advancements
    in aerial warfare during the World War II era.

    Understanding the term OBOE in this context highlights its relevance in military history and tactics.
    [end quoted "search assist"]
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Billy G.@42-contact-42@pugleaf.net to news.software.nntp on Wed Nov 11 17:45:53 2026
    From Newsgroup: news.software.nntp

    On 11/11/26 15:28, D wrote:
    On Wed, 11 Nov 2026 10:09:29 +0000, Billy G. wrote:
    On 11/11/26 09:19, Billy G. wrote:
    Two years late, but here is a start:

    OBOE!

    just for fun . . .

    https://duckduckgo.com/?q=oboe+acronym&assist=true
    The acronym OBOE stands for "Observation Bombing of Enemy," a World War II >> military term for a bombing technique.
    Yamaha Corporation timbreandorchestration.org

    Definition of OBOE

    The acronym OBOE stands for "Observation Bombing of Enemy." This term
    originated during World War II and refers to a specific bombing technique
    used by military forces.

    just for stupids . . .

    https://duckduckgo.com/?q=OBOE+programming+acronym

    OBOE stands for Off By One Error, which is a common logic error in
    programming where a loop or calculation is off by one iteration or
    value, often due to incorrect use of inequality operators.

    https://en.wikipedia.org/wiki/Off-by-one_error
    --
    .......
    Billy G. (go-while)
    https://pugleaf.net
    @Newsgroup: rocksolid.nodes.help
    irc.pugleaf.net:6697 (SSL) #lounge
    TOR-IRC: ij7fmstcudrwyty5p6iloodeenur4wksovlhbyrfubat7eixplmrxvqd.onion:6667 --- Synchronet 3.22a-Linux NewsLink 1.2
  • From noreply@noreply@mixmin.net to news.software.nntp on Wed Nov 11 19:03:24 2026
    From Newsgroup: news.software.nntp

    On Wed, 11 Nov 2026 17:45:53 +0000, Billy G. wrote:
    On 11/11/26 15:28, D wrote:
    On Wed, 11 Nov 2026 10:09:29 +0000, Billy G. wrote:
    On 11/11/26 09:19, Billy G. wrote:
    Two years late, but here is a start:

    OBOE!

    just for fun . . .
    https://duckduckgo.com/?q=oboe+acronym&assist=true
    The acronym OBOE stands for "Observation Bombing of Enemy," a World War II >>> military term for a bombing technique.
    Yamaha Corporation timbreandorchestration.org
    Definition of OBOE
    The acronym OBOE stands for "Observation Bombing of Enemy." This term
    originated during World War II and refers to a specific bombing technique >>> used by military forces.

    just for stupids . . .
    https://duckduckgo.com/?q=OBOE+programming+acronym

    OBOE stands for Off By One Error, which is a common logic error in programming where a loop or calculation is off by one iteration or
    value, often due to incorrect use of inequality operators.

    https://en.wikipedia.org/wiki/Off-by-one_error

    didn't know that, though i have heard of the vitruvian architect:

    (using tor browser 16.0.1)
    https://en.wikipedia.org/wiki/Off-by-one_error
    ...
    A fencepost error (occasionally called a telegraph pole, lamp-post,
    or picket fence error) is a specific type of off-by-one error. An
    early description of this error appears in the works of Vitruvius.[1]
    ...
    Citations
    1. Pollio, Marcus Vitruvius. "Book III, Chapter 4, Paragraph 3".
    De Architectura [On Architecture]. Translated by Thayer, Bill.
    [end quoted excerpt]

    if "4 cubits make a man", 1/4 of any man's height equals 1 cubit

    https://duckduckgo.com/?q=tallest+shortest+man+ever&assist=true
    The tallest man in recorded history is Sultan Kosen from Turkey,
    who stands at 251 cm (8 ft 3 in), while the shortest man ever
    verified is Chandra Bahadur Dangi from Nepal, measuring 54.6 cm
    (1 ft 9.5 in).

    kosen's cubit would be about 4.6 times longer than dangi's cubit,
    the classic biblical short cubit estimate is ~17.5"/~44.5 cm, so
    the average "short" man stands ~5'-10"/~178 cm, seems reasonable,
    maybe tribal, the long cubit could be attributable to "tall" men
    --- Synchronet 3.22a-Linux NewsLink 1.2