From Newsgroup: news.software.nntp

Has a LIST command variant that combines several of the foundational
LIST commands ever been considered? With group metadata split up between
LIST ACTIVE, ACTIVE.TIMES, and NEWSGROUPS (and COUNTS if you need that
too), if clients end up needing all this info, it would be more
efficient to use one command to get it all + cut down on RTTs, etc.

I was thinking of doing something like the below to at least support my
own vertically integrated NNTP software (server + client), as I want to
be able to get all this from a client efficiently without storing any
state there, and doing a LIST ACTIVE, followed by several iterations of ACTIVE.TIMES, NEWSGROUPS, and COUNTS to fill in the missing data is just
plain dumb to me. Since I use an extended active file format that has everything, it would be easy for me to fulfill such a request; I realize
the same is not true of most of other software. Notwithstanding that,
and before I independently start making up new commands, has this ever
been considered before, and not pursued for any reason? Might such a
command ever be standardized in the future?

Syntax

LIST EVERYTHING [wildmat]

The LIST EVERYTHING command returns a combination of the information
returned by the LIST (ACTIVE), LIST ACTIVE.TIMES, LIST NEWSGROUPS, and
LIST COUNT commands in a single response to allow clients to more
efficiently request all this data than using multiple LIST commands in succession.

The information is returned as a multi-line data block following the 215 response code and contains one line per newsgroup. Each line of this
list MUST consist of eight fields separated from each other by a TAB, in
the following order:

* The name of the newsgroup
* The reported high water mark for the group
* The reported low water mark for the group
* The estimated number of articles in the group
* The current status of the group on this server
* The time when this group was created on this news server, measured in seconds since the start of January 1, 1970
* The entity that created the newsgroup
* A short description of the group

If no description is available for a group, the description and TAB
separating the creator entity and description MAY be omitted.

Newsgroups that would be included in the response to LIST ACTIVE (with
the same wildmat argument, if provided) MUST be included. Conversely,
any groups present in the LIST EVERYTHING response MUST also be returned
in the LIST ACTIVE response.

The meanings of these parameters are as provided for the corresponding
LIST commands (LIST ACTIVE, LIST ACTIVE.TIMES, and LIST NEWSGROUPS from
RFC 3977 and LIST COUNT from RFC 6048).

Example (TAB has been represented as a | for visual clarity):

[C] LIST EVERYTHING misc.test
[S] 215 List of newsgroups follows
[S] misc.test|322|310|11|y|1780168665|John Smith
<newsmaster@example.com>|A miscellaneous test group
[S] .
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

InterLinked <nntp@phreaknet.org> spake the secret code <10vfdpf$12fb7$1@dont-email.me> thusly:

Has a LIST command variant that combines several of the foundational
LIST commands ever been considered? With group metadata split up between >LIST ACTIVE, ACTIVE.TIMES, and NEWSGROUPS (and COUNTS if you need that
too), if clients end up needing all this info, it would be more
efficient to use one command to get it all + cut down on RTTs, etc.

You can use multiple connections to obtain the information in
parallel.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/1/2026 5:25 PM, Richard wrote:

InterLinked <nntp@phreaknet.org> spake the secret code <10vfdpf$12fb7$1@dont-email.me> thusly:

Has a LIST command variant that combines several of the foundational
LIST commands ever been considered? With group metadata split up between
LIST ACTIVE, ACTIVE.TIMES, and NEWSGROUPS (and COUNTS if you need that
too), if clients end up needing all this info, it would be more
efficient to use one command to get it all + cut down on RTTs, etc.

You can use multiple connections to obtain the information in
parallel.

That's a dumb workaround for something the protocol COULD do much more efficiently. I will not be doing that.

It seems to me there is no reason not to do it the way I proposed... I
realize NNTP is pretty stagnant compared to IMAP but this just makes sense.

Another use case I've encountered is being able to limit the # of items returned in certain commands, like OVER; otherwise if you only want the
first 50 items in a group, the most efficient thing seems to be to do a LISTGROUP first to get the bounds, but even then you still have to read
the whole response and throw it away... seems like a "LIMIT" option
would help but I haven't thought about this problem as deeply.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

InterLinked in news.software.nntp:

[...]

Another use case I've encountered is being able to limit the # of items >returned in certain commands, like OVER; otherwise if you only want the
first 50 items in a group, the most efficient thing seems to be to do a >LISTGROUP first to get the bounds, but even then you still have to read
the whole response and throw it away... seems like a "LIMIT" option
would help but I haven't thought about this problem as deeply.

| GROUP de.test
| 211 613 310064 310710 de.test

No LISTGROUP needed.

| OVER 310700-
| 224 Overview information for 310700- follows
[...]

| OVER 310650-310699
[...]

Marcel (Lines: 29)
--
ro!roCroCro< ro!roCro< ro!roCro< ro!roCroCroCroCroCroCroCro< ro!roCroCroCroCroCroCro< ..67..
ro!roCroCroCro> roe roe ro#roCroCroCroCroCroCro> roe ro!roCroCroCro> ro!roCroCroCroCro> ro#roCro< ro!roCro> ..67..
roCroCro> ro!roCroCroCro> ro#roCroCroCroCroCroCroCro< ro#roCro> ro!roCroCroCro> ro!roCro< ro!roCroCro> ro#roCro< ro!roCroCroCroCroCroCroCroCroCro< ro!roCro< ro!roCroCro<
ro#roCroCroCroCroCroCroCroCroCroCroCroCroCroCro> ro#roCroCroCroCroCro> ro#roCroCro> ro#roCroCro> ..55..ro#roCro> ro#roCro> ro#roCroC
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/2/2026 4:11 AM, Marcel Logen wrote:

InterLinked in news.software.nntp:

[...]

Another use case I've encountered is being able to limit the # of items
returned in certain commands, like OVER; otherwise if you only want the
first 50 items in a group, the most efficient thing seems to be to do a
LISTGROUP first to get the bounds, but even then you still have to read
the whole response and throw it away... seems like a "LIMIT" option
would help but I haven't thought about this problem as deeply.

| GROUP de.test
| 211 613 310064 310710 de.test

No LISTGROUP needed.

Good point, but that's the easy case. If even one article is missing,
you don't know where without doing a LISTGROUP. And if you don't, you
could try getting the first 50 and only end up with 40.

Also, I question if a GROUP response with no gaps can be relied upon,
given that INN advertises a false high water mark (based on the last
article received, not the highest existing article), and similarly the
count may just be an estimate. My software always tells the truth, but
it seems that can't be relied on in general.

IMAP doesn't have this problem, because you can use sequence numbers
rather than UIDs when paginating. NNTP only has the "UID" equivalent, no sequence numbers, which honestly is probably the better choice but
raises a few unique issues here not present in IMAP.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

InterLinked <nntp@phreaknet.org> writes:

Good point, but that's the easy case. If even one article is missing, you don't know where without doing a LISTGROUP. And if you don't, you could
try getting the first 50 and only end up with 40.

Also, I question if a GROUP response with no gaps can be relied upon,
given that INN advertises a false high water mark (based on the last
article received, not the highest existing article), and similarly the
count may just be an estimate. My software always tells the truth, but it seems that can't be relied on in general.

Are you running some kind of aggressive spam filtering or otherwise
expecting to do lots of article deletion? It feels like you are heavily optimizing for groups with deleted articles, but in practice on Usenet
this is usually extremely rare, particularly if one disables
unauthenticatd cancels (which is common these days).

On most servers, the one time in a hundred when you ask for 50 articles
and only get 49 is barely worth thinking about.
--
Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>

Please post questions rather than mailing me directly.
<https://www.eyrie.org/~eagle/faqs/questions.html> explains why.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/2/2026 2:02 PM, Russ Allbery wrote:

InterLinked <nntp@phreaknet.org> writes:

Good point, but that's the easy case. If even one article is missing, you
don't know where without doing a LISTGROUP. And if you don't, you could
try getting the first 50 and only end up with 40.

Also, I question if a GROUP response with no gaps can be relied upon,
given that INN advertises a false high water mark (based on the last
article received, not the highest existing article), and similarly the
count may just be an estimate. My software always tells the truth, but it
seems that can't be relied on in general.

Are you running some kind of aggressive spam filtering or otherwise
expecting to do lots of article deletion? It feels like you are heavily optimizing for groups with deleted articles, but in practice on Usenet
this is usually extremely rare, particularly if one disables
unauthenticatd cancels (which is common these days).

On most servers, the one time in a hundred when you ask for 50 articles
and only get 49 is barely worth thinking about.

I don't intend to, but it's possible. I've been experimenting with
sucking articles using a reading connection recently just to toy with
what groups and articles I want. I'm down to fewer than 900 groups of
the 23,000 on Eternal September (i.e. excluding dead groups, excluding political groups, excluding spam groups, etc.), and even then,
individual groups can still get junk. Ideally, I aim to reject this all
at connection time, but as I refine my filters, I could miss something
and decide to retroactively delete received articles, especially if the
group has a long or unlimited retention and there is a real cost to
keeping garbage in the spool. Besides that, was thinking about
performance in the general case, not just mine.

But it's good to know this is uncommon, thank you for that datapoint. I
guess I could just blindly ask for LOW... LOW + (N - 1) for a range, and
if it comes back short and under the high water mark, then do a
LISTGROUP and proceed from there. That will probably perform better on average.

So I guess the need for "RANGE" is not very pressing, but a "LIST
EVERYTHING" would still be useful. I'm doing LIST, LIST ACTIVE.TIMES,
LIST NEWSGROUPS, and LIST COUNTS serially right now when listing groups,
and since I have < 1000 groups, and I'm testing on the same test server,
it's not very noticeable, but I could see this adding a lot of lag for larger/remote news servers.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

Hi InterLinked,

I'm doing LIST, LIST ACTIVE.TIMES,
LIST NEWSGROUPS, and LIST COUNTS serially right now when listing groups

As LIST COUNTS contains the information provided by LIST (ACTIVE), you
could skip the LIST command when the news server advertises the LIST
COUNTS capability.

Also, I suggest that you cache the results of LIST ACTIVE.TIMES and LIST NEWSGROUPS for a given server, and that you just sent it again when
either the user wants a refresh (via a dedicated button to refresh the descriptions and creation dates) or you notice a change of status a
creation or a removal in the newsgroups list received with LIST or LIST COUNTS.
I guess it would do the job and save you commands.
--
Julien |eLIE

-2-aLes quatre |oges de l'homme sont-a: la petite enfance, l'enfance,
l'adolescence, l'obsolescence.-a-+ (Art Linkletter)

--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/2/26 14:12, InterLinked wrote:

On 6/2/2026 2:02 PM, Russ Allbery wrote:

InterLinked <nntp@phreaknet.org> writes:

Good point, but that's the easy case. If even one article is missing,
you
don't know where without doing a LISTGROUP. And if you don't, you could
try getting the first 50 and only end up with 40.

Also, I question if a GROUP response with no gaps can be relied upon,
given that INN advertises a false high water mark (based on the last
article received, not the highest existing article), and similarly the
count may just be an estimate. My software always tells the truth,
but it
seems that can't be relied on in general.

Are you running some kind of aggressive spam filtering or otherwise
expecting to do lots of article deletion? It feels like you are heavily
optimizing for groups with deleted articles, but in practice on Usenet
this is usually extremely rare, particularly if one disables
unauthenticatd cancels (which is common these days).

On most servers, the one time in a hundred when you ask for 50 articles
and only get 49 is barely worth thinking about.

I don't intend to, but it's possible. I've been experimenting with
sucking articles using a reading connection recently just to toy with
what groups and articles I want. I'm down to fewer than 900 groups of
the 23,000 on Eternal September (i.e. excluding dead groups, excluding political groups, excluding spam groups, etc.), and even then,
individual groups can still get junk. Ideally, I aim to reject this all
at connection time, but as I refine my filters, I could miss something
and decide to retroactively delete received articles, especially if the group has a long or unlimited retention and there is a real cost to
keeping garbage in the spool. Besides that, was thinking about
performance in the general case, not just mine.

But it's good to know this is uncommon, thank you for that datapoint. I guess I could just blindly ask for LOW... LOW + (N - 1) for a range, and
if it comes back short and under the high water mark, then do a
LISTGROUP and proceed from there. That will probably perform better on average.

So I guess the need for "RANGE" is not very pressing, but a "LIST EVERYTHING" would still be useful. I'm doing LIST, LIST ACTIVE.TIMES,
LIST NEWSGROUPS, and LIST COUNTS serially right now when listing groups,
and since I have < 1000 groups, and I'm testing on the same test server, it's not very noticeable, but I could see this adding a lot of lag for larger/remote news servers.

If you are not using pipelining, it is an easy win to eliminate some
RTTs when you know what you want prior and need to get it from multiple commands. RTT is the main benefit for inn's nnrpd right now. I want to
take a look and see if it can also be used to build up some I/O
parallelism on the backend.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/2/2026 5:32 PM, Julien |eLIE wrote:

Hi InterLinked,

I'm doing LIST, LIST ACTIVE.TIMES, LIST NEWSGROUPS, and LIST COUNTS
serially right now when listing groups

As LIST COUNTS contains the information provided by LIST (ACTIVE), you
could skip the LIST command when the news server advertises the LIST
COUNTS capability.

Ah, yes, I'd assumed that the list of groups could differ from ACTIVE,
but that's only true of ACTIVE.TIMES and NEWSGROUPS since those
traditionally involve separate files - I can't think why the COUNT
listing wouldn't be identical to ACTIVE since it should use the same
file. At least that brings me down from 3 avoidable RTTs to 2.

Also, I suggest that you cache the results of LIST ACTIVE.TIMES and LIST NEWSGROUPS for a given server, and that you just sent it again when
either the user wants a refresh (via a dedicated button to refresh the descriptions and creation dates) or you notice a change of status a
creation or a removal in the newsgroups list received with LIST or LIST COUNTS.

If I were building a "real" offline client, I would do that. I wrote
this mainly for testing, and I don't want to cache or save anything,
it's fully stateless - intentionally - as I always want to see what's on
the server. Sort of like some webmail software (and the mail clients I
have written are similar, they cache nothing and retrieve everything
using IMAP - important on shared systems like bulletin board systems
where caching is counterproductive, since it's requesting everything
from the local mail server anyways - though this is less of an issue
with news since everyone has the same (or part of the same) group
selection).

IMAP has evolved to support "online only" clients relatively well with extensions like LIST-STATUS and STATUS=SIZE, etc. My suggested
optimization really only applies to online NNTP clients, which I'll
grant are not much of a thing, but it still seems like a niche where it
would be useful. Apologies if I'm rocking the boat by proposing
extensions everyone else things are dumb :)
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/2/2026 6:25 PM, Kevin Bowling wrote:

On 6/2/26 14:12, InterLinked wrote:

On 6/2/2026 2:02 PM, Russ Allbery wrote:
So I guess the need for "RANGE" is not very pressing, but a "LIST
EVERYTHING" would still be useful. I'm doing LIST, LIST ACTIVE.TIMES,
LIST NEWSGROUPS, and LIST COUNTS serially right now when listing
groups, and since I have < 1000 groups, and I'm testing on the same
test server, it's not very noticeable, but I could see this adding a
lot of lag for larger/remote news servers.

If you are not using pipelining, it is an easy win to eliminate some
RTTs when you know what you want prior and need to get it from multiple commands.-a RTT is the main benefit for inn's nnrpd right now.-a I want to take a look and see if it can also be used to build up some I/O
parallelism on the backend.

Yeah, in the actual C code for sending/sucking articles, I'm obviously pipelining commands like CHECK, TAKETHIS, ARTICLE, etc. Not sure if
there are other good candidates for pipelining I missed that are worth it?

I'm not at the moment in the web client (which is really just a simple
~500 line PHP script I wrote for testing, I would describe it as similar
to a view-only csiph.com with no searching, no threading, no caching,
and more metadata shown, basically - for example you don't show article numbers or group creation info and I want to see that for testing and
nerdy reasons).

And RTT is just one thing really, the other was just general
client/server efficiency. Yes, there are ways to "work around it", but
it would be so much more efficient for my server to give the client all
this info at once in one go, but there's nothing standardized in NNTP to facilitate that. From a purist perspective, all that wasted work irks me
- the server has to work harder and the client has to work harder when
they could use such a command to mutually both be more efficient.
There's also the bandwidth usage. Repeating all the group names each
time for the responses is SO wasteful... especially if you have tens of thousands of groups. Assuming an average group name length of 20
characters, a server with 23,000 groups wastes an ENTIRE MB just
repeating group names in ACTIVE.TIMES and NEWSGROUPS. Put this on a
33.6k dial-up link, and you have wasted over FOUR MINUTES of the
client's bandwidth for NO reason. (And even on fast links, if you pay
for outbound bandwidth, that adds up over time.)

Sure, maybe an offline client would only do it once and not every time.
But why waste a user's time even once when those four minutes could be
easily saved? It is pure bloat. We can easily do better than this.

Anyways, if I were to add my own non-standard LIST EVERYTHING command
(and I do plan to), would it be recommended to name it something like
LIST XEVERYTHING or can I just use whatever keyword I want? I realize practically speaking it makes no difference, clients would ignore
anything they don't recognize, but I'd like to try to be "compliant" if
there is a standard for non-standardized subcommands/keywords.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/3/26 14:30, Julien |eLIE wrote:

Hi InterLinked,

Assuming an average group name length of 20 characters, a server with
23,000 groups wastes an ENTIRE MB just repeating group names in
ACTIVE.TIMES and NEWSGROUPS. Put this on a 33.6k dial-up link, and you
have wasted over FOUR MINUTES of the client's bandwidth for NO reason.

You may want to use the COMPRESS command; it works pretty well on LIST responses.

I was going to suggest this as well. Even on localhost, the overhead of compression is lower (or at least shifts it correctly to user) than the overhead of socket buffering+wakeups for moving this type of data
between processes.

The only thing I really want from the NNTP protocol/server for readers
for csiph-web is a THREAD command because I suspect it would be a lot
more efficient to do server side but I'd need to prove it out.

Anyways, if I were to add my own non-standard LIST EVERYTHING command
(and I do plan to), would it be recommended to name it something like
LIST XEVERYTHING or can I just use whatever keyword I want?

RFC 6648 deprecated the use of the "X" prefix so you can use whatever
name you want.
Maybe LIST ACTIVE.ALL or something like that would be better than EVERYTHING?-a (What EVERYTHING refers to is not clear.)

FWIW, as you say you are nerdy and want the maximum available
information about newsgroups, the NAS protocol is worth mentioning (RFC 4707).-a I once implemented a part of it.-a Feel free to play with it:
-a-a-a telnet nas.trigofacile.com 991

You would have interesting information to show about hierarchies and newsgroups, even deleted ones :)
Unfortunately, no one maintains a NAS database with extensive data. Mine
is just a proof of concept, with only a few information manually added.
For instance:

HIER fr
611 Information follows for hierarchies
Name: fr
Status: Complete
Description: French language
Charter: http://www.usenet-fr.net/fur/usenet/presentation-fr.html
Netiquette: http://www.usenet-fr.net/fr-chartes/netiquette.html
Netiquette: http://www.usenet-fr.net/fr-chartes/savoir-communiquer.html Netiquette: http://www.usenet-fr.net/fr-chartes/emily-postnews.html Netiquette: http://www.usenet-fr.net/fur/usenet/bons-usages.html
Netiquette: http://www.usenet-fr.net/fur/usenet/repondre-sur-usenet.html Netiquette: http://www.usenet-fr.net/spam.html
Rules: https://www.usenet-fr.net/creation.html
Ctl-Send-Adr: control@usenet-fr.news.eu.org
Ctl-Newsgroup: fr.usenet.distribution
Language: fr
Charset: ISO-8859-1
Charset: ISO-8859-15
Charset: UTF-8
Encoding: text/plain
Newsgroup-Type: Discussion
Hier-Type: Global
Hier-Type: Non-commercial
Name-Length: 41
Comp-Length: 18
Date-Create: 19930310000000
Source: https://www.usenet-fr.net/
Ctl-PGP-Key:
U control@usenet-fr.news.eu.org
B 4096
I 970EB10B
L https://www.usenet-fr.net/pgp-fr-2020.txt
F D5F3 69B2 9757 3622 0153-a 54E7 FA42 3E89 970E B10B
V GnuPG v2.1.18
K------BEGIN PGP PUBLIC KEY BLOCK-----
K-Version: GnuPG v2.1.18
K-Comment: fr.* hierarchy
K-
K-mQINBF+4LkcBEADLTAOPM6z/nj1zFox1KZEY44GTSwUaB5gBflJMTjC2izhh1Q6N K-bsfya4REfRijENkqOFGV/QkWQ8Va/ru+74yOKvLDdiD0RqGjvazlgvOGRLaTcxbG K-w6e4cAddbZ2O65XsqfNB+K11WQdgL42OAY6aGTrbj17rLANHFSiGd+gVhL3U3DNE
[...]
K-=vYcy
K -----END PGP PUBLIC KEY BLOCK-----

.

DATA fr.bienvenue.questions
612 Information follows for newsgroups
Name: fr.bienvenue.questions
Status: Removed
Description: Les premi|?res questions sur Usenet. (O||-a? Comment-a?) Date-Create: 19971016173454
Date-Delete: 20111219233002
Replacement: fr.bienvenue

.

DATA fr.comp.usenet.serveurs
612 Information follows for newsgroups
Name: fr.comp.usenet.serveurs
Status: Unmoderated
Description: Les logiciels serveurs de news Usenet.
Charter: http://www.usenet-fr.net/fur/chartes/comp.usenet.serveurs.html Date-Create: 20050606205005

.

Just mentioning it :)

--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/3/2026 5:30 PM, Julien |eLIE wrote:

Hi InterLinked,

Assuming an average group name length of 20 characters, a server with
23,000 groups wastes an ENTIRE MB just repeating group names in
ACTIVE.TIMES and NEWSGROUPS. Put this on a 33.6k dial-up link, and you
have wasted over FOUR MINUTES of the client's bandwidth for NO reason.

You may want to use the COMPRESS command; it works pretty well on LIST responses.

Yes, but zero bytes are still better than a smaller number of them :)

Anyways, if I were to add my own non-standard LIST EVERYTHING command
(and I do plan to), would it be recommended to name it something like
LIST XEVERYTHING or can I just use whatever keyword I want?

RFC 6648 deprecated the use of the "X" prefix so you can use whatever
name you want.
Maybe LIST ACTIVE.ALL or something like that would be better than EVERYTHING?-a (What EVERYTHING refers to is not clear.)

Noted, I'm not married to the name (that was the first thing that popped
into my head), but this makes sense to me so I'll use this instead
unless there are other suggestions.

If I write this up more formally, for this or other extensions, is it
worth submitting these anywhere or should I just keep it to myself for
now? I know the NNTP working group is basically dead. I do have other extensions in mind down the line too, especially catering to the "online
only" / multiple client use case.

FWIW, as you say you are nerdy and want the maximum available
information about newsgroups, the NAS protocol is worth mentioning (RFC 4707).-a I once implemented a part of it.-a Feel free to play with it:
-a-a-a telnet nas.trigofacile.com 991

You would have interesting information to show about hierarchies and newsgroups, even deleted ones :)
Unfortunately, no one maintains a NAS database with extensive data. Mine
is just a proof of concept, with only a few information manually added.

Very interesting. I think I did come across NAS a few times in my
readings. I'll wait until I've finished with the basic NNTP stuff and
then move onto exploring this :)
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/3/2026 7:09 PM, Kevin Bowling wrote:

On 6/3/26 14:30, Julien |eLIE wrote:

Hi InterLinked,

Assuming an average group name length of 20 characters, a server with
23,000 groups wastes an ENTIRE MB just repeating group names in
ACTIVE.TIMES and NEWSGROUPS. Put this on a 33.6k dial-up link, and
you have wasted over FOUR MINUTES of the client's bandwidth for NO
reason.

You may want to use the COMPRESS command; it works pretty well on LIST
responses.

I was going to suggest this as well.-a Even on localhost, the overhead of compression is lower (or at least shifts it correctly to user) than the overhead of socket buffering+wakeups for moving this type of data
between processes.

Sure, but again, not having to send the data *at all* means you don't
have to move *any* data around. And now you have to expend CPU cycles compressing data that wasn't needed to begin with. I get that there are workarounds, but I can simply do better.

The only thing I really want from the NNTP protocol/server for readers
for csiph-web is a THREAD command because I suspect it would be a lot
more efficient to do server side but I'd need to prove it out.

I think everything is probably more efficient for the server to do if possible, and in general I prefer the server to do as much work as
possible, though that's from my perspective of designing clients and
servers where the end work gets done on one of my machines either way at
the end of the day (and in general I prefer servers to do as much of the
work as possible regardless). If I were a large scale provider,
obviously there is an incentive to force the client to do more work,
though I think that's less efficient overall. (Aside: maybe this is one
reason why the big freemail providers' IMAP servers suck and have no features.)

Besides THREAD, I think there are probably other IMAP extensions that
might make sense to port over to NNTP. Maybe SEARCH, for one? Right now,
if I wanted to add a search to my online-only NNTP client, well, I can't.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/3/26 20:08, InterLinked wrote:

On 6/3/2026 7:09 PM, Kevin Bowling wrote:

On 6/3/26 14:30, Julien |eLIE wrote:

Hi InterLinked,

Assuming an average group name length of 20 characters, a server
with 23,000 groups wastes an ENTIRE MB just repeating group names in
ACTIVE.TIMES and NEWSGROUPS. Put this on a 33.6k dial-up link, and
you have wasted over FOUR MINUTES of the client's bandwidth for NO
reason.

You may want to use the COMPRESS command; it works pretty well on
LIST responses.

I was going to suggest this as well.-a Even on localhost, the overhead
of compression is lower (or at least shifts it correctly to user) than
the overhead of socket buffering+wakeups for moving this type of data
between processes.

Sure, but again, not having to send the data *at all* means you don't
have to move *any* data around. And now you have to expend CPU cycles compressing data that wasn't needed to begin with. I get that there are workarounds, but I can simply do better.

No argument. I do LIST ACTIVE and LIST NEWSGROUPS for my group
listings, the later just to get descriptions. But this information is
highly cachable, which is probably true in most readers.

The only thing I really want from the NNTP protocol/server for readers
for csiph-web is a THREAD command because I suspect it would be a lot
more efficient to do server side but I'd need to prove it out.

I think everything is probably more efficient for the server to do if possible, and in general I prefer the server to do as much work as
possible, though that's from my perspective of designing clients and
servers where the end work gets done on one of my machines either way at
the end of the day (and in general I prefer servers to do as much of the work as possible regardless). If I were a large scale provider,
obviously there is an incentive to force the client to do more work,
though I think that's less efficient overall. (Aside: maybe this is one reason why the big freemail providers' IMAP servers suck and have no features.)

Besides THREAD, I think there are probably other IMAP extensions that
might make sense to port over to NNTP. Maybe SEARCH, for one? Right now,
if I wanted to add a search to my online-only NNTP client, well, I can't.

I guess many clients are somewhat ossified to the protocol. For example
I mostly use Thunderbird out of habit and I am not sure how, in current
state, it could cope with a global search result set.

For THREAD, it is kind of a unique problem to the multi-user agent like
a web interface. The cost of threading locally for a single user is
basically "free" because they'll want the OVER stuff anyway for listings.

So there is some chicken-and-egg here. But from a protocol perspective
I guess both THREAD and SEARCH would be a pretty trivial addition -- the complexity is all on the server and client implementations to figure out.

I found out search is fairly difficult to do well, the csiph-web index
was quite an odyssey.

I suspect some admins would want some search server like solr, elastic
or quickwit. So it would have to be plugins that allows for such
things. And then some indexing process. Mine is librified, in process,
with Tantivy (think Lucene or Xapian) and the indexing are blocking
async tasks (coroutine, threads or processes for C). Probably not that different than the way Dovecot does FTS.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

Kevin Bowling wrote:

Besides THREAD, I think there are probably other IMAP extensions that
might make sense to port over to NNTP. Maybe SEARCH, for one? Right now,
if I wanted to add a search to my online-only NNTP client, well, I can't.

So there is some chicken-and-egg here. But from a protocol perspective
I guess both THREAD and SEARCH would be a pretty trivial addition -- the complexity is all on the server and client implementations to figure out.

JFTR: SEARCH was discussed/proposeed before back in 1998, see <https://datatracker.ietf.org/doc/html/draft-ietf-nntpext-srch-00>
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

InterLinked <nntp@phreaknet.org> spake the secret code <10vl0uq$2hsmv$1@dont-email.me> thusly:

On 6/1/2026 5:25 PM, Richard wrote:

InterLinked <nntp@phreaknet.org> spake the secret code
<10vfdpf$12fb7$1@dont-email.me> thusly:

Has a LIST command variant that combines several of the foundational
LIST commands ever been considered? With group metadata split up between >>> LIST ACTIVE, ACTIVE.TIMES, and NEWSGROUPS (and COUNTS if you need that
too), if clients end up needing all this info, it would be more
efficient to use one command to get it all + cut down on RTTs, etc.

You can use multiple connections to obtain the information in
parallel.

That's a dumb workaround for something the protocol COULD do much more >efficiently. I will not be doing that.

I'm sorry that I solved your problem before waiting for new protocols
to be adopted worldwide across news servers.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On 6/12/2026 3:54 PM, Richard wrote:

InterLinked <nntp@phreaknet.org> spake the secret code <10vl0uq$2hsmv$1@dont-email.me> thusly:

On 6/1/2026 5:25 PM, Richard wrote:

InterLinked <nntp@phreaknet.org> spake the secret code
<10vfdpf$12fb7$1@dont-email.me> thusly:

Has a LIST command variant that combines several of the foundational
LIST commands ever been considered? With group metadata split up between >>>> LIST ACTIVE, ACTIVE.TIMES, and NEWSGROUPS (and COUNTS if you need that >>>> too), if clients end up needing all this info, it would be more
efficient to use one command to get it all + cut down on RTTs, etc.

You can use multiple connections to obtain the information in
parallel.

That's a dumb workaround for something the protocol COULD do much more
efficiently. I will not be doing that.

I'm sorry that I solved your problem before waiting for new protocols
to be adopted worldwide across news servers.

The original question was explicitly about extending the protocol to be
more efficient, not how to do make do with the protocol as it currently exists. Of course there are many ways to do that, some better than
others. (Using multiple connections is worse in some respects because it requires even *more* bandwidth than is already being wasted, so in the
context of slow connections, this is a bad idea.)

In the same vein, most protocol extensions are not "necessary" but do
make things easier/better in ways. I am working with my own client and
server so I can optimize independent of what other news servers are doing.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

Hi InterLinked,

Assuming an average group name length of 20
characters, a server with 23,000 groups wastes an ENTIRE MB just
repeating group names in ACTIVE.TIMES and NEWSGROUPS. Put this on a
33.6k dial-up link, and you have wasted over FOUR MINUTES of the
client's bandwidth for NO reason.

You may want to use the COMPRESS command; it works pretty well on LIST responses.

Anyways, if I were to add my own non-standard LIST EVERYTHING command
(and I do plan to), would it be recommended to name it something like
LIST XEVERYTHING or can I just use whatever keyword I want?

RFC 6648 deprecated the use of the "X" prefix so you can use whatever
name you want.
Maybe LIST ACTIVE.ALL or something like that would be better than
EVERYTHING? (What EVERYTHING refers to is not clear.)



FWIW, as you say you are nerdy and want the maximum available
information about newsgroups, the NAS protocol is worth mentioning (RFC
4707). I once implemented a part of it. Feel free to play with it:
telnet nas.trigofacile.com 991

You would have interesting information to show about hierarchies and newsgroups, even deleted ones :)
Unfortunately, no one maintains a NAS database with extensive data.
Mine is just a proof of concept, with only a few information manually added. For instance:


HIER fr
611 Information follows for hierarchies
Name: fr
Status: Complete
Description: French language
Charter: http://www.usenet-fr.net/fur/usenet/presentation-fr.html
Netiquette: http://www.usenet-fr.net/fr-chartes/netiquette.html
Netiquette: http://www.usenet-fr.net/fr-chartes/savoir-communiquer.html Netiquette: http://www.usenet-fr.net/fr-chartes/emily-postnews.html
Netiquette: http://www.usenet-fr.net/fur/usenet/bons-usages.html
Netiquette: http://www.usenet-fr.net/fur/usenet/repondre-sur-usenet.html Netiquette: http://www.usenet-fr.net/spam.html
Rules: https://www.usenet-fr.net/creation.html
Ctl-Send-Adr: control@usenet-fr.news.eu.org
Ctl-Newsgroup: fr.usenet.distribution
Language: fr
Charset: ISO-8859-1
Charset: ISO-8859-15
Charset: UTF-8
Encoding: text/plain
Newsgroup-Type: Discussion
Hier-Type: Global
Hier-Type: Non-commercial
Name-Length: 41
Comp-Length: 18
Date-Create: 19930310000000
Source: https://www.usenet-fr.net/
Ctl-PGP-Key:
U control@usenet-fr.news.eu.org
B 4096
I 970EB10B
L https://www.usenet-fr.net/pgp-fr-2020.txt
F D5F3 69B2 9757 3622 0153 54E7 FA42 3E89 970E B10B
V GnuPG v2.1.18
K------BEGIN PGP PUBLIC KEY BLOCK-----
K-Version: GnuPG v2.1.18
K-Comment: fr.* hierarchy
K-
K-mQINBF+4LkcBEADLTAOPM6z/nj1zFox1KZEY44GTSwUaB5gBflJMTjC2izhh1Q6N K-bsfya4REfRijENkqOFGV/QkWQ8Va/ru+74yOKvLDdiD0RqGjvazlgvOGRLaTcxbG K-w6e4cAddbZ2O65XsqfNB+K11WQdgL42OAY6aGTrbj17rLANHFSiGd+gVhL3U3DNE
[...]
K-=vYcy
K -----END PGP PUBLIC KEY BLOCK-----

.

DATA fr.bienvenue.questions
612 Information follows for newsgroups
Name: fr.bienvenue.questions
Status: Removed
Description: Les premi|?res questions sur Usenet. (O||-a? Comment-a?) Date-Create: 19971016173454
Date-Delete: 20111219233002
Replacement: fr.bienvenue

.

DATA fr.comp.usenet.serveurs
612 Information follows for newsgroups
Name: fr.comp.usenet.serveurs
Status: Unmoderated
Description: Les logiciels serveurs de news Usenet.
Charter: http://www.usenet-fr.net/fur/chartes/comp.usenet.serveurs.html Date-Create: 20050606205005

.



Just mentioning it :)
--
Julien |eLIE

-2-aLe cercle est le plus long chemin d'un point au m|-me point.-a-+ (Tom
Stoppard, _Every Good Boy Deserves Favour_)

--- Synchronet 3.22a-Linux NewsLink 1.2