• usenet group statistics

    From validator@21:1/5 to All on Sat Aug 24 19:43:48 2024
    I'm looking for ideas on how to get statistics for a specific Usenet
    group without the necessity of downloading all messages.
    For example, I'm interested in stats like the number of messages
    in the last month, the number of unique users over a month, etc.

    I've tried writing some scripts using telnet/nc and lynx,
    but it seems that most servers have protections against
    mass scraping of content.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D@21:1/5 to validator on Sat Aug 24 22:20:42 2024
    On Sat, 24 Aug 2024 19:43:48 -0000 (UTC), validator <altvalidator@pm.me> wrote: >I'm looking for ideas on how to get statistics for a specific Usenet
    group without the necessity of downloading all messages.
    For example, I'm interested in stats like the number of messages
    in the last month, the number of unique users over a month, etc.
    I've tried writing some scripts using telnet/nc and lynx,
    but it seems that most servers have protections against
    mass scraping of content.

    much prefer 40tude dialog overall as a newsreader, but for "one-click" newsgroup statistics, xananews is also portable and works wonderfully:

    https://github.com/graemeg/xananews/releases/tag/v1.21
    Release v1.21 Latest
    graemeg released this Aug 22, 2017
    xananews_32bit_v1.21.zip >https://github.com/graemeg/xananews/releases/download/v1.21/xananews_32bit_v1.21.zip
    (xananews_32bit_v1.21.zip / 2.55 MB)

    e.g., newsgroup statistics for "alt.fan.usenet" for the past 365 days:

    XanaNews Statistic for alt.fan.usenet. 8/24/2024 2:28:49 PM
    From article 455 (8/29/2023 6:16:28 AM) to article 890 (8/24/2024 1:43:48 PM) >Number of threads ................... 66
    Number of articles .................. 437
    Average articles per thread ......... 6.62
    Number of unanswered posts .......... 38
    Number of posts from XanaNews users .. 0
    Top Threads
    Ranking Articles Subject
    ------- -------- ----------------------------------
    1 88 If you were to design a netnews protocol today...
    2 49 How can we prevent the relentless spam targeting usenet groups?
    3 42 https://www.fidonet.org/
    4 32 Re: google groups users
    5 22 Touhou: USENET in Gensokyo?
    6 18 net.martyr
    7 14 Remembering the 90s flame wars: a simpler time of cyberbullying
    8 14 Best mac app for reading and posting to newsgroups?
    9 13 Current top myths about Usenet
    10 12 What is Usenet? (Was: Re: Online talk: "Federation and moderation: Usenet as the original decentralized social network")
    11 11 Re: social media disruption
    12 11 Did Google Groups disconnect from newsgroups yet?
    13 10 Usenet Newsgroups Part I - I find some Usenet Archives on CD-ROM
    14 9 Examples of Abuse
    15 7 How to fix the internet (MIT Technology Review)
    16 7 Poll: Why are you not a Usenet newsgroup moderator? (Warning: Sarcastic Content)
    17 6 the best freeware newsreaders
    18 5 Column width
    19 5 Scoring rules - part 1
    20 4 Sample .newsrc file for recommended newsgroups (UPDATED)
    21 3 Subject matter newsgroups: Threading the needle between arcanery and kookery
    22 3 in TB (ThunderBird), Middle-pane -- (unlike GG) there's no Message-count for each thread
    23 3 TSFAQ Links to Prof. Timo Salmi's FAQ materials (archive.org, 2012)
    24 3 "Usenet is a cesspool, a dung heap." - Patrick A. Townson, The UNIX-HATERS Handbook (1994)
    25 2 Trying to define Usenet by what it isn't (or should not be)
    26 2 ignore me, I need to know if this works
    27 2 Newsreader killfiles, the "Nunchucks of Usenet"
    28 2 Re: Re: google groups users
    Top Posters
    Ranking Articles Name Most Used Newsreader
    ------- -------- -------------------------- --------------------
    1 79 D
    2 41 Paul W. Schleck
    3 20 Scott Dorsey
    4 18 Adam H. Kerman
    5 15 Frank Slootweg
    6 15 Sn!pe
    7 14 candycanearter07
    8 12 The Running Man
    9 10 Stainless Steel Rat
    10 9 Grant Taylor
    11 8 Retro Guy
    12 8 Steve Bonine
    13 7 Mima-sama
    14 7 Lawrence D'Oliveiro
    15 7 Steven M. O'Neill
    16 7 Marco Moock
    17 6 Richard Kettlewell
    18 6 Computer Nerd Kev
    19 6 rdh
    20 6 The Real Bev
    21 6 El Kabong
    22 6 Stefan Ram
    23 6 Kijin Seija
    24 5 yeti
    25 5 Oriole
    26 5 vallor
    27 4 CSS Dixieland
    28 4 sticks
    29 4 Takane Yamashiro
    30 4 Parodper
    31 4 Rich
    32 4 Kerr-Mudd, John
    33 3 floffy@gallaxial.com
    34 3 Borax Man
    35 3 George Musk
    36 3 Nomen Nescio
    37 3 Dan Cross
    38 3 DrunkenThon
    39 3 Anton Shepelev
    40 3 D. Ray
    41 3 Hen Hanna
    42 2 Andy K.
    43 2 Samuel Christie
    44 2 John
    45 2 Francis
    46 2 Bozo User
    47 2 William Stickers
    48 2 Andy Burns
    49 2 immibis
    50 2 Johanne Fairchild
    51 1 Randolph Fritz
    52 1 validator
    53 1 Dan Purgert
    54 1 user.q2vkh
    55 1 6R1MR34P3R
    56 1 Silver Skull
    57 1 1megameter
    58 1 Sean Lynch
    59 1 Bradley K. Sherman
    60 1 Ted Heise
    61 1 georgemoody
    62 1 ?? Good Guy ??
    63 1 Jack
    64 1 pschleck
    65 1 Mr ╓n!on
    66 1 Tim Skirvin
    67 1 Mack A. Damia
    68 1 David Lesher
    69 1 Richard Harnden
    70 1 Tristan Miller
    71 1 ExistingNull
    72 1 Rene Kita
    73 1 Rayner Lucas
    74 1 HenHanna
    75 1 LucLan
    76 1 Spawn
    77 1 Kyonshi
    78 1 Rink
    79 1 oldernow
    80 1 rek2 hispagatos
    81 1 Ant
    82 1 Louis Epstein
    83 1 Alterego
    Top Newsreaders
    Ranking Articles Newsreader Users >------- -------- -------------------------------------------- -----
    1 436 <unknown> 83
    [end quote]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Marco Moock@21:1/5 to All on Sun Aug 25 11:53:44 2024
    On 24.08.2024 um 19:43 Uhr validator wrote:

    I've tried writing some scripts using telnet/nc and lynx,
    but it seems that most servers have protections against
    mass scraping of content.

    You could operate your own server or ask an operator to create
    statistics for you.

    Some also operate innreport and publish the contents, e.g. https://www.eternal-september.org/stats/index.html
    https://news.nk.ca/

    If those statistics are good for you, you can search the web for
    "Daily Usenet report"

    --
    kind regards
    Marco

    Send spam to 1724521428muell@cartoonies.org

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From vallor@21:1/5 to All on Mon Aug 26 19:14:14 2024
    On Sun, 25 Aug 2024 11:53:44 +0200, Marco Moock <mm+usenet-es@dorfdsl.de>
    wrote in <vaeuv9$1q1gt$4@dont-email.me>:

    On 24.08.2024 um 19:43 Uhr validator wrote:

    I've tried writing some scripts using telnet/nc and lynx,
    but it seems that most servers have protections against mass scraping
    of content.

    You could operate your own server or ask an operator to create
    statistics for you.

    Some also operate innreport and publish the contents, e.g. https://www.eternal-september.org/stats/index.html https://news.nk.ca/

    If those statistics are good for you, you can search the web for "Daily Usenet report"

    Might be able to get the information from the group's overview data...

    --
    -v

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From vallor@21:1/5 to All on Tue Aug 27 13:00:46 2024
    On Sat, 24 Aug 2024 19:43:48 -0000 (UTC), validator <altvalidator@pm.me>
    wrote in <134a4ebb8707ad5d96c835f832f03692e0545ee0@i2pn2.org>:

    I'm looking for ideas on how to get statistics for a specific Usenet
    group without the necessity of downloading all messages.
    For example, I'm interested in stats like the number of messages
    in the last month, the number of unique users over a month, etc.

    I've tried writing some scripts using telnet/nc and lynx,
    but it seems that most servers have protections against
    mass scraping of content.

    I just posted a perl script to comp.os.linux.advocacy that
    tallies posts from posters in a given group for a
    given Date: regex. It doesn't download any articles, just
    uses xover.

    It's a work in progress, but it shows how to do it
    using the NNTP "xover" command.

    Right now, it only looks at the last 10000 articles in a group. If
    I can find a parsedate for perl that understands the Usenet date format,
    I'd have it do a binary search with multiple xover commands to find
    the first article in a given period.

    --
    -v

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D@21:1/5 to vallor on Tue Aug 27 17:03:52 2024
    On Tue, 27 Aug 2024 13:00:46 -0000 (UTC), vallor <vallor@cultnix.org> wrote: >On Sat, 24 Aug 2024 19:43:48 -0000 (UTC), validator <altvalidator@pm.me> >wrote in <134a4ebb8707ad5d96c835f832f03692e0545ee0@i2pn2.org>:
    I'm looking for ideas on how to get statistics for a specific Usenet
    group without the necessity of downloading all messages.
    For example, I'm interested in stats like the number of messages
    in the last month, the number of unique users over a month, etc.
    I've tried writing some scripts using telnet/nc and lynx,
    but it seems that most servers have protections against
    mass scraping of content.

    I just posted a perl script to comp.os.linux.advocacy that
    tallies posts from posters in a given group for a
    given Date: regex. It doesn't download any articles, just
    uses xover.
    It's a work in progress, but it shows how to do it
    using the NNTP "xover" command.
    Right now, it only looks at the last 10000 articles in a group. If
    I can find a parsedate for perl that understands the Usenet date format,
    I'd have it do a binary search with multiple xover commands to find
    the first article in a given period.

    hmm... "comp.os.linux.advocacy"
    using 40tude dialog, localhost
    5/9/2020-8/27/2024 (1571 days)

    133357 total
    73981 xpost
    16899 googl

    that newsgroup is spam-flooded

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)