| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 23 |
| Nodes: | 6 (0 / 6) |
| Uptime: | 54:40:22 |
| Calls: | 583 |
| Files: | 1,139 |
| D/L today: |
179 files (27,921K bytes) |
| Messages: | 111,801 |
Hello!
I currently notice that the polling of new messages stops every time at
the same group.
How can I further diagnose that?
Nov 27 10:19:53 rocksolid news.i2pn2.org: de.alt.admin: 211 262 1100 1364 de.alt.admin
Nov 27 10:19:53 rocksolid news.i2pn2.org: for de.alt.admin We are up to date Nov 27 10:19:53 rocksolid OPENING Local server: Resource id #434
Nov 27 10:19:53 rocksolid Updating threads for: de.alt.admin...
Nov 27 10:19:53 rocksolid Threads updated for: de.alt.admin
Nov 27 10:19:53 rocksolid CLOSING Local server: Resource id #434
Nov 27 10:19:53 rocksolid Retrieving articles for: de.alt.anime...
Nov 27 10:19:53 rocksolid news.i2pn2.org: de.alt.anime: 211 1 50 50 de.alt.anime
Nov 27 10:19:53 rocksolid news.i2pn2.org: for de.alt.anime We are up to date Nov 27 10:19:53 rocksolid OPENING Local server: Resource id #447
Nov 27 10:19:53 rocksolid Updating threads for: de.alt.anime...
Nov 27 10:19:53 rocksolid Threads updated for: de.alt.anime
Nov 27 10:19:53 rocksolid CLOSING Local server: Resource id #447
If I restart the job, it does the same and stops here.
Marco Moock wrote:
Hello!
I currently notice that the polling of new messages stops every time at
the same group.
How can I further diagnose that?
Nov 27 10:19:53 rocksolid news.i2pn2.org: de.alt.admin: 211 262 1100 1364 de.alt.admin
Nov 27 10:19:53 rocksolid news.i2pn2.org: for de.alt.admin We are up to date
Nov 27 10:19:53 rocksolid OPENING Local server: Resource id #434
Nov 27 10:19:53 rocksolid Updating threads for: de.alt.admin...
Nov 27 10:19:53 rocksolid Threads updated for: de.alt.admin
Nov 27 10:19:53 rocksolid CLOSING Local server: Resource id #434
Nov 27 10:19:53 rocksolid Retrieving articles for: de.alt.anime...
Nov 27 10:19:53 rocksolid news.i2pn2.org: de.alt.anime: 211 1 50 50 de.alt.anime
Nov 27 10:19:53 rocksolid news.i2pn2.org: for de.alt.anime We are up to date
Nov 27 10:19:53 rocksolid OPENING Local server: Resource id #447
Nov 27 10:19:53 rocksolid Updating threads for: de.alt.anime...
Nov 27 10:19:53 rocksolid Threads updated for: de.alt.anime
Nov 27 10:19:53 rocksolid CLOSING Local server: Resource id #447
If I restart the job, it does the same and stops here.
First, you might look in <spooldir>/rocksolid/<server>:<port>-remote_groups.dat and see what it
thinks the last article pulled for rec.arts.anime. It's a serialized
array, so look for something like:
"rocksolid.social";i:244" for whatever group you wish. i2pn2.org shows
that '50' is the highest number on the server, so you should see 50 or
51, but you have no articles in the group on your site, which is
strange.
Try clearing the group:
change to root or www user.
php <config_dir>/scripts/maintenance -reset de.alt.anime
and let me know what happens.
On Wed, 27 Nov 2024 10:49:57 -0000 (UTC)
Retro Guy <retroguy@novabbs.org> wrote:
Try clearing the group:
change to root or www user.
php <config_dir>/scripts/maintenance -reset de.alt.anime
and let me know what happens.I meant:
php <config_dir>/scripts/maintenance.php -reset de.alt.anime
On 27.11.2024 03:58 Uhr Retro Guy wrote:
On Wed, 27 Nov 2024 10:49:57 -0000 (UTC)
Retro Guy <retroguy@novabbs.org> wrote:
Try clearing the group:
change to root or www user.
php <config_dir>/scripts/maintenance -reset de.alt.anime
and let me know what happens.I meant:
php <config_dir>/scripts/maintenance.php -reset de.alt.anime
Now the other groups are being polled.
Did you find the cause of the situation?
I found that if a group contained zero messages, spoolnews would exit
(sort of, but more complicated). I added checking for that, logging
for that, then continuing to the next group.
I tested by adding some groups with zero messages on the server, and
it seems to work fine on my test server. This was all back the first
time we discussed this issue, and was in the 'devel' you installed.
This current issue MAY just be a broken group on your site, which we
can hopefully fix with maintenance.php.
On 27.11.2024 05:00 Uhr Retro Guy wrote:
I found that if a group contained zero messages, spoolnews would exit
(sort of, but more complicated). I added checking for that, logging
for that, then continuing to the next group.
I tested by adding some groups with zero messages on the server, and
it seems to work fine on my test server. This was all back the first
time we discussed this issue, and was in the 'devel' you installed.
This current issue MAY just be a broken group on your site, which we
can hopefully fix with maintenance.php.
I did this successfully with some groups, but it fails with one:
Maintenance
Nov 27 16:34:05 rocksolid de.soc.misc 423 No such article number 24800
Nov 27 16:34:05 rocksolid de.soc.misc 423 No such article number 24801
Nov 27 16:34:06 rocksolid de.soc.misc 423 No such article number 24802
Nov 27 16:34:06 rocksolid de.soc.misc 423 No such article number 24803
Nov 27 16:34:06 rocksolid de.soc.misc 423 No such article number 24804
Nov 27 16:34:06 rocksolid de.soc.misc 423 No such article number 24805
Nov 27 16:34:06 rocksolid de.soc.misc 423 No such article number 24806
Nov 27 16:34:06 rocksolid de.soc.misc 423 No such article number 24807
Nov 27 16:34:06 rocksolid de.soc.misc 423 No such article number 24808
Nov 27 16:34:06 rocksolid OPENING Local server: Resource id #1942
Nov 27 16:34:06 rocksolid Updating threads for: de.soc.misc...
Nov 27 16:34:06 rocksolid Threads updated for: de.soc.misc
Nov 27 16:34:06 rocksolid CLOSING Local server: Resource id #1942
You are running as user: www-data
Starting Import...
Reset: de.soc.misc
Current group pointer for de.soc.misc: 24809
New group pointer for de.soc.misc: 1
root@pi-dach:~#
root@pi-dach:~# sudo -u www-data php
/etc/rslight/scripts/maintenance.php -reset de.soc.misc
You are running as user: www-data
Starting Import...
Reset: de.soc.misc
Current group pointer for de.soc.misc: 1
New group pointer for de.soc.misc: 1
root@pi-dach:~#
It then starts from the beginning when articles are polled.
I know that article 20000 exists, so try:
maintenance.php -refill de.soc.misc 6000
On 27.11.2024 10:47 Uhr Retro Guy wrote:
I know that article 20000 exists, so try:
maintenance.php -refill de.soc.misc 6000
Spooling: de.soc.misc 1758
Spooling: de.soc.misc 1759
Spooling: de.soc.misc 1760
Spooling: de.soc.misc 1761
Spooling: de.soc.misc 1762
Spooling: de.soc.misc 1763
Spooling: de.soc.misc 1764
423 No such article number 20573
423 No such article number 20574
423 No such article number 20575
423 No such article number 20576
423 No such article number 20577
Does it ever log anything up to article number 24808?
Something is broken with that group on your server and I don't know
what it is right now. I'm currently running what I think is close to
your settings on a test site to see what happens.
On 28.11.2024 09:50 Uhr Retro Guy wrote:
Something is broken with that group on your server and I don't know
what it is right now. I'm currently running what I think is close to
your settings on a test site to see what happens.
Tell me which dbs you need, I can provide them for your for testing.
On Thu, 28 Nov 2024 16:17:14 +0000, Marco Moock wrote:
On 28.11.2024 09:50 Uhr Retro Guy wrote:
Something is broken with that group on your server and I don't know
what it is right now. I'm currently running what I think is close to
your settings on a test site to see what happens.
Tell me which dbs you need, I can provide them for your for testing.
Thanks, but I'm downloading everything from i2pn2 to make sure I have a
full install to match yours. If there is a problem with one or more of
your .db3 files, I want to see if it happens here too.
It should be done downloading later today, and I'll have a duplicate of
your site.
One thing I'm thinking of is maybe exhausting memory with so many groups
in one section, but I can't check that until I'm done.
What is different between our setups? I don't know right now.
Could you please delete <spooldir>/de.soc.misc-overboard.dat please.
This will delete the overview for that group, which will be recreated
when someone clicks to view it. I'd like to see if the newer articles
are in it.
On 28.11.2024 20:08 Uhr Retro Guy wrote:
What is different between our setups? I don't know right now.
Dunno, but if you want I can give you a tarball of /etc/rslight and /var/spool/rslight.
Could you please delete <spooldir>/de.soc.misc-overboard.dat please.
This will delete the overview for that group, which will be recreated
when someone clicks to view it. I'd like to see if the newer articles
are in it.
Done.
On Fri, 29 Nov 2024 7:50:34 +0000, Marco Moock wrote:
On 28.11.2024 20:08 Uhr Retro Guy wrote:
What is different between our setups? I don't know right now.
Dunno, but if you want I can give you a tarball of /etc/rslight and /var/spool/rslight.
Could you please delete <spooldir>/de.soc.misc-overboard.dat
please. This will delete the overview for that group, which will
be recreated when someone clicks to view it. I'd like to see if
the newer articles are in it.
Done.
Are you having this trouble with any other groups besides de.soc.misc?
I see that overboard is still blank for the group. On my test site, it
is fine. Could you please send to me, <spooldir>/de.soc.misc-* (all
in a zip or tgz or whatever is fine)
On 29.11.2024 08:10 Uhr Retro Guy wrote:
On Fri, 29 Nov 2024 7:50:34 +0000, Marco Moock wrote:
On 28.11.2024 20:08 Uhr Retro Guy wrote:
What is different between our setups? I don't know right now.
Dunno, but if you want I can give you a tarball of /etc/rslight and
/var/spool/rslight.
Could you please delete <spooldir>/de.soc.misc-overboard.dat
please. This will delete the overview for that group, which will
be recreated when someone clicks to view it. I'd like to see if
the newer articles are in it.
Done.
Are you having this trouble with any other groups besides de.soc.misc?
Haven't seen yet.
I see that overboard is still blank for the group. On my test site, it
is fine. Could you please send to me, <spooldir>/de.soc.misc-* (all
in a zip or tgz or whatever is fine)
After running maintenance, it looks ok and the group is currently empty.
Deleting <ffa5d4e0-e@cbd-4770-9304-9e740e53ff5d> from de.soc.misc
Deleting <ffae661d-9@d10-426d-97c2-943704a37856> from de.soc.misc
Deleting <ffcdf4f6-7@411-4e08-95db-eef9d7f0701f> from de.soc.misc
Deleting <ffdcfc4a-7@468-4297-8270-7867962a0e71> from de.soc.misc
Current group pointer for de.soc.misc: 24809
New group pointer for de.soc.misc: 1
root@pi-dach:~# sudo -u www-data php
/etc/rslight/scripts/maintenance.php -reset de.soc.misc
You are running as user: www-data
Starting Import...
Reset: de.soc.misc
Current group pointer for de.soc.misc: 1
New group pointer for de.soc.misc: 1
root@pi-dach:~#
Now polling:
ds for: de.soc.kultur.misc...
Nov 29 09:45:40 rocksolid Threads updated for: de.soc.kultur.misc
Nov 29 09:45:40 rocksolid CLOSING Local server: Resource id #6314
Nov 29 09:45:40 rocksolid Retrieving articles for: de.soc.misc...
Nov 29 09:45:40 rocksolid news.i2pn2.org: de.soc.misc: 211 20563 53
24808 de.soc.misc
Nov 29 09:45:41 rocksolid news.i2pn2.org de.soc.misc (requested:
overview 24708-24808 received 224 No articles in 24708-24808)
Nov 29 09:45:41 rocksolid de.soc.misc 423 No such article number 24708
Nov 29 09:45:41 rocksolid de.soc.misc 423 No such article number 24709
Nov 29 09:45:41 rocksolid de.soc.misc 423 No such article number 24710
Nov 29 09:45:41 rocksolid de.soc.misc 423 No such article number 24711
Nov 29 09:45:41 rocksolid de.soc.misc 423 No such article number 24712
Nov 29 09:45:41 rocksolid de.soc.misc 423 No such article number 24713
Nov 29 09:45:41 rocksolid de.soc.misc 423 No such article number 24714
Nov 29 09:45:41 rocksolid de.soc.misc 423 No such article number 24715
Nov 29 09:45:42 rocksolid de.soc.misc 423 No such article number 24716
Nov 29 09:45:42 rocksolid de.soc.misc 423 No such article number 24717
Nov 29 09:45:42 rocksolid de.soc.misc 423 No such article number 24718
Nov 29 09:45:42 rocksolid de.soc.misc 423 No such article
[...]
Nov 29 09:45:51 rocksolid de.soc.misc 423 No such article number 24807
Nov 29 09:45:51 rocksolid de.soc.misc 423 No such article number 24808
Nov 29 09:45:51 rocksolid OPENING Local server: Resource id #6428
Nov 29 09:45:51 rocksolid Updating threads for: de.soc.misc...
Nov 29 09:45:51 rocksolid Threads updated for: de.soc.misc
Nov 29 09:45:51 rocksolid CLOSING Local server: Resource id #6428
Now stops again.
On Fri, 29 Nov 2024 9:57:59 +0000, Marco Moock wrote:<snip>
On 29.11.2024 08:10 Uhr Retro Guy wrote:
On Fri, 29 Nov 2024 7:50:34 +0000, Marco Moock wrote:
On 28.11.2024 20:08 Uhr Retro Guy wrote:
What is different between our setups? I don't know right now.
Dunno, but if you want I can give you a tarball of /etc/rslight and
/var/spool/rslight.
Could you please delete <spooldir>/de.soc.misc-overboard.dat
please. This will delete the overview for that group, which will
be recreated when someone clicks to view it. I'd like to see if
the newer articles are in it.
Done.
Are you having this trouble with any other groups besides de.soc.misc?
Haven't seen yet.
I see that overboard is still blank for the group. On my test site, it
is fine. Could you please send to me, <spooldir>/de.soc.misc-* (all
in a zip or tgz or whatever is fine)
After running maintenance, it looks ok and the group is currently empty.
Deleting <ffa5d4e0-e@cbd-4770-9304-9e740e53ff5d> from de.soc.misc
Deleting <ffae661d-9@d10-426d-97c2-943704a37856> from de.soc.misc
Deleting <ffcdf4f6-7@411-4e08-95db-eef9d7f0701f> from de.soc.misc
Deleting <ffdcfc4a-7@468-4297-8270-7867962a0e71> from de.soc.misc
Current group pointer for de.soc.misc: 24809
New group pointer for de.soc.misc: 1
root@pi-dach:~# sudo -u www-data php
/etc/rslight/scripts/maintenance.php -reset de.soc.misc
You are running as user: www-data
Starting Import...
Reset: de.soc.misc
Current group pointer for de.soc.misc: 1
New group pointer for de.soc.misc: 1
root@pi-dach:~#
Now polling:
Yes, just so the answer is posted here also (not just email), you need
to change the settings in overrides.php to go back more articles:
// spoolnews.php
'maxarticles_per_run' => 1000,
'maxfirstrequest' => 1000,
Change these to higher numbers. Looks like you had '100'. Since this is
a group full of bot spam, a lot of articles are missing after NoCeM.
On Fri, 29 Nov 2024 11:08:31 +0000, Retro Guy wrote:
On Fri, 29 Nov 2024 9:57:59 +0000, Marco Moock wrote:<snip>
On 29.11.2024 08:10 Uhr Retro Guy wrote:
On Fri, 29 Nov 2024 7:50:34 +0000, Marco Moock wrote:
On 28.11.2024 20:08 Uhr Retro Guy wrote:
What is different between our setups? I don't know right now.
Dunno, but if you want I can give you a tarball of /etc/rslight and
/var/spool/rslight.
Could you please delete <spooldir>/de.soc.misc-overboard.dat
please. This will delete the overview for that group, which will
be recreated when someone clicks to view it. I'd like to see if
the newer articles are in it.
Done.
Are you having this trouble with any other groups besides de.soc.misc?
Haven't seen yet.
I see that overboard is still blank for the group. On my test site, it >>>> is fine. Could you please send to me, <spooldir>/de.soc.misc-* (all
in a zip or tgz or whatever is fine)
After running maintenance, it looks ok and the group is currently empty. >>>
Deleting <ffa5d4e0-e@cbd-4770-9304-9e740e53ff5d> from de.soc.misc
Deleting <ffae661d-9@d10-426d-97c2-943704a37856> from de.soc.misc
Deleting <ffcdf4f6-7@411-4e08-95db-eef9d7f0701f> from de.soc.misc
Deleting <ffdcfc4a-7@468-4297-8270-7867962a0e71> from de.soc.misc
Current group pointer for de.soc.misc: 24809
New group pointer for de.soc.misc: 1
root@pi-dach:~# sudo -u www-data php
/etc/rslight/scripts/maintenance.php -reset de.soc.misc
You are running as user: www-data
Starting Import...
Reset: de.soc.misc
Current group pointer for de.soc.misc: 1
New group pointer for de.soc.misc: 1
root@pi-dach:~#
Now polling:
Yes, just so the answer is posted here also (not just email), you need
to change the settings in overrides.php to go back more articles:
// spoolnews.php
'maxarticles_per_run' => 1000,
'maxfirstrequest' => 1000,
Change these to higher numbers. Looks like you had '100'. Since this is
a group full of bot spam, a lot of articles are missing after NoCeM.
The suggestion to check what articles exist on server first is a good
one, and I now see that 'listgroup' is the answer (not xover).
I'll be working on this soon and it will hopefully get spoolnews.php
running much more accurately and cleanly.
On Fri, 29 Nov 2024 11:33:13 +0000, Retro Guy wrote:
On Fri, 29 Nov 2024 11:08:31 +0000, Retro Guy wrote:
On Fri, 29 Nov 2024 9:57:59 +0000, Marco Moock wrote:<snip>
On 29.11.2024 08:10 Uhr Retro Guy wrote:
On Fri, 29 Nov 2024 7:50:34 +0000, Marco Moock wrote:Haven't seen yet.
On 28.11.2024 20:08 Uhr Retro Guy wrote:
What is different between our setups? I don't know right now.
Dunno, but if you want I can give you a tarball of /etc/rslight and >>>>>> /var/spool/rslight.
Could you please delete <spooldir>/de.soc.misc-overboard.dat
please. This will delete the overview for that group, which will >>>>>>> be recreated when someone clicks to view it. I'd like to see if
the newer articles are in it.
Done.
Are you having this trouble with any other groups besides de.soc.misc? >>>>
I see that overboard is still blank for the group. On my test site, it >>>>> is fine. Could you please send to me, <spooldir>/de.soc.misc-* (all
in a zip or tgz or whatever is fine)
After running maintenance, it looks ok and the group is currently empty. >>>>
Deleting <ffa5d4e0-e@cbd-4770-9304-9e740e53ff5d> from de.soc.misc
Deleting <ffae661d-9@d10-426d-97c2-943704a37856> from de.soc.misc
Deleting <ffcdf4f6-7@411-4e08-95db-eef9d7f0701f> from de.soc.misc
Deleting <ffdcfc4a-7@468-4297-8270-7867962a0e71> from de.soc.misc
Current group pointer for de.soc.misc: 24809
New group pointer for de.soc.misc: 1
root@pi-dach:~# sudo -u www-data php
/etc/rslight/scripts/maintenance.php -reset de.soc.misc
You are running as user: www-data
Starting Import...
Reset: de.soc.misc
Current group pointer for de.soc.misc: 1
New group pointer for de.soc.misc: 1
root@pi-dach:~#
Now polling:
Yes, just so the answer is posted here also (not just email), you need
to change the settings in overrides.php to go back more articles:
// spoolnews.php
'maxarticles_per_run' => 1000,
'maxfirstrequest' => 1000,
Change these to higher numbers. Looks like you had '100'. Since this is
a group full of bot spam, a lot of articles are missing after NoCeM.
The suggestion to check what articles exist on server first is a good
one, and I now see that 'listgroup' is the answer (not xover).
I'll be working on this soon and it will hopefully get spoolnews.php
running much more accurately and cleanly.
I have it working now. Just pulled exactly 100 existing articles from
remote when 'maxfirstrequest' set to 100.
I'll test for a while, then release.