| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 23 |
| Nodes: | 6 (0 / 6) |
| Uptime: | 52:05:03 |
| Calls: | 583 |
| Files: | 1,139 |
| Messages: | 111,529 |
Hello!
I notice that on my instance some groups are note being updated on the
system itself.
The articles itself are on i2pn2.org.
Some other groups are updated, but it is not limited to one non-working group.
How can I further diagnose that?
On Wed, 13 Nov 2024 15:21:48 +0000, Marco Moock wrote:
Hello!
I notice that on my instance some groups are note being updated on
the system itself.
The articles itself are on i2pn2.org.
Some other groups are updated, but it is not limited to one
non-working group.
How can I further diagnose that?
Well that's not good :(
Can you check if the articles exist when connecting directly to the
rslight nntp server?
Are the groups that fail ALL groups below a certain line in
groups.txt?
Can you grep a message-id of a failed article in spoolnews.log and see
if there is a log entry?
How is your disk space?
On 13.11.2024 um 15:51 Uhr Retro Guy wrote:
On Wed, 13 Nov 2024 15:21:48 +0000, Marco Moock wrote:
Hello!
I notice that on my instance some groups are note being updated on
the system itself.
The articles itself are on i2pn2.org.
Some other groups are updated, but it is not limited to one
non-working group.
How can I further diagnose that?
Well that's not good :(
Can you check if the articles exist when connecting directly to the
rslight nntp server?
The internal server doesn't have the article.
Are the groups that fail ALL groups below a certain line in
groups.txt?
Looks a bit like, because the rocksolid stuff is working, but at least
2 groups in de.* are/were also working.
On Wed, 13 Nov 2024 20:28:15 +0100, Marco Moock wrote:
On 13.11.2024 um 15:51 Uhr Retro Guy wrote:
On Wed, 13 Nov 2024 15:21:48 +0000, Marco Moock wrote:
Hello!
I notice that on my instance some groups are note being updated on
the system itself.
The articles itself are on i2pn2.org.
Some other groups are updated, but it is not limited to one
non-working group.
How can I further diagnose that?
Well that's not good :(
Can you check if the articles exist when connecting directly to the
rslight nntp server?
The internal server doesn't have the article.
Are the groups that fail ALL groups below a certain line in
groups.txt?
Looks a bit like, because the rocksolid stuff is working, but at least
2 groups in de.* are/were also working.
Can you check spoolnews.log and see if it stops spooling groups after the last group that works. Maybe an error like can't connect to remote server
or such?
Once it gets to the groups that do not work, if it ever does, are there errors in spoolnews about those groups?
I'm seeing a lot of:
<your server> can't read: Connection reset by peer
Which only started today at:
Nov 13 04:25:25
Any changes made at that time?
Maybe it's my end, but no such logs from my rslight servers, not that it means it's not i2pn2 problem.
On Wed, 13 Nov 2024 13:42:49 -0700, Retro Guy wrote:
On Wed, 13 Nov 2024 20:28:15 +0100, Marco Moock wrote:
On 13.11.2024 um 15:51 Uhr Retro Guy wrote:
On Wed, 13 Nov 2024 15:21:48 +0000, Marco Moock wrote:
Hello!
I notice that on my instance some groups are note being updated on
the system itself.
The articles itself are on i2pn2.org.
Some other groups are updated, but it is not limited to one
non-working group.
How can I further diagnose that?
Well that's not good :(
Can you check if the articles exist when connecting directly to the
rslight nntp server?
The internal server doesn't have the article.
Are the groups that fail ALL groups below a certain line in
groups.txt?
Looks a bit like, because the rocksolid stuff is working, but at least
2 groups in de.* are/were also working.
Can you check spoolnews.log and see if it stops spooling groups after the
last group that works. Maybe an error like can't connect to remote server
or such?
Once it gets to the groups that do not work, if it ever does, are there
errors in spoolnews about those groups?
I'm seeing a lot of:
<your server> can't read: Connection reset by peer
Which only started today at:
Nov 13 04:25:25
Any changes made at that time?
Maybe it's my end, but no such logs from my rslight servers, not that it
means it's not i2pn2 problem.
Also, try i2pn2.org port 8991
On Wed, 13 Nov 2024 13:43:58 -0700, Retro Guy wrote:
On Wed, 13 Nov 2024 13:42:49 -0700, Retro Guy wrote:
On Wed, 13 Nov 2024 20:28:15 +0100, Marco Moock wrote:
On 13.11.2024 um 15:51 Uhr Retro Guy wrote:
On Wed, 13 Nov 2024 15:21:48 +0000, Marco Moock wrote:
Hello!
I notice that on my instance some groups are note being updated on >>>>>> the system itself.
The articles itself are on i2pn2.org.
Some other groups are updated, but it is not limited to one
non-working group.
How can I further diagnose that?
Well that's not good :(
Can you check if the articles exist when connecting directly to the
rslight nntp server?
The internal server doesn't have the article.
Are the groups that fail ALL groups below a certain line in
groups.txt?
Looks a bit like, because the rocksolid stuff is working, but at least >>>> 2 groups in de.* are/were also working.
Can you check spoolnews.log and see if it stops spooling groups after the >>> last group that works. Maybe an error like can't connect to remote server >>> or such?
Once it gets to the groups that do not work, if it ever does, are there
errors in spoolnews about those groups?
I'm seeing a lot of:
<your server> can't read: Connection reset by peer
Which only started today at:
Nov 13 04:25:25
Any changes made at that time?
Maybe it's my end, but no such logs from my rslight servers, not that it >>> means it's not i2pn2 problem.
Also, try i2pn2.org port 8991
If you don't mind, could you email me some files please?
I'm looking for spoolnews.log, debug.log and, in <spooldir>/rocksolid there should be a file <server>:<port>-remote_groups.dat
The last file will show the last article received for each group from the server.
I did just realize that all my sites pull from i2pn2 via an internal
network, not i2pn2.org but internal ip addresses at the hosting site. Maybe there's a packet loss issue at their (hosting) end? I don't really know
yet.
On Wed, 13 Nov 2024 15:17:59 -0700, Retro Guy wrote:
On Wed, 13 Nov 2024 13:43:58 -0700, Retro Guy wrote:
On Wed, 13 Nov 2024 13:42:49 -0700, Retro Guy wrote:
On Wed, 13 Nov 2024 20:28:15 +0100, Marco Moock wrote:
On 13.11.2024 um 15:51 Uhr Retro Guy wrote:
On Wed, 13 Nov 2024 15:21:48 +0000, Marco Moock wrote:
Hello!
I notice that on my instance some groups are note being updated on >>>>>>> the system itself.
The articles itself are on i2pn2.org.
Some other groups are updated, but it is not limited to one
non-working group.
How can I further diagnose that?
Well that's not good :(
Can you check if the articles exist when connecting directly to the >>>>>> rslight nntp server?
The internal server doesn't have the article.
Are the groups that fail ALL groups below a certain line in
groups.txt?
Looks a bit like, because the rocksolid stuff is working, but at least >>>>> 2 groups in de.* are/were also working.
Can you check spoolnews.log and see if it stops spooling groups after
the
last group that works. Maybe an error like can't connect to remote
server
or such?
Once it gets to the groups that do not work, if it ever does, are there >>>> errors in spoolnews about those groups?
I'm seeing a lot of:
<your server> can't read: Connection reset by peer
Which only started today at:
Nov 13 04:25:25
Any changes made at that time?
Maybe it's my end, but no such logs from my rslight servers, not that it >>>> means it's not i2pn2 problem.
Also, try i2pn2.org port 8991
If you don't mind, could you email me some files please?
I'm looking for spoolnews.log, debug.log and, in <spooldir>/rocksolid
there
should be a file <server>:<port>-remote_groups.dat
The last file will show the last article received for each group from
the
server.
I did just realize that all my sites pull from i2pn2 via an internal
network, not i2pn2.org but internal ip addresses at the hosting site.
Maybe
there's a packet loss issue at their (hosting) end? I don't really know
yet.
I've modified spoolnews.php to try to isolate such issues, and add
logging
to find exactly when/where it happens.
If we don't find a simple answer, I'll ask that you apply the new spoolnews.php, which I will need to send you, to test further. It is not
yet in devel on gitlab, so no sense downloading what is there.
This change, and everything else from devel is now in master on
gitlab: https://gitlab.com/rslight-public/rocksolid-light
This version is live on all my sites now.
If there are further issues with this, we will address them asap.
On 14.11.2024 um 15:12 Uhr Retro Guy wrote:
This change, and everything else from devel is now in master on
gitlab: https://gitlab.com/rslight-public/rocksolid-light
This version is live on all my sites now.
If there are further issues with this, we will address them asap.
Articles are now being downloaded and spooled, but in the web I get no
such group for all groups.
The group list is still there, but everything else isn't accessible or
gone.
Please try restarting your nntp server:
touch /etc/rslight/nntp.reload
Then wait until cron.php runs again and see if that fixes it. It may
be simply an update to rslight-lib.php causing the issue.
On 14.11.2024 um 16:28 Uhr Retro Guy wrote:
Please try restarting your nntp server:
touch /etc/rslight/nntp.reload
Then wait until cron.php runs again and see if that fixes it. It may
be simply an update to rslight-lib.php causing the issue.
I did that, but still nothing in the webserver.
How can I further investigate that?
On Thu, 14 Nov 2024 17:52:26 +0100
Marco Moock <mm+usenet-es@dorfdsl.de> wrote:
On 14.11.2024 um 16:28 Uhr Retro Guy wrote:
Please try restarting your nntp server:
touch /etc/rslight/nntp.reload
Then wait until cron.php runs again and see if that fixes it. It may
be simply an update to rslight-lib.php causing the issue.
I did that, but still nothing in the webserver.
How can I further investigate that?
Wow, I really broke it!
I need an hour or so to fix this. I can duplicate the issue here on a test site.
I will be working on this right now!
On Thu, 14 Nov 2024 17:52:26 +0100
Marco Moock <mm+usenet-es@dorfdsl.de> wrote:
On 14.11.2024 um 16:28 Uhr Retro Guy wrote:
Please try restarting your nntp server:
touch /etc/rslight/nntp.reload
Then wait until cron.php runs again and see if that fixes it. It may
be simply an update to rslight-lib.php causing the issue.
I did that, but still nothing in the webserver.
How can I further investigate that?
Wow, I really broke it!
I need an hour or so to fix this. I can duplicate the issue here on a test site.
I will be working on this right now!
I have just emailed to you a new copy of newsportal.php and some instructions.
On Thu, 14 Nov 2024 17:52:26 +0100
Marco Moock <mm+usenet-es@dorfdsl.de> wrote:
On 14.11.2024 um 16:28 Uhr Retro Guy wrote:
Please try restarting your nntp server:
touch /etc/rslight/nntp.reload
Then wait until cron.php runs again and see if that fixes it. It may
be simply an update to rslight-lib.php causing the issue.
I did that, but still nothing in the webserver.
How can I further investigate that?
Wow, I really broke it!
I need an hour or so to fix this. I can duplicate the issue here on a test site.
I will be working on this right now!
On 14.11.2024 um 10:41 Uhr Retro Guy wrote:
I have just emailed to you a new copy of newsportal.php and some
instructions.
This email got rejected because the outgoing server of your provider is listed in uceprotect level 1 (it sent enough spam to spamtraps).
I've now whitelisted the IP 104.245.209.246.
Please relay mail to me directly if possible to avoid that and tell it
the provider, so they cut off the spammer.
On Thu, 14 Nov 2024 17:41:05 +0000, Retro Guy wrote:
On Thu, 14 Nov 2024 10:05:56 -0700
Retro Guy <retroguy@i2pn2.org> wrote:
On Thu, 14 Nov 2024 17:52:26 +0100
Marco Moock <mm+usenet-es@dorfdsl.de> wrote:
On 14.11.2024 um 16:28 Uhr Retro Guy wrote:
Please try restarting your nntp server:
touch /etc/rslight/nntp.reload
Then wait until cron.php runs again and see if that fixes it. It may >>>> > be simply an update to rslight-lib.php causing the issue.
I did that, but still nothing in the webserver.
How can I further investigate that?
Wow, I really broke it!
I need an hour or so to fix this. I can duplicate the issue here on a
test site.
I will be working on this right now!
I have just emailed to you a new copy of newsportal.php and some
instructions.
Please let me know what happens. It works on my test site (of course it
does)
I have pushed the change I emailed you about to master, along with a
change to spoolnews.php.
On Thu, 14 Nov 2024 10:05:56 -0700
Retro Guy <retroguy@i2pn2.org> wrote:
On Thu, 14 Nov 2024 17:52:26 +0100
Marco Moock <mm+usenet-es@dorfdsl.de> wrote:
On 14.11.2024 um 16:28 Uhr Retro Guy wrote:
Please try restarting your nntp server:
touch /etc/rslight/nntp.reload
Then wait until cron.php runs again and see if that fixes it. It may
be simply an update to rslight-lib.php causing the issue.
I did that, but still nothing in the webserver.
How can I further investigate that?
Wow, I really broke it!
I need an hour or so to fix this. I can duplicate the issue here on a
test site.
I will be working on this right now!
I have just emailed to you a new copy of newsportal.php and some instructions.
Please let me know what happens. It works on my test site (of course it
does)
I have pushed the change I emailed you about to master, along with a
change to spoolnews.php.
On 14.11.2024 um 19:01 Uhr Retro Guy wrote:
I have pushed the change I emailed you about to master, along with a
change to spoolnews.php.
It fixed the missing groups.
Some were rebuild according to the message in the webserver, but now
work.
Although, I noticed that it didn't poll all articles from some threads,
e.g. https://pi-dach.dorfdsl.de/rocksolid/article-flat.php?id=2472&group=de.comp.os.unix.linux.misc#2472
https://pi-dach.dorfdsl.de/rocksolid/article-flat.php?id=2481&group=de.comp.os.unix.linux.misc#2481
Some messages are still missing from the web interface at all (but are
on i2pn2.org) and are in the log:
/var/spool/rslight/log/debug.log:Nov 14 15:44:17 rocksolid Used Injection-Date 1731592628 for: <vh4vjk$k3tg$1@gwaiyur.mb-net.net> /var/spool/rslight/log/spoolnews.log:Nov 14 15:44:17 rocksolid Spooling: de.comp.os.unix.linux.misc:76 <vh4vjk$k3tg$1@gwaiyur.mb-net.net>
<vh4vjk$k3tg$1@gwaiyur.mb-net.net> <vguh8f$bm0$1@akk3-dmz.akk.uni-karlsruhe.de>
On Thu, 14 Nov 2024 19:48:37 +0000, Marco Moock wrote:
On 14.11.2024 um 19:01 Uhr Retro Guy wrote:
I have pushed the change I emailed you about to master, along with a
change to spoolnews.php.
It fixed the missing groups.
Some were rebuild according to the message in the webserver, but now
work.
Although, I noticed that it didn't poll all articles from some threads,
e.g.
https://pi-dach.dorfdsl.de/rocksolid/article-flat.php?id=2472&group=de.comp.os.unix.linux.misc#2472
https://pi-dach.dorfdsl.de/rocksolid/article-flat.php?id=2481&group=de.comp.os.unix.linux.misc#2481
Some messages are still missing from the web interface at all (but are
on i2pn2.org) and are in the log:
/var/spool/rslight/log/debug.log:Nov 14 15:44:17 rocksolid Used
Injection-Date 1731592628 for: <vh4vjk$k3tg$1@gwaiyur.mb-net.net>
/var/spool/rslight/log/spoolnews.log:Nov 14 15:44:17 rocksolid Spooling:
de.comp.os.unix.linux.misc:76 <vh4vjk$k3tg$1@gwaiyur.mb-net.net>
<vh4vjk$k3tg$1@gwaiyur.mb-net.net>
<vguh8f$bm0$1@akk3-dmz.akk.uni-karlsruhe.de>
Yes, I see the same issue also. I'm working on it but it might be the
case that we lose a few articles, which is unfortunate.
Please let it run for a while to allow thread_load_newsserver() to run
(via cron), which only runs a few times per hour. It MIGHT fix this, but
it may not also.
There is a way to force it, but it's time consuming. If you have a
critical group missing critical articles, you can rebuild the entire
group with (run as root or www user):
php /etc/rslight/scripts/maintenance -reset <group name>
This will erase ALL the articles and poll them again according to the quantity you wish to go back from the server in overrides.inc.php:
'maxfirstrequest' => 100, (use a number here for how many articles to go
back from newest on server)
On Thu, 14 Nov 2024 20:00:18 +0000, Retro Guy wrote:
On Thu, 14 Nov 2024 19:48:37 +0000, Marco Moock wrote:
On 14.11.2024 um 19:01 Uhr Retro Guy wrote:
I have pushed the change I emailed you about to master, along with a
change to spoolnews.php.
It fixed the missing groups.
Some were rebuild according to the message in the webserver, but now
work.
Although, I noticed that it didn't poll all articles from some threads,
e.g.
https://pi-dach.dorfdsl.de/rocksolid/article-flat.php?id=2472&group=de.comp.os.unix.linux.misc#2472
https://pi-dach.dorfdsl.de/rocksolid/article-flat.php?id=2481&group=de.comp.os.unix.linux.misc#2481
Some messages are still missing from the web interface at all (but are
on i2pn2.org) and are in the log:
/var/spool/rslight/log/debug.log:Nov 14 15:44:17 rocksolid Used
Injection-Date 1731592628 for: <vh4vjk$k3tg$1@gwaiyur.mb-net.net>
/var/spool/rslight/log/spoolnews.log:Nov 14 15:44:17 rocksolid Spooling: >>> de.comp.os.unix.linux.misc:76 <vh4vjk$k3tg$1@gwaiyur.mb-net.net>
<vh4vjk$k3tg$1@gwaiyur.mb-net.net>
<vguh8f$bm0$1@akk3-dmz.akk.uni-karlsruhe.de>
Yes, I see the same issue also. I'm working on it but it might be the
case that we lose a few articles, which is unfortunate.
Please let it run for a while to allow thread_load_newsserver() to run
(via cron), which only runs a few times per hour. It MIGHT fix this, but
it may not also.
There is a way to force it, but it's time consuming. If you have a
critical group missing critical articles, you can rebuild the entire
group with (run as root or www user):
php /etc/rslight/scripts/maintenance -reset <group name>
This will erase ALL the articles and poll them again according to the
quantity you wish to go back from the server in overrides.inc.php:
'maxfirstrequest' => 100, (use a number here for how many articles to go
back from newest on server)
BTW, I just did the above on https://news.novabbs.org for rocksolid.nodes.help and it did work.
If you happen to be using caching, restart memcacheD, or if using files, clear out the disk cache directory, else it will continue to pull from
the cache.
I think I'll add a switch to maintenance.php to '-reset' all groups in
any given section, instead of needing to do them one at a time. Maybe tomorrow.
On Thu, 14 Nov 2024 22:56:10 +0000, Retro Guy wrote:
On Thu, 14 Nov 2024 20:00:18 +0000, Retro Guy wrote:
On Thu, 14 Nov 2024 19:48:37 +0000, Marco Moock wrote:
On 14.11.2024 um 19:01 Uhr Retro Guy wrote:
I have pushed the change I emailed you about to master, along with a >>>>> change to spoolnews.php.
It fixed the missing groups.
Some were rebuild according to the message in the webserver, but now
work.
Although, I noticed that it didn't poll all articles from some threads, >>>> e.g.
https://pi-dach.dorfdsl.de/rocksolid/article-flat.php?id=2472&group=de.comp.os.unix.linux.misc#2472
https://pi-dach.dorfdsl.de/rocksolid/article-flat.php?id=2481&group=de.comp.os.unix.linux.misc#2481
Some messages are still missing from the web interface at all (but are >>>> on i2pn2.org) and are in the log:
/var/spool/rslight/log/debug.log:Nov 14 15:44:17 rocksolid Used
Injection-Date 1731592628 for: <vh4vjk$k3tg$1@gwaiyur.mb-net.net>
/var/spool/rslight/log/spoolnews.log:Nov 14 15:44:17 rocksolid Spooling: >>>> de.comp.os.unix.linux.misc:76 <vh4vjk$k3tg$1@gwaiyur.mb-net.net>
<vh4vjk$k3tg$1@gwaiyur.mb-net.net>
<vguh8f$bm0$1@akk3-dmz.akk.uni-karlsruhe.de>
Yes, I see the same issue also. I'm working on it but it might be the
case that we lose a few articles, which is unfortunate.
Please let it run for a while to allow thread_load_newsserver() to run
(via cron), which only runs a few times per hour. It MIGHT fix this, but >>> it may not also.
There is a way to force it, but it's time consuming. If you have a
critical group missing critical articles, you can rebuild the entire
group with (run as root or www user):
php /etc/rslight/scripts/maintenance -reset <group name>
This will erase ALL the articles and poll them again according to the
quantity you wish to go back from the server in overrides.inc.php:
'maxfirstrequest' => 100, (use a number here for how many articles to go >>> back from newest on server)
BTW, I just did the above on https://news.novabbs.org for
rocksolid.nodes.help and it did work.
If you happen to be using caching, restart memcacheD, or if using files,
clear out the disk cache directory, else it will continue to pull from
the cache.
I think I'll add a switch to maintenance.php to '-reset' all groups in
any given section, instead of needing to do them one at a time. Maybe
tomorrow.
Ok, this is done. It's in devel on gitlab.
The only file that is needed to be updated is <config_dir>/scripts/maintenance.php
It will reset ALL groups in an entire section. It will take quite a
while!
If you choose to use it:
1. Check your settings in <config_dir>/overrides.php:
// spoolnews.php
'maxarticles_per_run' => 100,
'maxfirstrequest' => 500,
'maxarticles_per_run' is how many articles to retrieve each run of
cron.php
'maxfirstrequest' is how far back to start, or how many articles to
subtract from the remote server's highest article number for the group
as a starting point.
2. Disable cron.php
3. Run php <config_dir>/scripts/maintenance.php -reset-section <section
name>
4. When it's eventually done, restart cron.php.
5. WAIT. It will take some time to retrieve the articles from the remote server, and then it will take some more time for rslight to rebuild it's number of articles for each group to display in the main page.
I did this on a test site and all went well.
BUT IT Will TAKE TIME.
There is a way to force it, but it's time consuming. If you have a
critical group missing critical articles, you can rebuild the entire
group with (run as root or www user):
php /etc/rslight/scripts/maintenance -reset <group name>
This will erase ALL the articles and poll them again according to the quantity you wish to go back from the server in overrides.inc.php:
On 14.11.2024 um 20:00 Uhr Retro Guy wrote:
There is a way to force it, but it's time consuming. If you have a
critical group missing critical articles, you can rebuild the entire
group with (run as root or www user):
php /etc/rslight/scripts/maintenance -reset <group name>
This will erase ALL the articles and poll them again according to the
quantity you wish to go back from the server in overrides.inc.php:
PHP Fatal error: Uncaught PDOException: SQLSTATE[HY000]: General error: 5 database is locked in /var/www/html/rocksolid/newsportal.php:3202
Stack trace:
#0 /var/www/html/rocksolid/newsportal.php(3202): PDOStatement->execute()
#1 /etc/rslight/scripts/maintenance.php(293): delete_message()
#2 /etc/rslight/scripts/maintenance.php(74): remove_articles()
#3 {main}
thrown in /var/www/html/rocksolid/newsportal.php on line 3202
I've shut down the cronjob systemd stuff during the run.
Is there another way to remove the group entirely and the restart from
the beginning for that group?
Or even all articles/groups, without removing settings etc.?
I would just try again, else use sqlite3.
DELETE FROM overview WHERE newsgroup=:group
DELETE FROM history WHERE newsgroup=:group
On 16.11.2024 um 12:11 Uhr Retro Guy wrote:
I would just try again, else use sqlite3.
DELETE FROM overview WHERE newsgroup=:group
DELETE FROM history WHERE newsgroup=:group
history doesn't exist, overview worked.
This worked.
I now ran the cronjob, but no articles in the overview yet, even when
they were downloaded. Did I miss something?
What group are you working on? I'm on your site and would like to see
if it's there or working or whatever it's doing.
On 16.11.2024 um 14:24 Uhr Retro Guy wrote:
What group are you working on? I'm on your site and would like to see
if it's there or working or whatever it's doing.
It was de.comp.os.linux.misc, but now one article got downloaded, so it
seems to be recovered.