From Newsgroup: news.software.nntp

I'm observing what feels like a bug in the calculation of "accepted size" in innd's checkpoint log lines with long-running connections. This may also be happening with status log lines, but I haven't focused on those.

In the process of feeding >1 billion articles to a new server. Both servers
are running INN and I'm using innxmit on the source machine. After starting innxmit, initially the calculation of the accepted size in the checkpoint log lines seems correct, but over time it seems to miscalculate.

With incominglogfrequency set to 100,000, initially each checkpoint reports accepted size around 220-280MB. Which, by my rough calculations, seems
correct. However, over time (8+ hours), I begin to see the accepted size fall dramatically to around 80-110MB for the same amount of articles.

I understand there is going to be fluctuation in the amount of data received per 100,000 articles, but if I stop the innxmit process and start it again, immediately the checkpoints start reporting back in the original range. Then after some time it begins to fall off.

I see this pattern repeat each time I stop/start innxmit (I have removed a lot of output for brevity):

% grep checkpoint log/news.notice | awk '{ print $23 }'
254853120
244031488
240500736
222146560
245587968
221945856
107102208 <--- begins to fall off dramatically
113074176
113967104
98664448
99794944
94420992
89972736
128991232
111230976
80273408
276527360 <--- stopped/started innxmit
254255360
271361536
253618304
271003648
256765952

Best I can tell through watching the CNFS buffer, filesystem, and network metrics, I'm always receiving about the same rate/amount of data throughout
the connection lifetime. I do not personally know how to prove this out further, so I'm throwing it out to the masses.
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

Hi Jesse,

I'm observing what feels like a bug in the calculation of "accepted size" in innd's checkpoint log lines with long-running connections. This may also be happening with status log lines, but I haven't focused on those.

In the process of feeding >1 billion articles to a new server. Both servers are running INN and I'm using innxmit on the source machine. After starting innxmit, initially the calculation of the accepted size in the checkpoint log lines seems correct, but over time it seems to miscalculate.

% grep checkpoint log/news.notice | awk '{ print $23 }'
254853120
244031488
240500736
222146560
245587968
221945856
107102208 <--- begins to fall off dramatically
113074176
113967104
98664448
99794944

[...]

Best I can tell through watching the CNFS buffer, filesystem, and network metrics, I'm always receiving about the same rate/amount of data throughout the connection lifetime.

After a quick glance at how innd computes the accepted size, I did not
find anything suspicious :(

I do not personally know how to prove this out
further, so I'm throwing it out to the masses.

Could you please add the -v flag to innxmit?
This way, it will write the accepted size of the articles it has sent,
from its point of view. It will be interesting to compare the values
between innd on your new server and innxmit on your source server.

It writes something like that when you stop it:

Dec 13 07:50:34 news innxmit[4066916]: news.hazardzone.fr stats offered
57805 accepted 57668 refused 6 rejected 99 missing 0 accsize 383582687
rejsize 1183910
Dec 13 07:50:34 news innxmit[4066916]: news.hazardzone.fr times user
2.449 system 11.016 elapsed 32855.176
--
Julien |eLIE

-2-aL'angle droit bout |a 90 degr|-s.-a-+

--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On Jul 17, 2026 at 2:32:21rC>AM CDT, "Julien |eLIE" <iulius@nom-de-mon-site.com.invalid> wrote:

Hi Jesse,

I'm observing what feels like a bug in the calculation of "accepted size" in >> innd's checkpoint log lines with long-running connections. This may also be >> happening with status log lines, but I haven't focused on those.

In the process of feeding >1 billion articles to a new server. Both servers >> are running INN and I'm using innxmit on the source machine. After starting >> innxmit, initially the calculation of the accepted size in the checkpoint log
lines seems correct, but over time it seems to miscalculate.

% grep checkpoint log/news.notice | awk '{ print $23 }'
254853120
244031488
240500736
222146560
245587968
221945856
107102208 <--- begins to fall off dramatically> 113074176
113967104
98664448
99794944

[...]

Best I can tell through watching the CNFS buffer, filesystem, and network
metrics, I'm always receiving about the same rate/amount of data throughout >> the connection lifetime.

After a quick glance at how innd computes the accepted size, I did not
find anything suspicious :(

I do not personally know how to prove this out
further, so I'm throwing it out to the masses.

Could you please add the -v flag to innxmit?
This way, it will write the accepted size of the articles it has sent,
from its point of view. It will be interesting to compare the values
between innd on your new server and innxmit on your source server.

It writes something like that when you stop it:

Dec 13 07:50:34 news innxmit[4066916]: news.hazardzone.fr stats offered
57805 accepted 57668 refused 6 rejected 99 missing 0 accsize 383582687 rejsize 1183910
Dec 13 07:50:34 news innxmit[4066916]: news.hazardzone.fr times user
2.449 system 11.016 elapsed 32855.176

I've added the -v flag to my latest run of innxmit. I assume, I'll compare the value from innxmit to innd's "closed" log line for that connection?

Trying to calculate all of the checkpoint lines from that connection seems daunting. I suppose this may be a good use case for AI, though. :-)
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

On Jul 17, 2026 at 2:32:21rC>AM CDT, "Julien |eLIE" <iulius@nom-de-mon-site.com.invalid> wrote:

Could you please add the -v flag to innxmit?
This way, it will write the accepted size of the articles it has sent,
from its point of view. It will be interesting to compare the values
between innd on your new server and innxmit on your source server.

It writes something like that when you stop it:

Dec 13 07:50:34 news innxmit[4066916]: news.hazardzone.fr stats offered
57805 accepted 57668 refused 6 rejected 99 missing 0 accsize 383582687 rejsize 1183910
Dec 13 07:50:34 news innxmit[4066916]: news.hazardzone.fr times user
2.449 system 11.016 elapsed 32855.176

There are discrepancies between innxmit and innd:

Jul 18 05:39:12 omnios-inn innxmit[14548]: [ID 738504 news.warning] 192.168.1.75 stats offered 27521180 accepted 27520730 refused 0 rejected 418 missing 0 accsize 87812383303 rejsize 696317
Jul 18 05:39:12 omnios-inn innxmit[14548]: [ID 452021 news.warning] 192.168.1.75 times user 476.884 system 3070.203 elapsed 39294.478

Jul 18 05:39:12 archive innd: [ID 702911 news.notice] 192.168.1.110:29 closed seconds 39295 accepted 27520762 refused 0 rejected 418 duplicate 32 accepted size 79449063424 duplicate size 99298 rejected size 696317
--- Synchronet 3.22a-Linux NewsLink 1.2

From Newsgroup: news.software.nntp

Hi Jesse,

There are discrepancies between innxmit and innd:

Jul 18 05:39:12 omnios-inn innxmit[14548]: [ID 738504 news.warning] 192.168.1.75 stats offered 27521180 accepted 27520730 refused 0 rejected 418 missing 0 accsize 87812383303 rejsize 696317
Jul 18 05:39:12 omnios-inn innxmit[14548]: [ID 452021 news.warning] 192.168.1.75 times user 476.884 system 3070.203 elapsed 39294.478

Jul 18 05:39:12 archive innd: [ID 702911 news.notice] 192.168.1.110:29 closed seconds 39295 accepted 27520762 refused 0 rejected 418 duplicate 32 accepted size 79449063424 duplicate size 99298 rejected size 696317

Thanks for the report.
innd sees about 8 GB less than innxmit indeed, for 27,520,730 articles.
It's worth investigating, hoping this is not due to how each of them
compute the length. They may have different Path and Xref header values
for instance, though the difference of about 300 bytes per article would
be too high for only that.
--
Julien |eLIE

-2-aMieux vaut allumer une bougie que maudire les t|-n|?bres.-a-+ (Lao Zi)

--- Synchronet 3.22a-Linux NewsLink 1.2