• Re: 2026.xml has reached the 1048576 limit size

    From Abdelkader Boudih@seuros@freebsd.org to muc.lists.freebsd.ports on Sat Sep 26 11:30:20 2026
    From Newsgroup: muc.lists.freebsd.ports

    --04d9a64d4a6a296517b64cbf95e0675626378e97
    Content-Type: text/plain
    Content-Transfer-Encoding: 7bit

    I think we should split the files once they reach certain level of entries (not size).

    2024.xml
    2025.xml
    2026.00.xml
    2026.xml
    2027.00.xml
    2027.01.xml
    2027.xml

    the year without the prefix still the current one before it get rotated. With amount of new software and audits, i expect this to rotate few time per year.

    Regards
    Abdelkader

    On Sat, 26 Sep 2026, at 11:13, Rodrigo Osorio wrote:
    Hi,

    For the records, security/vuxml/vuln/2026.xml has reached the 1048576
    limit size,
    requiring the use of --push-option=i-know-this-file-is-big flag to
    commit it.
    And we still have 3 months left before 2027.

    Regards
    -- rodrigo



    --04d9a64d4a6a296517b64cbf95e0675626378e97
    Content-Type: text/html
    Content-Transfer-Encoding: quoted-printable

    <!DOCTYPE html><html><head><title></title></head><body><div>I think we s=
    hould split the files once they reach certain level of &nbsp;entries (no=
    t size).</div><div><br></div><div>2024.xml</div><div>2025.xml</div><div>= 2026.00.xml</div><div>2026.xml</div><div class=3D"align-start" style=3D"= text-align:start;">2027.00.xml</div><div class=3D"align-start" style=3D"= text-align:start;">2027.01.xml</div><div class=3D"align-start" style=3D"= text-align:start;">2027.xml</div><div class=3D"align-start" style=3D"tex= t-align:start;"><br></div><div class=3D"align-start" style=3D"text-align= :start;">the year without the prefix still the current one before it get=
    rotated. With amount of new software and audits, i expect this to rotat=
    e few time per year.&nbsp;</div><div class=3D"align-start" style=3D"text= -align:start;"><br></div><div class=3D"align-start" style=3D"text-align:= start;">Regards</div><div class=3D"align-start" style=3D"text-align:star= t;">Abdelkader</div><div class=3D"align-start" style=3D"text-align:start= ;"><br></div><div>On Sat, 26 Sep 2026, at 11:13, Rodrigo Osorio wrote:</= div><blockquote type=3D"cite" id=3D"qt" style=3D""><div>Hi,</div><div><b= r></div><div>For the records,&nbsp;security/vuxml/vuln/2026.xml has reac=
    hed the&nbsp;1048576&nbsp;</div><div>limit size,</div><div>requiring the=
    use of --push-option=3Di-know-this-file-is-big flag to&nbsp;</div><div>= commit it.</div><div>And we still have 3 months left before 2027.</div><= div><br></div><div>Regards</div><div>-- rodrigo</div><div><br></div><div= ><br></div><div><br></div></blockquote></body></html> --04d9a64d4a6a296517b64cbf95e0675626378e97--


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Robert Clausecker@fuz@fuz.ooo to muc.lists.freebsd.ports on Sat Sep 26 14:32:17 2026
    From Newsgroup: muc.lists.freebsd.ports

    A bit aside,

    I would honestly prefer if each port had its own vuln.xml file and we build the central file
    by means of "find ${PORTSDIR} -name vuln.xml -depth 2". This would avoid the size issue
    and entirely remove the crazy annoying merge conflicts I get basically every time I have to
    add a vuxml entry.

    Yours,
    Robert Clausecker

    On Sat, Sep 26, 2026 at 11:30:20AM +0000, Abdelkader Boudih wrote:
    I think we should split the files once they reach certain level of entries (not size).

    2024.xml
    2025.xml
    2026.00.xml
    2026.xml
    2027.00.xml
    2027.01.xml
    2027.xml

    the year without the prefix still the current one before it get rotated. With amount of new software and audits, i expect this to rotate few time per year.

    Regards
    Abdelkader

    On Sat, 26 Sep 2026, at 11:13, Rodrigo Osorio wrote:
    Hi,

    For the records, security/vuxml/vuln/2026.xml has reached the 1048576 limit size,
    requiring the use of --push-option=i-know-this-file-is-big flag to
    commit it.
    And we still have 3 months left before 2027.

    Regards
    -- rodrigo



    --
    () ascii ribbon campaign - for an encoding-agnostic world
    /\ - against html email - against proprietary attachments


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Jamie Landeg-Jones@jamie@catflap.org to muc.lists.freebsd.ports on Sat Sep 26 21:35:24 2026
    From Newsgroup: muc.lists.freebsd.ports

    "Abdelkader Boudih" <seuros@FreeBSD.org> wrote:

    I think we should split the files once they reach certain level of entries (not size).

    2024.xml
    2025.xml
    2026.00.xml
    2026.xml
    2027.00.xml
    2027.01.xml
    2027.xml

    the year without the prefix still the current one before it get rotated. With amount of new software and audits, i expect this to rotate few time per year.

    I don't like rotating file names for anything but log files. If
    you're going to split the files anyway, why not go with constant
    forms such as 2026Q1...2026Q4 or YYYY-MM?

    If you're worried about compatibility with users scripts, you could make 2026.xml a link or copy of the latest one - that would achieve the same
    level of compatibility as your suggestion.

    And as user scripts that want to access more than just the latest entries
    will need to be modified anyway, it will be much easier to change the
    scripts filename format, than trying to chase "sliding filenames'

    Cheers, Jamie



    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Dan Mahoney \(Ports\)@freebsd@gushi.org to muc.lists.freebsd.ports on Sat Sep 26 14:01:55 2026
    From Newsgroup: muc.lists.freebsd.ports

    ...thanks LLM's.
    -Dan
    On Sep 26, 2026, at 1:35rC>PM, Jamie Landeg-Jones <jamie@catflap.org> wrote:

    "Abdelkader Boudih" <seuros@FreeBSD.org> wrote:

    I think we should split the files once they reach certain level of entries (not size).

    2024.xml
    2025.xml
    2026.00.xml
    2026.xml
    2027.00.xml
    2027.01.xml
    2027.xml

    the year without the prefix still the current one before it get rotated. With amount of new software and audits, i expect this to rotate few time per year.

    I don't like rotating file names for anything but log files. If
    you're going to split the files anyway, why not go with constant
    forms such as 2026Q1...2026Q4 or YYYY-MM?

    If you're worried about compatibility with users scripts, you could make 2026.xml a link or copy of the latest one - that would achieve the same
    level of compatibility as your suggestion.

    And as user scripts that want to access more than just the latest entries will need to be modified anyway, it will be much easier to change the
    scripts filename format, than trying to chase "sliding filenames'

    Cheers, Jamie

    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Tomoaki AOKI@junchoon@dec.sakura.ne.jp to muc.lists.freebsd.ports on Sun Sep 27 09:17:36 2026
    From Newsgroup: muc.lists.freebsd.ports

    On Sat, 26 Sep 2026 14:32:17 +0000
    Robert Clausecker <fuz@fuz.ooo> wrote:

    A bit aside,

    I would honestly prefer if each port had its own vuln.xml file and we build the central file
    by means of "find ${PORTSDIR} -name vuln.xml -depth 2". This would avoid the size issue
    and entirely remove the crazy annoying merge conflicts I get basically every time I have to
    add a vuxml entry.

    It looks handy for each ports maintainers.

    But doesn't it mean that each entries appears randomly in
    date / time?

    Is there any handy tool to sort entries by date / time
    after merge? (I'm not enough familiar with this kind of tools
    for XML or any other "structured" texts.)


    And not to bother ports-secteam on every updates on individual
    ports, the merge procedure would be wanted to be automated.
    This may hit what I couldn't find the way to resolve for review
    D50282 and caused abandoning it.

    To go this way, merges may be needed to be done by each consumers
    like FreshBSD, or provide merged one outside git repo.
    (Or quite frequently bother ports-secteam to merge, sort and commit.)

    Anyway, FreshPorts is too helpful to put aside WRT this kind
    of changes. "How easy for dvl@ to adapt" would need to be
    considered among 1st class things.


    Regards.


    Yours,
    Robert Clausecker

    On Sat, Sep 26, 2026 at 11:30:20AM +0000, Abdelkader Boudih wrote:
    I think we should split the files once they reach certain level of entries (not size).

    2024.xml
    2025.xml
    2026.00.xml
    2026.xml
    2027.00.xml
    2027.01.xml
    2027.xml

    the year without the prefix still the current one before it get rotated. With amount of new software and audits, i expect this to rotate few time per year.

    Regards
    Abdelkader

    On Sat, 26 Sep 2026, at 11:13, Rodrigo Osorio wrote:
    Hi,

    For the records, security/vuxml/vuln/2026.xml has reached the 1048576 limit size,
    requiring the use of --push-option=i-know-this-file-is-big flag to commit it.
    And we still have 3 months left before 2027.

    Regards
    -- rodrigo




    --
    () ascii ribbon campaign - for an encoding-agnostic world
    /\ - against html email - against proprietary attachments

    --
    Tomoaki AOKI <junchoon@dec.sakura.ne.jp>


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Alexander Burke@alex@alexburke.ca to muc.lists.freebsd.ports on Sun Sep 27 02:12:16 2026
    From Newsgroup: muc.lists.freebsd.ports

    Hello,
    Putting things in perspective, 2TB WD Black drive with ZFS currently returns from running `time find /usr/ports -name Makefile -depth 2` on my system (currently under Firefox memory pressure) after first run, next run within minutes of the first, and 3rd run immediately when 2nd run completes:
    435.76 real-a 1.04 user-a 9.01 sys
    152.68 real-a 0.63 user-a 4.79 sys
    3.78 real-a-a-a 0.43 user-a 3.35 sys
    It sounds like your pool could very much benefit from a special VDEV: https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Pool%20Structure/Special%20vdev.html
    Just a word of warning, it must be redundant (so a couple of small MLC or decent TLC SSDs will do nicely), and if the pool contains any RAIDZ VDEVs (or any VDEVs of different ashift) then the special VDEV cannot be removed once added.
    Sometimes I will copy data from and back to disk to get ZFS to write it more optimally or do other practices that cause more organized structure to be written as it gets bad with ZFS fragmentation over time as git does scattered write/modify/delete throughout the tree.
    `zfs rewrite -P` might be helpful for you: https://openzfs.github.io/openzfs-docs/man/master/8/zfs-rewrite.8.html
    Cheers,
    Alex
    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Edward Sanford Sutton, III@mirror176@hotmail.com to muc.lists.freebsd.ports on Sat Sep 26 22:39:50 2026
    From Newsgroup: muc.lists.freebsd.ports

    On 9/26/26 21:33, Jamie Landeg-Jones wrote:
    "Edward Sanford Sutton, III" <mirror176@hotmail.com> wrote:

    though you would also want to prune distfiles and anything with a path
    starting with a capital letter to make the run less wasteful.
    In that same thought I always wonder why distfiles isn't capitalized
    like the other non-port containing folders.

    That would make things easier! Don't forget "packages" too!
    I forget about that one which is a side effect of using
    poudriere/tinderbox for package building + repo management.


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Mathieu Arnold@mat@freebsd.org to muc.lists.freebsd.ports on Sun Sep 27 10:56:28 2026
    From Newsgroup: muc.lists.freebsd.ports


    --l55gezd6byeq3mma
    Content-Type: text/plain; charset=iso-8859-1
    Content-Disposition: inline
    Content-Transfer-Encoding: quoted-printable

    On Sat, Sep 26, 2026 at 01:13:20PM +0200, Rodrigo Osorio wrote:
    Hi,
    =20
    For the records,=A0security/vuxml/vuln/2026.xml has reached the=A01048576=
    limit
    size,
    requiring the use of --push-option=3Di-know-this-file-is-big flag to comm=
    it
    it.
    And we still have 3 months left before 2027.

    This is not really a problem, I am half tempted to ignore the vuxml .xml
    files from the hook, but I am not sure it is a good idea, we would not
    want a bad file to get in.

    It's mostly a reminder for you to check that you did not commit
    something you did not want to.

    In any way, we're are most probably moving away from vuxml in the
    future, so there is no real need to spend too much time trying to
    re-invent and fix something that is on its last leg.

    --=20
    Mathieu Arnold

    --l55gezd6byeq3mma
    Content-Type: application/pgp-signature; name="signature.asc"

    -----BEGIN PGP SIGNATURE-----

    iQITBAABCgB9FiEE9XJBpJetWizkEBUef2IOCp6dQb4FAmq42jtfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEY1 NzI0MUE0OTdBRDVBMkNFNDEwMTUxRTdGNjIwRTBBOUU5RDQxQkUACgkQf2IOCp6d Qb7M9wv/WmdNnq1fpcDQbQr3VGYqUVHEXw52+FhhVGsGaYqOfr4efseemttPZdQu Nvb+lqIWl4lwtHNIwwGdCrMKhMKYy4ojNX4Vp//dsy8DaVsuoErg7oofx325ODUg ByVo/CD18yfXOYVXOU/Tm5rOeazCn8+n4ZqTPcG6t1RaqlikmAH08sHwUWXtYW57 HwTemjrpAKX/iNcDA4KuUIIGoOvPCFng3ffyyikEwdvHHSLjKBH2gl5lHkVZu/ku js7xaADqc3xR+nydomn/xQ7tP5gTQeRcoZsY47XYnwZI3/ikWZReYJtqJAUtO1Rg mtQXLkCVYPYkrpGBtS0hc/Q6xQ/RVFsqmZouqf63w1++fsE/djFGD1AzfO7CXFxz S8loZQ9I6Ug3Lhxam6YxRw2ohzKt/rtsRd7qz7zWKbJcKlAjTxDmrkwYpF48vaZl gnZ/ImEWG98yW+LihwP/l7A0DW1IGyY0qabgPosZ/f1kjcF4o6rq1xAgSVwX2mhx
    J7OAFkND
    =o8kg
    -----END PGP SIGNATURE-----

    --l55gezd6byeq3mma--


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2
  • From Piotr Smyrak@ps.ports@smyrak.com to muc.lists.freebsd.ports on Sun Sep 27 11:48:11 2026
    From Newsgroup: muc.lists.freebsd.ports

    On Sat, 26 Sep 2026 14:32:17 +0000
    Robert Clausecker <fuz@fuz.ooo> wrote:

    A bit aside,

    I would honestly prefer if each port had its own vuln.xml file and we
    build the central file by means of "find ${PORTSDIR} -name vuln.xml
    -depth 2". This would avoid the size issue and entirely remove the
    crazy annoying merge conflicts I get basically every time I have to
    add a vuxml entry.

    My understanding of current situation is that security/vuxml port is
    owned by ports-secteam@, while the proposed dispersal would make people
    feel that the distributed files are owned per port. While in practical
    terms it would also make the files cumbersome to monitor and manage by
    the team in question.

    Also, the conflicts are mostly induced by the fact that new entries are
    added at the top of the file.
    --
    Piotr Smyrak


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.22a-Linux NewsLink 1.2