From Newsgroup: muc.lists.freebsd.ports

--00000000000011f778063b635fbc
Content-Type: text/plain; charset="UTF-8"

Hi,

As a rule, Github offers dynamic tarball generation. Dynamic tarballs do
not exist at Gighub, they are generated on user's request from the sources, after user clicks on a special link. Which link points to virtual (non-existent) directory on Github. This technology is supported by
FreeBSD's USE_GITHUB pragma.

But sometime project owner chooses to generate static tarballs by hand, and places them into a special and real directory at Github. This approach is
used to escape cyclic dependencies or abnormally long compilations.

Cf. discussion here: https://lists.freebsd.org/pipermail/freebsd-ports/2020-July/119000.html

The technology of static tarballs is NOT supported by FreeBSD's USE_GITHUB pragma. To comply with it, FreeBSD's porters have to use something like:

MASTER_SITES= https://github.com/uriparser/uriparser/releases/download/uriparser-${DISTVERSION}/

Note that as it happens MASTER_SITES here obligatory involve a directory
name with explicit version number.

It is exactly the point where Portscout fails. Because it keeps looking for
new tarballs inside the directory of the old tarball. I have explicitly
checked it by running portscout from my local host and asking to check a
couple of ports of this kind.

The total number of problematic ports is 905:
grep -RIl -E -e 'MASTER_SITES=.*//github.com/.*VERSION' --include Makefile --exclude-dir Mk /usr/ports > gh-static-names.txt

Full list gh-static-names.txt of these problematic ports (which use static tarballs from Github) is here: https://drive.google.com/file/d/13ExEXbtF5e1Rpgtu-YrsaO7-3FWVJ-7w/view?usp=sharing

Once portscout fails to process them, a workaround to trace new versions
is: maintainer can subscribe to appropriate notifications from Github.

Do you know a way how developers or maintainers of portscout could be
kindly asked to elaborate portscout to cope with this kind of ports?
What PR (and where) can be created to address this issue?

Regards, Sergei

--00000000000011f778063b635fbc
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi,<br><br>As a rule, Github offers dynamic tarball genera= tion. Dynamic tarballs do not exist at Gighub, they are generated on user&#= 39;s request from the sources, after user clicks on a special link. Which l= ink points to virtual (non-existent) directory on Github. This technology i=
s supported by FreeBSD&#39;s USE_GITHUB pragma.<br><br>But sometime project=
owner chooses to generate static tarballs by hand, and places them into a = special and real directory at Github. This approach is used to escape cycli=
c dependencies or abnormally long compilations. <br><br>Cf. discussion here= :<br><a href=3D"https://lists.freebsd.org/pipermail/freebsd-ports/2020-July= /119000.html">https://lists.freebsd.org/pipermail/freebsd-ports/2020-July/1= 19000.html</a><br><br>The technology of static tarballs is NOT supported by=
FreeBSD&#39;s USE_GITHUB pragma. To comply with it, FreeBSD&#39;s porters = have to use something like:<br><br>MASTER_SITES=3D <a href=3D"https://githu=
b.com/uriparser/uriparser/releases/download/uriparser-${DISTVERSION}/">http= s://github.com/uriparser/uriparser/releases/download/uriparser-${DISTVERSIO= N}/</a><br><br>Note that as it happens MASTER_SITES here obligatory involve=
a directory name with explicit version number. <br><br>It is exactly the p= oint where Portscout fails. Because it keeps looking for new tarballs insid=
e the directory of the old tarball. I have explicitly checked it by running=
portscout from my local host and asking to check a couple of ports of this=
kind.<br><br>The total number of problematic ports is 905:<br>grep -RIl -E=
-e &#39;MASTER_SITES=3D.*//<a href=3D"http://github.com/.*VERSION">github.= com/.*VERSION</a>&#39; --include Makefile --exclude-dir Mk /usr/ports &gt; = gh-static-names.txt<br><br>Full list gh-static-names.txt of these problemat=
ic ports (which use static tarballs from Github) is here:<br><a href=3D"htt= ps://drive.google.com/file/d/13ExEXbtF5e1Rpgtu-YrsaO7-3FWVJ-7w/view?usp=3Ds= haring">https://drive.google.com/file/d/13ExEXbtF5e1Rpgtu-YrsaO7-3FWVJ-7w/v= iew?usp=3Dsharing</a><br><br>Once portscout fails to process them, a workar= ound to trace new versions is: maintainer can subscribe to appropriate noti= fications from Github.<br><br>Do you know a way how developers or maintaine=
rs of portscout could be kindly asked to elaborate portscout to cope with t= his kind of ports? <br>What PR (and where) can be created to address this i= ssue?<br><br>Regards, Sergei<br></div>

--00000000000011f778063b635fbc--


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.21a-Linux NewsLink 1.2

From Newsgroup: muc.lists.freebsd.ports


--Apple-Mail=_651A0147-07A4-417E-AEA2-E05CFFF2884E
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii

On Aug 2, 2025, at 17:17, Sergei Vyshenski <svysh.fbsd@gmail.com> =

wrote:

=20
Hi,
=20
As a rule, Github offers dynamic tarball generation. Dynamic tarballs =

do not exist at Gighub, they are generated on user's request from the = sources, after user clicks on a special link. Which link points to =
virtual (non-existent) directory on Github. This technology is supported =
by FreeBSD's USE_GITHUB pragma.

=20
But sometime project owner chooses to generate static tarballs by =

hand, and places them into a special and real directory at Github. This = approach is used to escape cyclic dependencies or abnormally long = compilations.

=20
Cf. discussion here:
=

https://lists.freebsd.org/pipermail/freebsd-ports/2020-July/119000.html

=20
The technology of static tarballs is NOT supported by FreeBSD's =

USE_GITHUB pragma. To comply with it, FreeBSD's porters have to use =
something like:

=20
MASTER_SITES=3D =

https://github.com/uriparser/uriparser/releases/download/uriparser-${DISTV= ERSION}/

=20
Note that as it happens MASTER_SITES here obligatory involve a =

directory name with explicit version number.

=20
It is exactly the point where Portscout fails. Because it keeps =

looking for new tarballs inside the directory of the old tarball. I have = explicitly checked it by running portscout from my local host and asking =
to check a couple of ports of this kind.

=20
The total number of problematic ports is 905:
grep -RIl -E -e 'MASTER_SITES=3D.*//github.com/.*VERSION' --include =

Makefile --exclude-dir Mk /usr/ports > gh-static-names.txt

=20
Full list gh-static-names.txt of these problematic ports (which use =

static tarballs from Github) is here:

=

https://drive.google.com/file/d/13ExEXbtF5e1Rpgtu-YrsaO7-3FWVJ-7w/view?usp= =3Dsharing

=20
Once portscout fails to process them, a workaround to trace new =

versions is: maintainer can subscribe to appropriate notifications from = Github.

=20
Do you know a way how developers or maintainers of portscout could be =

kindly asked to elaborate portscout to cope with this kind of ports?

What PR (and where) can be created to address this issue?
=20
Regards, Sergei

Hi,

I forgot when but sometimes ago there was a discussion in GH that from a = certain
time forward they will not guaranty the checksum/hashes for the =
dynamically
generated tarballs. That is actually against how our current ports =
framework
works. In case the upstream tarballs hash changes the port will fail =
during
fetch. So from a certain time forward MASTER_SITES is/will be the =
preferred or
recommended method for upstream dists. So we cannot avoid that.

Now from a maintainer's point of view of portscout I am not exactly a = maintainer
by choice but by circumstances. The original development of portscout =
has seized
I do not know since when. But I am often responsible to maintain the = portscout
instance that is running in our cluster that is =
https://portscout.freebsd.org <https://portscout.freebsd.org/>.
So I just maintain in a sense like whatever is needed to do to support =
newer
perl versions and regular host updates. If someone has a patch which =
works I
will be happy to test and add those changes to the tree.

Kind regards,
Moin

--Apple-Mail=_651A0147-07A4-417E-AEA2-E05CFFF2884E
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=signature.asc
Content-Type: application/pgp-signature;
name=signature.asc
Content-Description: Message signed with OpenPGP

-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEETfdREoUGjQZKBS+fvbm1phfAvJEFAmiOMiVfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDRE Rjc1MTEyODUwNjhEMDY0QTA1MkY5RkJEQjlCNUE2MTdDMEJDOTEACgkQvbm1phfA vJFAiA/9HfZw6f5myRm/kheYyFck1QAGbiexclsE94KfpLiNv7aBJCgl6l5wVTch LZ5ymhAY8Aqf4qTFGYJOdPsijLkPagHjdWY9o9WtZ4uHnDDrwQB72J6DDsIjP9/d ZPYFBhsisuP3kVBGKpEUp4oX+DLVcZTl4o7xMRHmmqpzKih8Y0FIMyxCAUgHzcHi MClqdt2SWcmLvT+LtlN8faQhj8qifcIrO0Xeb7fGm6XTQyRES8jYSflj/utXp4GO KF3qN7JBPLKmqTBfgCOod4fwHSLjmEoTqil6NWzGCui1qZ31omtvJl1AETKrREJg B3k4fWZLrIKkh/ZWxUcQqXZHx9/q74K8TTG/WmwjczV0qS0P+KcsQLv9nTXWv3sV b8j+mDAZcNB9NXrH7U73Crmfz83/sNe37wSk8WCNnbXr9GKha75imHfdubrMT4/X +Z6RxzIniPHlOga38uQpqlNx2Bnaa9FqwIrTtsLZgpEYLCoioRY33GjwJ6ngxrok FLTt2/gir5hJfOIY/5IE30f1W7G4zC7YP0oiBVyw8oQ/ZqAka+ORw810Ejng20wV 9EdYTuEX7vZ/R4muIO9MIU9WE0n+N9vGPQl9Z7TLnwwwZfm9tnOHNjt4cj/rIs4I UWfBLlkSi0R0iR1T2vE1uaCjIv3sv650DCNA9eAvLBBvW9Mz4+U=
=jKLi
-----END PGP SIGNATURE-----

--Apple-Mail=_651A0147-07A4-417E-AEA2-E05CFFF2884E--


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.21a-Linux NewsLink 1.2