Hello,
I'm trying to solve a FreshPorts.org problem: ports which have PORTVERSION set by something outside their Makefile (when I say PORTVERSION here, I am also considering DISTVERSION). Slave ports are one example, and they are solved: There is a direct link between a slave port and the master port.
The problem at hand is Mk/bsd.default-versions
I'm posting my approach for sharing and for peer review.
Use lang/ports as an example.
[0:15 mydev dvl /usr/ports/lang/python] % grep PORTVERSION Makefile PORTVERSION= ${PYTHON_DEFAULT}
PYTHON_DEFAULT is defined within Mk/bsd.default-versions - I will refer to those magic values as *_DEFAULT.
My goal: when PYTHON_DEFAULT is modified (in Mk/bsd.default-versions), refresh lang/python, and any other ports which use PYTHON_DEFAULT for PORTVERSION.
First step: get a list of *_DEFAULT values from Mk/bsd.default-versions - that's done. [1]
Next: get a list of ports which use those *_DEFAULT values - I have a first draft of that. [2]
How can you tell if a port uses PYTHON_DEFAULT? I'm using make -V like this:
[0:14 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V PORTVERSION -V DISTVERSION
3.12
3.12
Now let's set
[0:15 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V PORTVERSION -V DISTVERSION PYTHON_DEFAULT=3.10
3.10
3.10
The values changed... so clearly this port depends on that.
Repeat for all 39 values. Over all 35313 ports. Takes about 2.5 hours. That's just for initial population of the data. For each commit to a port, that check will be run again to keep the relationships up to date.
I still have some debugging of that script to do. Some of it is clearly wrong and some ports are missing. They are edge cases. The gist in [1] has the list so far.
So far, it's a list of 14 ports - I find it ironic that so much code was written and run to find so few ports. And only 7 of the *_DEFAULT values are actually used for PORTVERSION. It's as if we could maintain that list manually (however, that goes against the core values of FreshPorts design).
Questions:
- can you think of a better way to do this?
- what other files hold magic values such as *_DEFAULT
Thank you.
1 - list of *_DEFAULT values - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-1-the-default-values-from-bsd-default-versions
2 - https://github.com/FreshPorts/DefaultVersions/blob/main/populate_port_default_version_deps.py - with much help from Claude
3 - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-3-port_default_version_variable
--
Dan Langille
dan@langille.org
/usr/ports/devel/electron40/Makefile.version /usr/ports/devel/electron41/Makefile.version /usr/ports/devel/electron42/Makefile.version /usr/ports/devel/electron43/Makefile.version /usr/ports/devel/electron44/Makefile.version /usr/ports/devel/mdds/Makefile.version /usr/ports/editors/vscode/Makefile.version /usr/ports/emulators/linux_base-rl9/Makefile.version /usr/ports/graphics/drm-515-kmod/Makefile.version /usr/ports/graphics/drm-61-kmod/Makefile.version /usr/ports/graphics/drm-612-kmod/Makefile.version /usr/ports/graphics/drm-66-kmod/Makefile.version /usr/ports/graphics/drm-latest-kmod/Makefile.version /usr/ports/lang/gcc6-aux/Makefile.version /usr/ports/lang/python27/Makefile.version /usr/ports/lang/python310/Makefile.version /usr/ports/lang/python311/Makefile.version /usr/ports/lang/python312/Makefile.version /usr/ports/lang/python313/Makefile.version /usr/ports/lang/python314/Makefile.version /usr/ports/lang/python315/Makefile.version /usr/ports/math/rubygem-narray/Makefile.version /usr/ports/math/vtk9/Makefile.version /usr/ports/science/InsightToolkit/Makefile.version /usr/ports/science/paraview/Makefile.version /usr/ports/shells/nushell/Makefile.version /usr/ports/sysutils/usacloud-core/Makefile.version /usr/ports/www/node20/Makefile.version
/usr/ports/www/node22/Makefile.version
/usr/ports/www/node24/Makefile.version
/usr/ports/www/node26/Makefile.version /usr/ports/x11-servers/xlibre-server/Makefile.version /usr/ports/x11/linux-rl9-xorg-libs/Makefile.version /usr/ports/x11/nvidia-driver/Makefile.version
I'm trying to solve a FreshPorts.org problem: ports which haveMaybe I' missing something, but why don't you use something like
PORTVERSION set by something outside their Makefile (when I say
PORTVERSION here, I am also considering DISTVERSION). Slave ports are
one example, and they are solved: There is a direct link between a slave
port and the master port.
The problem at hand is Mk/bsd.default-versions
I'm posting my approach for sharing and for peer review.
Use lang/ports as an example.
[0:15 mydev dvl /usr/ports/lang/python] % grep PORTVERSION Makefile PORTVERSION= ${PYTHON_DEFAULT}
PYTHON_DEFAULT is defined within Mk/bsd.default-versions - I will refer to those magic values as *_DEFAULT.
My goal: when PYTHON_DEFAULT is modified (in Mk/bsd.default-versions), refresh lang/python, and any other ports which use PYTHON_DEFAULT for PORTVERSION.
First step: get a list of *_DEFAULT values from Mk/bsd.default-versions - that's done. [1]
Next: get a list of ports which use those *_DEFAULT values - I have a first draft of that. [2]
grep -r -l PYTHON_DEFAULT /usr/ports/ --include Makefile | cut -d /-f 1-5
"Dan Langille" <dan@langille.org> wrote:
I'm trying to solve a FreshPorts.org problem: ports which have
PORTVERSION set by something outside their Makefile (when I say
PORTVERSION here, I am also considering DISTVERSION). Slave ports are
one example, and they are solved: There is a direct link between a slave
port and the master port.
The problem at hand is Mk/bsd.default-versions
I'm posting my approach for sharing and for peer review.
Use lang/ports as an example.
[0:15 mydev dvl /usr/ports/lang/python] % grep PORTVERSION Makefile
PORTVERSION= ${PYTHON_DEFAULT}
PYTHON_DEFAULT is defined within Mk/bsd.default-versions - I will refer to those magic values as *_DEFAULT.
My goal: when PYTHON_DEFAULT is modified (in Mk/bsd.default-versions),
refresh lang/python, and any other ports which use PYTHON_DEFAULT for
PORTVERSION.
First step: get a list of *_DEFAULT values from Mk/bsd.default-versions - that's done. [1]
Next: get a list of ports which use those *_DEFAULT values - I have a first draft of that. [2]
Maybe I' missing something, but why don't you use something like
grep -r -l PYTHON_DEFAULT /usr/ports/ --include Makefile | cut -d /-f 1-5
for each *_DEFAULT value and feed the results to the differential make
-V check?
On Tue, Sep 15, 2026, at 6:58 PM, Hilko Meyer wrote:Because it matches OPTIONS_DEFAULT. I tried something similar first and
"Dan Langille" <dan@langille.org> wrote:
I'm trying to solve a FreshPorts.org problem: ports which have
PORTVERSION set by something outside their Makefile (when I say
PORTVERSION here, I am also considering DISTVERSION). Slave ports are
one example, and they are solved: There is a direct link between a slave >>> port and the master port.
The problem at hand is Mk/bsd.default-versions
I'm posting my approach for sharing and for peer review.
Use lang/ports as an example.
[0:15 mydev dvl /usr/ports/lang/python] % grep PORTVERSION Makefile
PORTVERSION= ${PYTHON_DEFAULT}
PYTHON_DEFAULT is defined within Mk/bsd.default-versions - I will refer to those magic values as *_DEFAULT.
My goal: when PYTHON_DEFAULT is modified (in Mk/bsd.default-versions),
refresh lang/python, and any other ports which use PYTHON_DEFAULT for
PORTVERSION.
First step: get a list of *_DEFAULT values from Mk/bsd.default-versions - that's done. [1]
Next: get a list of ports which use those *_DEFAULT values - I have a first draft of that. [2]
Maybe I' missing something, but why don't you use something like
> grep -r -l PYTHON_DEFAULT /usr/ports/ --include Makefile | cut -d /
-f 1-5
for each *_DEFAULT value and feed the results to the differential make
-V check?
That's a good idea. It certainly reduces the number of `make -V` checks needed. That grep could be expanded too, to something like:
% grep -rEo --include='Makefile*' --include='*.mk' --exclude-dir=work --exclude-dir=.git '\b[A-Z0-9_]+_DEFAULT\b' /usr/ports | wc -l
4643
Which finds 4643 matches.
grep -rEo --include='Makefile*' --include='*.mk' --exclude-dir=work --exclude-dir=.git '\b(NODEJS|OPENLDAP|PYTHON|PYTHON2|RUBY|SAMBA|SSL|SUDO)+_DEFAULT\b' /usr/ports | sort -uAnd I would exlude /usr/ports/Mk/ too.
I compared the list of ports in my gist aginst that output. I found the following where missing (false negatives):Hm, modifying the expression to something like this '\b(NODEJS|OPENLDAP|PYTHON|PYTHON2|RUBY|SAMBA|SSL|SUDO)+_(DEFAULT|DISTVERSION|PORTVERSION)\b'
databases/py-gdbm - uses PYTHON_DISTVERSION
databases/py-sqlite3 - same
lang/python-doc-html - slave port : MASTERDIR= ${.CURDIR}/../python-doc-html lang/python-tools - same
lang/python-tools - PYTHON_DISTVERSION
www/node - NODEJS_PORTVERSION
x11-toolkits/py-tkinter - PYTHON_DISTVERSION
Any path where the variable reaches PORTVERSION without appearing in that port's Makefile becomes a silent miss. It's the false negatives.Well, I did before I completely understood your point. :-)
Yes, we could start adding exceptions and grepping for them all.
The problem: FreshPorts cannot dictate ports tree standards. It can only try to keep up.True, but the ports tree standards could be in a way that doesn't make
Hello.
This will be an approach to unrelated parts, but...
For example, for the following two commands, in my environment, the
time it takes for a response to return is clearly different.
make -C /usr/ports/multimedia/libopenshot/ -V PORTVERSION -V DISTVERSION
make -C /usr/ports/multimedia/libopenshot/ -D _INCLUDE_USES_COMPILER_MK
-V PORTVERSION -V DISTVERSION
It seems that the logic of what happens when -D
_INCLUDE_USES_COMPILER_MK is added can be understood by looking at Mk/Uses/compiler.mk.
The dependencies change because the contents are skipped over, but if
that does not cause even the version number to change, it is thought
that this could become a usable method.
I apologize if that method has already been adopted :)
Hilko Meyer <hilko.meyer@gmx.de> wrote:Is the above, this: port.version : ${PORTVERSION} (PKG: ${PORTVERSION}_${PORTREVISION})
The problem: FreshPorts cannot dictate ports tree standards. It can only try to keep up.
True, but the ports tree standards could be in a way that doesn't make
it harder for tools like Freshports.
I have a tool that requires such variables too, and with all the edge cases that kept popping up, I finally gave up, and now have a background job
that simply does make -V from a sandbox user and caches the results. It automatically fires for updates to a ports Makefile (which I realise could miss some changes), but also runs a complete fresh-run every 4 weeks or so.
So, some standard way of extracting all port variables without requiring
make -V would be welcome.
02:42 (31.0-#C 798) (6) "www" jamie@catflap% getallextattr ~git-mirrors/repos/freebsd-ports/latest/www/w3m/Makefile ________________________________________________________________________________________________________________________________________
/usr/native/git-mirrors/repos/freebsd-ports/latest/www/w3m/Makefile:
Thu May 7 22:47:29 BST 2026 : git-mirrors git-mirrors -rw-r--r-- -
3014 bytes (Namespace: user)
port.comment : Pager/text-based WWW browser
port.flavors :
port.license : w3m
port.maintainer : nobutaka@FreeBSD.org
port.version : 0.5.6 (PKG: 0.5.6_1)
port.www : https://git.sr.ht/~rkta/w3m ________________________________________________________________________________________________________________________________________--
Hello,
I'm trying to solve a FreshPorts.org problem: ports which have
PORTVERSION set by something outside their Makefile (when I say
PORTVERSION here, I am also considering DISTVERSION). Slave ports are
one example, and they are solved: There is a direct link between a
slave port and the master port.
The problem at hand is Mk/bsd.default-versions
I'm posting my approach for sharing and for peer review.
Use lang/ports as an example.
[0:15 mydev dvl /usr/ports/lang/python] % grep PORTVERSION Makefile PORTVERSION= ${PYTHON_DEFAULT}
PYTHON_DEFAULT is defined within Mk/bsd.default-versions - I will refer
to those magic values as *_DEFAULT.
My goal: when PYTHON_DEFAULT is modified (in Mk/bsd.default-versions), refresh lang/python, and any other ports which use PYTHON_DEFAULT for PORTVERSION.
First step: get a list of *_DEFAULT values from Mk/bsd.default-versions
- that's done. [1]
Next: get a list of ports which use those *_DEFAULT values - I have a
first draft of that. [2]
How can you tell if a port uses PYTHON_DEFAULT? I'm using make -V like this:
[0:14 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V
PORTVERSION -V DISTVERSION
3.12
3.12
Now let's set
[0:15 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V
PORTVERSION -V DISTVERSION PYTHON_DEFAULT=3.10
3.10
3.10
The values changed... so clearly this port depends on that.
Repeat for all 39 values. Over all 35313 ports. Takes about 2.5 hours. That's just for initial population of the data. For each commit to a
port, that check will be run again to keep the relationships up to date.
I still have some debugging of that script to do. Some of it is clearly wrong and some ports are missing. They are edge cases. The gist in [1]
has the list so far.
So far, it's a list of 14 ports - I find it ironic that so much code
was written and run to find so few ports. And only 7 of the *_DEFAULT
values are actually used for PORTVERSION. It's as if we could maintain
that list manually (however, that goes against the core values of
FreshPorts design).
Questions:
- can you think of a better way to do this?
- what other files hold magic values such as *_DEFAULT
Thank you.
1 - list of *_DEFAULT values - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-1-the-default-values-from-bsd-default-versions
2 - https://github.com/FreshPorts/DefaultVersions/blob/main/populate_port_default_version_deps.py
- with much help from Claude
3 - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-3-port_default_version_variable
- can you think of a better way to do this?
- what other files hold magic values such as *_DEFAULT
On Mon, Sep 14, 2026 at 09:00:14PM -0400, Dan Langille wrote:
- can you think of a better way to do this?
- what other files hold magic values such as *_DEFAULT
I can see another way, but asynchronous, you regularly, like every hour
or so, run make index, or something similar to retrieve the package
origin, name and version of all ports.
For each port, you compare the version of the one you have in your
database, and for each that differ, it means their version is not
defined in their port, you then run a git bisect from the previous
commit you did the comparison to the new one, running make -V PKGVERSION
for each one, and after a few steps, you'll find out which commit
changed the version, and you can then use that commit to run whatever is usually run when a port is updated.
The creation of INDEX takes time. By the time it is ready, more commits h=ave
occurred and INDEX is already out of date and some PORTVERSION values have changed.
Le jeu. 17 sept. 26 |a 17:00:55 +0200, Dan Langille <dan@langille.org>I hope I understand the following correctly.
|-crivait-a:
The creation of INDEX takes time. By the time it is ready, more commits have >> occurred and INDEX is already out of date and some PORTVERSION values have >> changed.
`make fetchindex` is fine.
Warning: INDEX-i depends on the system version.
Le jeu. 17 sept. 26 =E0 17:00:55 +0200, Dan Langille <dan@langille.org>have
=E9crivait=A0:
=20
The creation of INDEX takes time. By the time it is ready, more commits=
aveoccurred and INDEX is already out of date and some PORTVERSION values h=
changed.=20
`make fetchindex` is fine.
On Thu, Sep 17, 2026 at 05:11:13PM +0200, Thierry Thomas wrote:That gave me an idea. I ran `make index` in a jail[1]. It took 747 seconds (or 12.5 minutes).
Le jeu. 17 sept. 26 |a 17:00:55 +0200, Dan Langille <dan@langille.org>
|-crivait-a:
The creation of INDEX takes time. By the time it is ready, more commits have
occurred and INDEX is already out of date and some PORTVERSION values have >> > changed.
`make fetchindex` is fine.
Well, it is faster, but it will not be what `make index` would output,
as it's a previously generated INDEX that was geneated from another
ports tree.
port.comment : Pager/text-based WWW browser
port.flavors :
port.license : w3m
port.maintainer : nobutaka@FreeBSD.org
port.version : 0.5.6 (PKG: 0.5.6_1)
Is the above, this: port.version : ${PORTVERSION} (PKG: ${PORTVERSION}_${PORTREVISION})
On Thu, Sep 17, 2026, at 12:42 PM, Mathieu Arnold wrote:One issue to overcome with INDEX: multiple packages for the same origin.
On Thu, Sep 17, 2026 at 05:11:13PM +0200, Thierry Thomas wrote:
Le jeu. 17 sept. 26 |a 17:00:55 +0200, Dan Langille <dan@langille.org>
|-crivait-a:
The creation of INDEX takes time. By the time it is ready, more commits have
occurred and INDEX is already out of date and some PORTVERSION values have
changed.
`make fetchindex` is fine.
Well, it is faster, but it will not be what `make index` would output,
as it's a previously generated INDEX that was geneated from another
ports tree.
That gave me an idea. I ran `make index` in a jail[1]. It took 747
seconds (or 12.5 minutes).
How does this sound for a strategy?
* periodic/cronjob inserts a job into the FreshPorts work queue
* freshports daemon (FPD) processes the queue in order, so commit
processing is not going on during that job
* FPD find the job, notes the time, and launches a background job to
`make index` (MI)
* FPD continues on with the queue
* ...
* 12.5 minutes later, as the MI job completes, it inserts a reconcile
job into the queue
* when FPD gets to the reconcile job, it knows the start time
associated with the MI
* FPD compares the INDEX against the database and compiles a list of
ports which don't match INDEX
* any ports with commits after that MI start time are not included in
that list
* The ports on that list are refreshed from their Makefiles (because
more than just PORTVERSION can change) in the usual way that FP handles
a commit
* FPD continues on with the queue
* ...
Hello,
I'm trying to solve a FreshPorts.org problem: ports which have PORTVERSION set by something outside their Makefile (when I say PORTVERSION here, I am also considering DISTVERSION). Slave ports are one example, and they are solved: There is a direct link between a slave port and the master port.
The problem at hand is Mk/bsd.default-versions
I'm posting my approach for sharing and for peer review.
Use lang/ports as an example.
[0:15 mydev dvl /usr/ports/lang/python] % grep PORTVERSION Makefile PORTVERSION= ${PYTHON_DEFAULT}
PYTHON_DEFAULT is defined within Mk/bsd.default-versions - I will refer to those magic values as *_DEFAULT.
My goal: when PYTHON_DEFAULT is modified (in Mk/bsd.default-versions), refresh lang/python, and any other ports which use PYTHON_DEFAULT for PORTVERSION.
First step: get a list of *_DEFAULT values from Mk/bsd.default-versions - that's done. [1]
Next: get a list of ports which use those *_DEFAULT values - I have a first draft of that. [2]
How can you tell if a port uses PYTHON_DEFAULT? I'm using make -V like this:
[0:14 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V PORTVERSION -V DISTVERSION
3.12
3.12
Now let's set
[0:15 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V PORTVERSION -V DISTVERSION PYTHON_DEFAULT=3.10
3.10
3.10
The values changed... so clearly this port depends on that.
Repeat for all 39 values. Over all 35313 ports. Takes about 2.5 hours. That's just for initial population of the data. For each commit to a port, that check will be run again to keep the relationships up to date.
I still have some debugging of that script to do. Some of it is clearly wrong and some ports are missing. They are edge cases. The gist in [1] has the list so far.
So far, it's a list of 14 ports - I find it ironic that so much code was written and run to find so few ports. And only 7 of the *_DEFAULT values are actually used for PORTVERSION. It's as if we could maintain that list manually (however, that goes against the core values of FreshPorts design).
Questions:
- can you think of a better way to do this?
- what other files hold magic values such as *_DEFAULT
Thank you.
1 - list of *_DEFAULT values - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-1-the-default-values-from-bsd-default-versions
2 - https://github.com/FreshPorts/DefaultVersions/blob/main/populate_port_default_version_deps.py - with much help from Claude
3 - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-3-port_default_version_variable
On Thu, 17 Sep 2026 18:01:14 -0400Can the default flavor be identified in INDEX?
"Dan Langille" <dan@langille.org> wrote:
On Thu, Sep 17, 2026, at 1:33 PM, Dan Langille wrote:
On Thu, Sep 17, 2026, at 12:42 PM, Mathieu Arnold wrote:
On Thu, Sep 17, 2026 at 05:11:13PM +0200, Thierry Thomas wrote:
Le jeu. 17 sept. 26 |a 17:00:55 +0200, Dan Langille <dan@langille.org> >> >>> |-crivait-a:
The creation of INDEX takes time. By the time it is ready, more commits have
occurred and INDEX is already out of date and some PORTVERSION values have
changed.
`make fetchindex` is fine.
Well, it is faster, but it will not be what `make index` would output,
as it's a previously generated INDEX that was geneated from another
ports tree.
That gave me an idea. I ran `make index` in a jail[1]. It took 747
seconds (or 12.5 minutes).
How does this sound for a strategy?
* periodic/cronjob inserts a job into the FreshPorts work queue
* freshports daemon (FPD) processes the queue in order, so commit
processing is not going on during that job
* FPD find the job, notes the time, and launches a background job to
`make index` (MI)
* FPD continues on with the queue
* ...
* 12.5 minutes later, as the MI job completes, it inserts a reconcile
job into the queue
* when FPD gets to the reconcile job, it knows the start time
associated with the MI
* FPD compares the INDEX against the database and compiles a list of
ports which don't match INDEX
* any ports with commits after that MI start time are not included in
that list
* The ports on that list are refreshed from their Makefiles (because
more than just PORTVERSION can change) in the usual way that FP handles >> > a commit
* FPD continues on with the queue
* ...
One issue to overcome with INDEX: multiple packages for the same origin.
If the versions are the same, no issue, but when they differ...
Which one is correct?
For example:
py311-tkinter-3.11.16_11
py312-tkinter-3.12.14_11
py313-tkinter-3.13.15_11
py313t-tkinter-3.13.15_11
py314-tkinter-3.14.7_11
py314t-tkinter-3.14.7_11
py315-tkinter-3.15.0.r2_11
All from x11-toolkits/py-tkinter
Which one should FreshPorts choose as the correct one? The existing practice is to use `make -V PORTVERSION`
This seems to affect only these ports:
databases/py-gdbm
databases/py-sqlite3
devel/libclc
devel/opencl-clang
devel/spirv-llvm-translator
x11-servers/xorg-server
Full output at https://gist.github.com/dlangille/105f15e6e0c219e1ef0fd747398ae8bf
Next, I'll load that data into a FreshPorts database, and see what differences I can pull back from a query.
That's all I have for now. I'll come back to this exercise later.
Thank you.
--
Dan Langille
dan@langille.org
It would be depending on FLAVORs.
IIRC, when mail/claws-mail still had GTK2 flavor, versions for
GTK2 was 3.*, while (currently remaining unflavored) GTK3 has 4.*.
So choosing for default flavor may be preferrable.
On 9/14/26 18:00, Dan Langille wrote:
Hello,
I'm trying to solve a FreshPorts.org problem: ports which have PORTVERSION set by something outside their Makefile (when I say PORTVERSION here, I am also considering DISTVERSION). Slave ports are one example, and they are solved: There is a direct link between a slave port and the master port.
The problem at hand is Mk/bsd.default-versions
I'm posting my approach for sharing and for peer review.
Use lang/ports as an example.
[0:15 mydev dvl /usr/ports/lang/python] % grep PORTVERSION Makefile
PORTVERSION= ${PYTHON_DEFAULT}
PYTHON_DEFAULT is defined within Mk/bsd.default-versions - I will refer to those magic values as *_DEFAULT.
My goal: when PYTHON_DEFAULT is modified (in Mk/bsd.default-versions), refresh lang/python, and any other ports which use PYTHON_DEFAULT for PORTVERSION.
First step: get a list of *_DEFAULT values from Mk/bsd.default-versions - that's done. [1]
Next: get a list of ports which use those *_DEFAULT values - I have a first draft of that. [2]
How can you tell if a port uses PYTHON_DEFAULT? I'm using make -V like this: >>
[0:14 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V PORTVERSION -V DISTVERSION
3.12
3.12
Now let's set
[0:15 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V PORTVERSION -V DISTVERSION PYTHON_DEFAULT=3.10
3.10
3.10
The values changed... so clearly this port depends on that.
Repeat for all 39 values. Over all 35313 ports. Takes about 2.5 hours. That's just for initial population of the data. For each commit to a port, that check will be run again to keep the relationships up to date.
I still have some debugging of that script to do. Some of it is clearly wrong and some ports are missing. They are edge cases. The gist in [1] has the list so far.
So far, it's a list of 14 ports - I find it ironic that so much code was written and run to find so few ports. And only 7 of the *_DEFAULT values are actually used for PORTVERSION. It's as if we could maintain that list manually (however, that goes against the core values of FreshPorts design).
Questions:
- can you think of a better way to do this?
I don't know what is better as I didn't finish a full workload to
this yet but in case it inspires better/faster minds I was thinking
along the line of...
You can get raw values with debugging information:
make -C /usr/ports/lang/python -dV -VPORTVERSION -VDISTVERSION
That makes it easier to identify the variable contains a reference to another variable but you then need to find where that variable is
defined. Another debug variable and now we can identify where
definitions are coming from in terms of filenames but will have a lot
more noise to sort out.
make -C /usr/ports/lang/python -dpV -VPKGVERSION 2>debug.txt
I was not able to use -dFdebug.txt to get it to write output to that
file and am not sure why.
This would allow us to avoid running make more than once if we can
get through the noise and it also makes it so we can tell what is being
set by external files.
I assume that freshports would need to both store the value of a
variable and store the list of files that are involved to set that
value.
By watching for changes to files, you now know which variables
are impacted by a file being updated. Not sure when it is worth trying
to manually track when that information changes in the file vs use make
to recreate values anytime the file gets updated but I'd assume that
some effort of doing make's job would be lighter on load.
Alternatively,
you could save some runs if you can extract select variables such as
from Mk/bsd.default-versions.mk so that instead of having to rerun make
for every use of a variable from it, you could get the value from one
run of a variable and use it for all occurrences if you have accurate non-substitution values stored and you tracked what file completes the substitution.
I was then attempting to use awk to process the debug output contents
but have kind of given up as awk is not making sense to me. Using an awk script if I put match($4,"}") inside of a print statement I get a number
of the position 17 if $4=${PYTHON_DEFAULT} but if I try to save it to a variable like
SubVarStart = match($4,"${")
SubVarEnd = match($4,"}")
then I end up with
SubVarStart='Parsing Makefile:2: PORTVERSION= ${PYTHON_DEFAULT}' SubVarEnd=''
which is not the equivalent to just printing the match function's output
nor can I make any sense of what I am logically getting instead. If I
cannot store the function's output then I cannot use the function as far
as I am aware of.
Not sure if awk was the right tool for the job or if it would get it
done alone or in how many passes but I assumed it would become much more efficient to do text processing on a make run's debug output to both get more details like the file that defines a variable and be able to walk through subvariables at much higher speed.
For anyone trying to build off of this, $2 collects /path/file.name:linenumber: , $3 collects VARIABLE (characters to set it vary: '=', ':=', '?=' and $4 collects the value of the variable. I'd
guess $5+ may be relevant when looking at other things like variables containing dependencies but I never went there yet.
- what other files hold magic values such as *_DEFAULT
It was already brought up in this thread that there is a wider issue
of external values being used to do more than set a port's version. That
is why I was trying to approach it by collecting information about a variable being defined against another variable, and then try to collect where that variable comes from.
--Thank you.
1 - list of *_DEFAULT values - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-1-the-default-values-from-bsd-default-versions
2 - https://github.com/FreshPorts/DefaultVersions/blob/main/populate_port_default_version_deps.py - with much help from Claude
3 - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-3-port_default_version_variable
On Thu, 17 Sep 2026 13:33:23 -0400I did not know.
"Dan Langille" <dan@langille.org> wrote:
On Thu, Sep 17, 2026, at 12:42 PM, Mathieu Arnold wrote:
On Thu, Sep 17, 2026 at 05:11:13PM +0200, Thierry Thomas wrote:
Le jeu. 17 sept. 26 |a 17:00:55 +0200, Dan Langille <dan@langille.org>
|-crivait-a:
The creation of INDEX takes time. By the time it is ready, more commits have
occurred and INDEX is already out of date and some PORTVERSION values have
changed.
`make fetchindex` is fine.
Well, it is faster, but it will not be what `make index` would output,
as it's a previously generated INDEX that was geneated from another
ports tree.
That gave me an idea. I ran `make index` in a jail[1]. It took 747 seconds (or 12.5 minutes).
How does this sound for a strategy?
* periodic/cronjob inserts a job into the FreshPorts work queue
* freshports daemon (FPD) processes the queue in order, so commit processing is not going on during that job
* FPD find the job, notes the time, and launches a background job to `make index` (MI)
* FPD continues on with the queue
* ...
* 12.5 minutes later, as the MI job completes, it inserts a reconcile job into the queue
* when FPD gets to the reconcile job, it knows the start time associated with the MI
* FPD compares the INDEX against the database and compiles a list of ports which don't match INDEX
* any ports with commits after that MI start time are not included in that list
* The ports on that list are refreshed from their Makefiles (because more than just PORTVERSION can change) in the usual way that FP handles a commit
* FPD continues on with the queue
* ...
1 - [17:03 mydev dvl /usr/ports] % sudo time make index
Generating INDEX-15 - please wait..--- describe.accessibility ---
--- describe.arabic ---
--- describe.archivers ---
...
--- describe.x11-toolkits ---
--- describe.x11-wm ---
Done. 747.38 real 2831.68 user 1706.13 sys
--
Dan Langille
dan@langille.org
IIUC, `make index` recursively runs `make describe`
per ALL ports, thus, take a long time.
And IIUC, FreshPorts updates information (thanks!) per commitYes. (my pleasure)
basis.
So, is it considerable to store the output of `make describe`What goal did you have in mind for that output? Did you see a way
for ports affected by the commit (i.e., store into same
directory structure under different ports top directory,
or store into any database of your choice)?
If you want json format instead, you can use `make describe-json`.
Hello,How about checking TIMESTAMP in the distinfo file? I can't imagine a
I'm trying to solve a FreshPorts.org problem: ports which have PORTVERSION set by something outside their Makefile (when I say PORTVERSION here, I am also considering DISTVERSION).
On Tue, Sep 15, 2026 at 4:01rC>AM Dan Langille <dan@langille.org> wrote:Commits get version bumps without files in their directory being touched. lang/python is the best example.
Hello,
I'm trying to solve a FreshPorts.org problem: ports which have PORTVERSION set by something outside their Makefile (when I say PORTVERSION here, I am also considering DISTVERSION).
How about checking TIMESTAMP in the distinfo file? I can't imagine a
version bump without touching distinfo.
On Thu, Sep 17, 2026, at 7:09 PM, Edward Sanford Sutton, III wrote:
On 9/14/26 18:00, Dan Langille wrote:
Hello,
I'm trying to solve a FreshPorts.org problem: ports which have PORTVERSION set by something outside their Makefile (when I say PORTVERSION here, I am also considering DISTVERSION). Slave ports are one example, and they are solved: There is a direct link between a slave port and the master port.
The problem at hand is Mk/bsd.default-versions
I'm posting my approach for sharing and for peer review.
Use lang/ports as an example.
[0:15 mydev dvl /usr/ports/lang/python] % grep PORTVERSION Makefile
PORTVERSION= ${PYTHON_DEFAULT}
PYTHON_DEFAULT is defined within Mk/bsd.default-versions - I will refer to those magic values as *_DEFAULT.
My goal: when PYTHON_DEFAULT is modified (in Mk/bsd.default-versions), refresh lang/python, and any other ports which use PYTHON_DEFAULT for PORTVERSION.
First step: get a list of *_DEFAULT values from Mk/bsd.default-versions - that's done. [1]
Next: get a list of ports which use those *_DEFAULT values - I have a first draft of that. [2]
How can you tell if a port uses PYTHON_DEFAULT? I'm using make -V like this:
[0:14 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V PORTVERSION -V DISTVERSION
3.12
3.12
Now let's set
[0:15 mydev dvl /usr/ports] % make -C /usr/ports/lang/python -V PORTVERSION -V DISTVERSION PYTHON_DEFAULT=3.10
3.10
3.10
The values changed... so clearly this port depends on that.
Repeat for all 39 values. Over all 35313 ports. Takes about 2.5 hours. That's just for initial population of the data. For each commit to a port, that check will be run again to keep the relationships up to date.
I still have some debugging of that script to do. Some of it is clearly wrong and some ports are missing. They are edge cases. The gist in [1] has the list so far.
So far, it's a list of 14 ports - I find it ironic that so much code was written and run to find so few ports. And only 7 of the *_DEFAULT values are actually used for PORTVERSION. It's as if we could maintain that list manually (however, that goes against the core values of FreshPorts design).
Questions:
- can you think of a better way to do this?
I don't know what is better as I didn't finish a full workload to
this yet but in case it inspires better/faster minds I was thinking
along the line of...
You can get raw values with debugging information:
make -C /usr/ports/lang/python -dV -VPORTVERSION -VDISTVERSION
Oh wow. For those following along at home:
[12:01 mydev dvl /usr/ports/sysutils/samdruckerclientshell] % make -C /usr/ports/lang/python -dV -VPORTVERSION -VDISTVERSION
${PYTHON_DEFAULT}
${PORTVERSION:S/:/::/g}
That makes it easier to identify the variable contains a reference to
another variable but you then need to find where that variable is
defined. Another debug variable and now we can identify where
definitions are coming from in terms of filenames but will have a lot
more noise to sort out.
make -C /usr/ports/lang/python -dpV -VPKGVERSION 2>debug.txt
I was not able to use -dFdebug.txt to get it to write output to that
file and am not sure why.
This would allow us to avoid running make more than once if we can
get through the noise and it also makes it so we can tell what is being
set by external files.
I assume that freshports would need to both store the value of a
variable and store the list of files that are involved to set that
value.
That sounds reasonable.
By watching for changes to files, you now know which variables
are impacted by a file being updated. Not sure when it is worth trying
to manually track when that information changes in the file vs use make
to recreate values anytime the file gets updated but I'd assume that
some effort of doing make's job would be lighter on load.
FreshPorts always wants to use make to extract data. It seems the only
way to ensure it has accurate information.
Alternatively,
you could save some runs if you can extract select variables such as
from Mk/bsd.default-versions.mk so that instead of having to rerun make
for every use of a variable from it, you could get the value from one
run of a variable and use it for all occurrences if you have accurate
non-substitution values stored and you tracked what file completes the
substitution.
That might miss other uses of that variable with the port. I'd really
want to run a full refrence
I was then attempting to use awk to process the debug output contents
but have kind of given up as awk is not making sense to me. Using an awk
script if I put match($4,"}") inside of a print statement I get a number
of the position 17 if $4=${PYTHON_DEFAULT} but if I try to save it to a
variable like
SubVarStart = match($4,"${")
SubVarEnd = match($4,"}")
then I end up with
SubVarStart='Parsing Makefile:2: PORTVERSION= ${PYTHON_DEFAULT}'
SubVarEnd=''
which is not the equivalent to just printing the match function's output
nor can I make any sense of what I am logically getting instead. If I
cannot store the function's output then I cannot use the function as far
as I am aware of.
Not sure if awk was the right tool for the job or if it would get it
done alone or in how many passes but I assumed it would become much more
efficient to do text processing on a make run's debug output to both get
more details like the file that defines a variable and be able to walk
through subvariables at much higher speed.
For anyone trying to build off of this, $2 collects
/path/file.name:linenumber: , $3 collects VARIABLE (characters to set it
vary: '=', ':=', '?=' and $4 collects the value of the variable. I'd
guess $5+ may be relevant when looking at other things like variables
containing dependencies but I never went there yet.
That overloads my brain so early in the morning. I can't imagine how to
do that yet.
- what other files hold magic values such as *_DEFAULT
It was already brought up in this thread that there is a wider issue
of external values being used to do more than set a port's version. That
is why I was trying to approach it by collecting information about a
variable being defined against another variable, and then try to collect
where that variable comes from.
That seems to be the main block for this problem: What affects what?
I am still pursuing the "`make index` then compare against the DB" proposal. That may be a simple solution.
Thank you.
1 - list of *_DEFAULT values - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-1-the-default-values-from-bsd-default-versions
2 - https://github.com/FreshPorts/DefaultVersions/blob/main/populate_port_default_version_deps.py - with much help from Claude
3 - https://gist.github.com/dlangille/fec174122c9c8fe4be892de8021710ad#file-3-port_default_version_variable
On Thu, Sep 17, 2026, at 1:33 PM, Dan Langille wrote:An update...
On Thu, Sep 17, 2026, at 12:42 PM, Mathieu Arnold wrote:
On Thu, Sep 17, 2026 at 05:11:13PM +0200, Thierry Thomas wrote:
Le jeu. 17 sept. 26 |a 17:00:55 +0200, Dan Langille <dan@langille.org> >>>> |-crivait-a:
The creation of INDEX takes time. By the time it is ready, more commits have
occurred and INDEX is already out of date and some PORTVERSION values have
changed.
`make fetchindex` is fine.
Well, it is faster, but it will not be what `make index` would output,
as it's a previously generated INDEX that was geneated from another
ports tree.
That gave me an idea. I ran `make index` in a jail[1]. It took 747
seconds (or 12.5 minutes).
How does this sound for a strategy?
* periodic/cronjob inserts a job into the FreshPorts work queue
* freshports daemon (FPD) processes the queue in order, so commit
processing is not going on during that job
* FPD find the job, notes the time, and launches a background job to
`make index` (MI)
* FPD continues on with the queue
* ...
* 12.5 minutes later, as the MI job completes, it inserts a reconcile
job into the queue
* when FPD gets to the reconcile job, it knows the start time
associated with the MI
* FPD compares the INDEX against the database and compiles a list of
ports which don't match INDEX
* any ports with commits after that MI start time are not included in
that list
* The ports on that list are refreshed from their Makefiles (because
more than just PORTVERSION can change) in the usual way that FP handles
a commit
* FPD continues on with the queue
* ...
One issue to overcome with INDEX: multiple packages for the same origin.
If the versions are the same, no issue, but when they differ...
Which one is correct?
For example:
py311-tkinter-3.11.16_11
py312-tkinter-3.12.14_11
py313-tkinter-3.13.15_11
py313t-tkinter-3.13.15_11
py314-tkinter-3.14.7_11
py314t-tkinter-3.14.7_11
py315-tkinter-3.15.0.r2_11
All from x11-toolkits/py-tkinter
Which one should FreshPorts choose as the correct one? The existing practice is to use `make -V PORTVERSION`
This seems to affect only these ports:
databases/py-gdbm
databases/py-sqlite3
devel/libclc
devel/opencl-clang
devel/spirv-llvm-translator
x11-servers/xorg-server
Full output at https://gist.github.com/dlangille/105f15e6e0c219e1ef0fd747398ae8bf
Next, I'll load that data into a FreshPorts database, and see what differences I can pull back from a query.
That's all I have for now. I'll come back to this exercise later.
On Thu, Sep 17, 2026, at 6:01 PM, Dan Langille wrote:
On Thu, Sep 17, 2026, at 1:33 PM, Dan Langille wrote:
On Thu, Sep 17, 2026, at 12:42 PM, Mathieu Arnold wrote:
On Thu, Sep 17, 2026 at 05:11:13PM +0200, Thierry Thomas wrote:
Le jeu. 17 sept. 26 |a 17:00:55 +0200, Dan Langille <dan@langille.org> >>>>> |-crivait-a:
The creation of INDEX takes time. By the time it is ready, more commits have
occurred and INDEX is already out of date and some PORTVERSION values have
changed.
`make fetchindex` is fine.
Well, it is faster, but it will not be what `make index` would output, >>>> as it's a previously generated INDEX that was geneated from another
ports tree.
That gave me an idea. I ran `make index` in a jail[1]. It took 747
seconds (or 12.5 minutes).
How does this sound for a strategy?
* periodic/cronjob inserts a job into the FreshPorts work queue
* freshports daemon (FPD) processes the queue in order, so commit
processing is not going on during that job
* FPD find the job, notes the time, and launches a background job to
`make index` (MI)
* FPD continues on with the queue
* ...
* 12.5 minutes later, as the MI job completes, it inserts a reconcile
job into the queue
* when FPD gets to the reconcile job, it knows the start time
associated with the MI
* FPD compares the INDEX against the database and compiles a list of
ports which don't match INDEX
* any ports with commits after that MI start time are not included in
that list
* The ports on that list are refreshed from their Makefiles (because
more than just PORTVERSION can change) in the usual way that FP handles >>> a commit
* FPD continues on with the queue
* ...
One issue to overcome with INDEX: multiple packages for the same origin.
If the versions are the same, no issue, but when they differ...
Which one is correct?
For example:
py311-tkinter-3.11.16_11
py312-tkinter-3.12.14_11
py313-tkinter-3.13.15_11
py313t-tkinter-3.13.15_11
py314-tkinter-3.14.7_11
py314t-tkinter-3.14.7_11
py315-tkinter-3.15.0.r2_11
All from x11-toolkits/py-tkinter
Which one should FreshPorts choose as the correct one? The existing
practice is to use `make -V PORTVERSION`
This seems to affect only these ports:
databases/py-gdbm
databases/py-sqlite3
devel/libclc
devel/opencl-clang
devel/spirv-llvm-translator
x11-servers/xorg-server
Full output at
https://gist.github.com/dlangille/105f15e6e0c219e1ef0fd747398ae8bf
Next, I'll load that data into a FreshPorts database, and see what
differences I can pull back from a query.
That's all I have for now. I'll come back to this exercise later.
An update...
I have a script which compares INDEX against the FreshPorts database.
Once fetched[1], it takes about 17 seconds to produce a list of ports
which will be refreshed.
Partial output is shown here. I have not checked the list. However, I
have confirmed accessibility/kdeaccessibility needs a refresh[2]. The
full output is in a gist[3].
Next tasks:
* refresh the ports in refresh.txt
* rerun the script - refresh.txt should then be empty
Thank you
$ time ./compare-index.sh
./compare-index.sh: comparing against /jails/freshports/usr/ports/INDEX-15 index_pkgversions.py: 38851 rows, 35205 origins, 0 malformed lines ./compare-index.sh: 38851 packages read from the INDEX
./compare-index.sh: 38851 rows loaded and compared
144 /var/db/freshports/cache/spooling/refresh.txt
0 /var/db/freshports/cache/spooling/not-in-index.txt
0 /var/db/freshports/cache/spooling/not-in-freshports.txt
18.30 real 0.40 user 0.12 sys
$ cat /var/db/freshports/cache/spooling/refresh.txt accessibility/kdeaccessibility
audio/pulseaudio-qt
databases/py-gdbm
databases/py-sqlite3
deskutils/kdepim
devel/boost-docs
devel/boost-jam
...
x11/nvidia-kmod
x11/plasma6-plasma
x11/plasma6-plasma-workspace
1 - INDEX is fetched rather than built for two main reasons:
* the repo FreshPorts uses is in a jail. It has no installed packages,
and no network access.
* `make fetchindex` is faster
2 - https://www.freshports.org/accessibility/kdeaccessibility/ shows 25.08.0
The ports tree shows:
[14:41 mydev dvl ~] % make -C /usr/ports/accessibility/kdeaccessibility
-V PORTVERSION
26.08.1
3 - https://gist.github.com/dlangille/74f5de44b4b8534674d45e35e2a0cb24So far, I think the code has proven this is feasible. More information than you wish you knew:
| Sysop: | Amessyroom |
|---|---|
| Location: | Fayetteville, NC |
| Users: | 74 |
| Nodes: | 6 (0 / 6) |
| Uptime: | 02:23:38 |
| Calls: | 1,194 |
| Files: | 1,353 |
| D/L today: |
2 files (1,590K bytes) |
| Messages: | 291,157 |