• Re: Poudriere: when fetching a package with porttree from a git source, the generation of logs is slow

    From Roger Marquis@marquis@roble.com to muc.lists.freebsd.ports on Sat Apr 11 08:06:23 2026
    From Newsgroup: muc.lists.freebsd.ports

    On Thu, 9 Apr 2026, Mark Millard wrote:
    My understanding was that when the ports tree ( /usr/ports/ above ) does
    not match the upstream version that can be fetched, including for any dependency mismatches or option mismatches ...

    Is this new behavior for 'make fetch'? Asking because we recently have
    had a relatively large number of poudriere fetch failures, most without sufficient logging of the cause. As we reuse /usr/ports/distfiles
    across architectures and versions it would be helpful if these
    intentional but unnecessary conditional failures could be overridden.

    Roger Marquis


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.21f-Linux NewsLink 1.2
  • From Mark Millard@marklmi@yahoo.com to muc.lists.freebsd.ports on Sat Apr 11 18:07:06 2026
    From Newsgroup: muc.lists.freebsd.ports

    On 4/11/26 17:31, Tatsuki Makino wrote:
    Hello.

    On 2026/04/12 0:06, Roger Marquis wrote:
    Is this new behavior for 'make fetch'?
    It is about the -b of poudriere bulk.
    In the current version, it seems that the following command is allowed
    to run even with poudriere, rather than poudriere-devel.

    poudriere bulk -j src -p git -z pkgfetch -b "http://localhost/packages/ src-default/" category/portname

    The -b option allows non-official packages to be fetched using values containing colons like this.
    And by setting up a web server to publish the packages created by
    poudriere, it can be used so that the packages are copied to other jail- porttree-set.
    This URL hosts a package created by poudriere bulk -j src -p default.

    And as an example of how to use it, the one I used recently is like this.

    poudriere bulk -j src -p default audio/libopenshot-audio multimedia/ libopenshot@all multimedia/openshot@all
    poudriere pkgclean -j src -p git -z pkgfetch ports-mgmt/pkg
    poudriere bulk -j src -p git -z pkgfetch ports-mgmt/pkg
    poudriere bulk -j src -p git -z pkgfetch -b "http://localhost/packages/ src-default/" -C -t audio/libopenshot-audio
    poudriere bulk -j src -p git -z pkgfetch -b "http://localhost/packages/ src-default/" -C -t multimedia/libopenshot@all
    poudriere bulk -j src -p git -z pkgfetch -b "http://localhost/packages/ src-default/" -C -t multimedia/openshot@all

    The key is to use poudriere pkgclean, delete everything except the pkg,
    and then let it fetch.
    And, for some reason, only pkg cannot be fetched, so we will run a check
    once to see if ports-mgmt/pkg exists.
    Furthermore, packages whose dependencies have been broken will be
    rebuilt, so the construction proceeds in multiple steps from the roots
    of the graph.

    So, since the fetch for this example package amounts to hundreds, I
    would like to avoid digging through the git log :)

    If I read the poudriere-devel code correctly (it is what I have
    installed), you have control of that through the likes of using:

    # poudriere -s GIT_TREE_DIRTY_CHECK=no bulk . . .

    "man poudriere" reports:

    -s var Inherit the default for var from the environment.


    -s var=value
    Set the default for var to the given value.

    There are a bunch of defaults defined (including GIT_TREE_DIRTY_CHECK):

    # grep ^: /usr/local/share/poudriere/common.sh | wc -l
    106

    (That may be an over or under count, but it gives an idea.)

    For reference:

    # grep GIT_TREE_DIRTY_CHECK /usr/local/share/poudriere/common.sh
    case "${GIT_TREE_DIRTY_CHECK-}" in
    : ${GIT_TREE_DIRTY_CHECK:=yes}


    and the code using it is:

    gghd_git_modified=no
    msg_n "Inspecting ${git_dir} for modifications to git checkout..."
    case "${GIT_TREE_DIRTY_CHECK-}" in
    no)
    gghd_git_modified=unknown
    ;;
    *)
    if git_tree_dirty "${git_dir:?}" "${inport}"; then
    gghd_git_modified=yes
    fi
    ;;
    esac
    echo " ${gghd_git_modified}"
    setvar "${gghd_git_modified_var}" "${gghd_git_modified}"

    So "no" avoids the get_tree_dirty activity.

    I'm not aware of GIT_TREE_DIRTY_CHECK being publicly documented, so it
    may be more subject to change/removal than things that are documented.


    Regards.



    --
    ===
    Mark Millard
    marklmi at yahoo.com


    --
    Posted automagically by a mail2news gateway at muc.de e.V.
    Please direct questions, flames, donations, etc. to news-admin@muc.de
    --- Synchronet 3.21f-Linux NewsLink 1.2