This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------bY2ZOAfTrkfY5Fe5TTYeExVu
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On 11/25/24 12:15 PM, Tim Harder wrote:

Hi all,

I've been working for a while on pkgcraft[1] and related tooling that
could have the potential to run various CI checks server-side as a pre-receive git hook verifying commits. Previously, this wasn't at all possible due to portage/repoman and pkgcore/pkgcheck being too slow to feasibly run and give feedback within a second or two of interactively pushing.

[...]

From all this, my main questions to the dev community are the following:

- Do you see value in running a service that can reject commits due to
  issues like invalid metadata during `git push`?

Yes please! This would be fantastic to have. Already, pushing is not
quite instantaneous since the hooks wait on e.g. pushing again to
mirrors, and I don't think waiting a couple seconds for significantly
greater peace of mind is at all a bad thing.

- Would the project be open to moving to a merge queue model?

At least for me, no. I don't think it's necessary as we can already see
huge reliability benefits without it, and the merge commit / signing
issues are what I'd personally consider to be an instant dealbreaker.

So I'd say there's a lot to lose and not much to gain.

- Are there others who would be interested in helping with the
  development, testing, and maintenance of a git-hook service if that is
  the chosen path?

I'd be happy at minimum to help test it, although in principle I'm not uninterested in the rest.


--
Eli Schwartz

--------------bY2ZOAfTrkfY5Fe5TTYeExVu--

-----BEGIN PGP SIGNATURE-----

wnsEABYIACMWIQTnFNnmK0TPZHnXm3qEp9ErcA0vVwUCZ9HxqAUDAAAAAAAKCRCEp9ErcA0vV2UF AP4nf90jS7Soc4+gfHdgd86bMR2JExDqxw20tSNbyih0EQEAkS6e1Tpzw2oM5vWtTklDbKUGud5L WBtbAYzwtMaGww0=
=495B
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi,

Sorry for the delayed reply, I was reminded of this email and
realised that I never did respond to it while we were discussing
git hooks today.

On 13/3/25 06:42, Eli Schwartz wrote:

On 11/25/24 12:15 PM, Tim Harder wrote:

From all this, my main questions to the dev community are the following:

- Do you see value in running a service that can reject commits due to
  issues like invalid metadata during `git push`?

Yes please! This would be fantastic to have. Already, pushing is not
quite instantaneous since the hooks wait on e.g. pushing again to
mirrors, and I don't think waiting a couple seconds for significantly
greater peace of mind is at all a bad thing.

I spent quite a bit of time fixing various QA issues (including tons of whitespace) reported by pkgcruft today, and the only thing better than
fixing QA issues is having them not make it into the repos in the first
place!

Would you be open to trialling this on GURU? It seems like a reasonable
target where it'd seriously reduce the amount of developer / trusted contributor overhead when merging commits.

If so, what would that process look like, and how would we go about
getting the implementation started?

Thanks for your work on QA tools so far!

Cheers,

Matt

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-05-21 Wed 03:57, Matt Jolly wrote:

Would you be open to trialling this on GURU? It seems like a reasonable >target where it'd seriously reduce the amount of developer / trusted >contributor overhead when merging commits.

It depends on how the infra side of GURU is set up. If it's semi-similar
to how the gentoo repo is handled I imagine it could be done, but note
that this service requires a decent amount of power behind it so the
latency between pushing commits and getting them accepted is minimal.
While it is quite a bit faster than repoman or pkgcheck, it still relies
a lot on bash which isn't known for speed in general. I'm not sure what
infra is able to provide in terms of power to make it feasible or not.

If so, what would that process look like, and how would we go about
getting the implementation started?

I've already hacked up a rough proof-of-concept (see my other recent
reply to this thread) that can be found at [1].

Thanks,
Tim

[1]: https://github.com/pkgcraft/pkgcraft/tree/main/crates/pkgcruft-git

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

For anyone interested,

I've hacked up an initial, rough implementation of a pkgcruft-git
service that enables verifying ebuild commits during git's pre-receive
hook phase.

Currently it comes with a simple script that runs a local demo instance targeting the gentoo repo by default.

You can find the code and demo instructions in the pkgcruft-git crate
directory of the pkgcraft repo [1].

If I can find the time, I'll try to write up a dev blog post about the
service design in more detail as well as next steps if there is interest
in trying to use it in any official fashion in the future.

Thanks,
Tim

[1]: https://github.com/pkgcraft/pkgcraft/tree/main/crates/pkgcruft-git

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 30/05/25 10:46, Tim Harder wrote:

For anyone interested,

I've hacked up an initial, rough implementation of a pkgcruft-git
service that enables verifying ebuild commits during git's pre-receive
hook phase.

Currently it comes with a simple script that runs a local demo instance targeting the gentoo repo by default.

You can find the code and demo instructions in the pkgcruft-git crate directory of the pkgcraft repo [1].

If I can find the time, I'll try to write up a dev blog post about the service design in more detail as well as next steps if there is interest
in trying to use it in any official fashion in the future.

Thanks,
Tim

[1]: https://github.com/pkgcraft/pkgcraft/tree/main/crates/pkgcruft-git

It would be very nice to have it run over the PRs we are already
receiving on github, personally I prefer a rebase-only workflow so merge
queue that imply actual git merges with all the problems they bring on bisecting and reading the history are an hard no for me.

lu

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-05-30 Fri 09:23, Luca Barbato wrote:

It would be very nice to have it run over the PRs we are already
receiving on github...

This requires different functionality than what is done for pre-receive
hook support more in line with what the pkgcruft github action[1]
already provides.

Of course, if it was integrated with pkgcruft-git it could be more
efficient to support features such as scan diffing, i.e. only showing
new results introduced by a PR, which the action already provides using
cache files.

On the other hand, the core functionality of pkgcraft and pkgcruft are
probably efficient enough to run directly on github CI for that task in
a reasonable amount of time if limited to the target packages assuming
the reports snapshot required for diffing is requested from an external pkgcruft-git service.

In either case, before adding to the ever-growing mountain of potential
work, I'd prefer to see if the base git hook functionality is useful in
any fashion or what else it needs to become so.

Thanks,
Tim

[1]: https://github.com/pkgcraft/pkgcruft-action

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Tim Harder <radhermit@gentoo.org> writes:

For anyone interested,

I've hacked up an initial, rough implementation of a pkgcruft-git
service that enables verifying ebuild commits during git's pre-receive
hook phase.

Currently it comes with a simple script that runs a local demo instance targeting the gentoo repo by default.

You can find the code and demo instructions in the pkgcruft-git crate directory of the pkgcraft repo [1].

I've packaged this now as dev-vcs/pkgcruft-git (live-only for now). I'm
using the pre-push support you added (thank you!) locally with the
following in .git/hooks/pre-push:

pkgcruft-git-pre-push origin git+ssh://git@git.gentoo.org/repo/gentoo.git

and going from there. I encourage others to give it a go and share
feedback too.

If I can find the time, I'll try to write up a dev blog post about the service design in more detail as well as next steps if there is interest
in trying to use it in any official fashion in the future.

Definitely. My hope is we can get this deployed on infra, but I think
it's going to be really valuable even without that locally.

Great work!

Thanks,
Tim

[1]: https://github.com/pkgcraft/pkgcraft/tree/main/crates/pkgcruft-git

thanks,
sam

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-06-02 Mon 21:57, Sam James wrote:

I've packaged this now as dev-vcs/pkgcruft-git (live-only for now). I'm
using the pre-push support you added (thank you!) locally with the
following in .git/hooks/pre-push:

pkgcruft-git-pre-push origin git+ssh://git@git.gentoo.org/repo/gentoo.git

and going from there. I encourage others to give it a go and share
feedback too.

I think it should also be possible to just symlink the binary into the
pre-push hooks dir or as the pre-push file itself since that's how I was testing it.

If I can find the time, I'll try to write up a dev blog post about the
service design in more detail as well as next steps if there is interest
in trying to use it in any official fashion in the future.

Definitely. My hope is we can get this deployed on infra, but I think
it's going to be really valuable even without that locally.

If infra (or anyone else) needs help, has questions, or requests... feel
free to ping me or open discussions/issues upstream. I won't always have
as much availability as I currently do for the next couple weeks, but
I'll try to respond when I can.

Thanks,
Tim

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)