Hello.

I try to setup a repository with a similar structure as Debian's
original, just smaller. In the Debian repository, e.g. bookworm main[1]
there are a lot Contents-*.gz files. My repository is created with
reprepro following the wiki[2]. But I cannot find information about the
missing Contents-* files. What is the intended workflow to create them?

The files of my packages are not included in the database of apt-file,
the mandatory ``apt-file update'' does not help to find files or show
files in my packages.

I guess the problems are related.

What can I do to solve those issues?

[1] https://ftp.debian.org/debian/dists/bookworm/main/
[2] https://wiki.debian.org/DebianRepository/SetupWithReprepro

--
kind regards
Frank

-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE86z15c6qwvuAkhy+zDIN/uu9BloFAmdi3iwACgkQzDIN/uu9 BlpHgQv/WeBkxVnTbVnGTVISWIP1hkWr9x2ATR+/0pYRp8E18b+YDXwoXbzkBkfp m1frxksZvH9BpaXQ5q0IEqal8muqNZaVKY+oXIKH2wifiwSmUbzTuAFZTdYoWdwE FRSt13V1cxy0GvZU3vg1E7OgdGzmUGTBBrBOVmYBcbCUqHjRkG5xUH7r9e2VGvMu aVdpeKSMxundLoocZv9ighBwpLZ4YnYSeV1mlTKyturY6XDKJi35WQbacx8htp6N xJw9WSKv3U3xbZxq5ct3DkkvT4SHvlZ6Z+aIIVPnweudxgmo1Cs+6uhRnAvjQfK/ 2GXaNToSulvjG6mmGl5CveZl2Upbc4CINlBvk1JH6+wwHRqMOL7uh3gFoGrBcp7/ EGCvcG2zjHF56b6bHZg81y15K4LnzuMm+a0ynUkqpTwlJ3MCrhCqshbOcCxrunIb 2Y0BAQ98EvaN/ILtiWZ9vKVNHlI/LcjstTpQjvkpgT4wYstx/VSh1A9opjEQzujo
EcfHga9O
=T9Ln
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 18 Dec 2024, Frank Guthausen wrote:

Hello.

I try to setup a repository with a similar structure as Debian's
original, just smaller. In the Debian repository, e.g. bookworm main[1]
there are a lot Contents-*.gz files. My repository is created with
reprepro following the wiki[2]. But I cannot find information about the missing Contents-* files. What is the intended workflow to create them?

The files of my packages are not included in the database of apt-file,
the mandatory ``apt-file update'' does not help to find files or show
files in my packages.

I guess the problems are related.

What can I do to solve those issues?

[1] https://ftp.debian.org/debian/dists/bookworm/main/
[2] https://wiki.debian.org/DebianRepository/SetupWithReprepro

I use apt-ftp-archive to do this so I can't help you with reprepo but
the first thing I'd check is whether the website looks reasonable.

I'm pretty sure the contents files aren't necessary for a working repo
(and I think apt-ftp-archive at least limits how much delta there is to
these files so they don't necessarily generate straight away)

I'm not sure which point is going wrong so below I've put the steps of
browsing the repo that needs to be available - if this structure all
exists but the Packages file is empty then it's presumably a reprepo problem.

So I have this sources file:

Types: deb
URIs: http://aptmirror.home.woodall.me.uk/local
Suites: bookworm
Components: main

when I browse to that website (not publicly available) I then see two directories /local/dists/ and /local/pool/
(I've dropped the http://apt...uk from all of the links and just
included the path)

/local/dists/bookworm/InRelease will need to have lines something like:
342869da630a775fe229d4bc2ebe09ff 246670 main/binary-amd64/Packages
82a2348876c6d58709fe6c928f05ba07 56864 main/binary-amd64/Packages.xz

And then those files need to exist here in my case (relative to the
InRelease file):
/local/dists/bookworm/main/binary-amd64/


In the Packages file you will to see the packages listed, here's one
example from mine:

Package: apt-cacher-ng
Architecture: amd64
Version: 3.7.5-1+~tjw12r1
Priority: net
Section: optional
Maintainer: Eduard Bloch <blade@debian.org>
Installed-Size: 1833
Pre-Depends: dpkg (>= 1.15.6)
Depends: libbz2-1.0, libc-ares2 (>= 1.16.0), libc6 (>= 2.34), libevent-2.1-7 (>= 2.1.8-stable), libevent-pthreads-2.1-7 (>= 2.1.8-stable), libfuse2 (>= 2.6), libgcc-s1 (>= 3.0), liblzma5 (>= 5.1.1alpha+20120614), libssl3 (>= 3.0.0), libstdc++6 (>= 12),
libsystemd0 (>= 210), libwrap0 (>= 7.6-4~), zlib1g (>= 1:1.2.0), debconf (>= 0.5) | debconf-2.0, adduser, sysvinit-utils (>= 3.05-4~) | lsb-base (>> 3.0-6)
Recommends: ca-certificates
Suggests: doc-base, avahi-daemon
Conflicts: logrotate (<< 3.8.0)
Filename: pool/main/a/apt-cacher-ng/apt-cacher-ng_3.7.5-1+~tjw12r1_amd64.deb Size: 684648
MD5sum: 1e07511c47939f7deed5f9f1b4a20dfb
SHA1: 187491c9490c4449b17c9d6acd9c3afcd380e340
SHA256: 4748419918c5093623df26158c5c2fab4a7fb64c220c3869b6d64636c04c97cf SHA512: 5c45eeb0e6f1a4597252e085bfcd481a95732e622b454b3082ebd2abf42723454f425c8fdff73efd7d3ac066969701502fea7c46c16546e8f8c5b8459c22bee4
Homepage: http://www.unix-ag.uni-kl.de/~bloch/acng/
Description: caching proxy server for software repositories
Apt-Cacher NG is a caching proxy for downloading packages from Debian-style
software repositories (or possibly from other types).
.
The main principle is that a central machine hosts the proxy for a local
network, and clients configure their APT setup to download through it.
Apt-Cacher NG keeps a copy of all useful data that passes through it, and when
a similar request is made, the cached copy of the data is delivered without
being re-downloaded.
.
Apt-Cacher NG has been designed from scratch as a replacement for
apt-cacher, but with a focus on maximizing throughput with low system
resource requirements. It can also be used as replacement for apt-proxy and
approx with no need to modify clients' sources.list files.

and then you see that Filename: line in there, that file needs to exist
too which I see when I browse to here:
/local/pool/main/a/apt-cacher-ng/


HTH.

Tim.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 18 Dec 2024 18:51:58 +0000 (GMT)
Tim Woodall <debianuser@woodall.me.uk> wrote:

On Wed, 18 Dec 2024, Frank Guthausen wrote:

The files of my packages are not included in the database of
apt-file, the mandatory ``apt-file update'' does not help to find
files or show files in my packages.

I guess the problems are related.

I'm pretty sure the contents files aren't necessary for a working repo
[...]

I'm not sure which point is going wrong [...]

The repository itself works fine. But I cannot use apt-file to show or
find content of packages. And all the Contents-*.gz files are missing.

I don't know any tool to generate them, and I don't know the workflow.
In Debian bookworm Release file[1] those files are included as visible
within e.g. the first 25 lines of this file.

[1] https://ftp.debian.org/debian/dists/bookworm/Release

--
kind regards
Frank

-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE86z15c6qwvuAkhy+zDIN/uu9BloFAmdjHtAACgkQzDIN/uu9 Blozmwv+I5GS4UBcoeaXrBXl1ry+aGdLkH9XQAKO/J6Ai44/hZE6FlrzyJYOKv4Q 6FpHrUdUr0pjrSJwXPXoUEWIMTH/JIPL4B20h5JUIj+GywlQa229RKht9gYMouvt l3SFXVT2igUS04NZpvFAwCCT3Fs3ZqNivjqqUuwAUu5Jhey4KT4AIquK6w7L0pHn pKMrE3zAq5eWUr7gvdy+WAj71mWXHyO43tcPa6jWhqdOQnQp3CuSEH5Pb9fBgD/n WXAVpGo7bWcCCCnpMtcPZJRH6gvJuw8nVqx1vXMBKUayNQMmasDvJLpHBM7M9CRW LrTgsUD8MwaOllq/zQZd0MiM0AB2UBXDI+if/xToDWRkOd28mfR/EGaytup0oUTu SrBfBgmRBE2k319OTNNtlmQO9G3Q8Is0cr8DIErKAPrJkgx4JRuVshtIGAwShvj7 CqYlk3e8UCqiCoaTxzuV6nYG3ESSqVobkmIxFdKApQbpRzhai+BTqA1Dskqn6sXA
V0g2C79s
=zF5t
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 18 Dec 2024 20:13:20 +0100
Frank Guthausen <fg.debian@shimps.de> wrote:

The repository itself works fine. But I cannot use apt-file to show or
find content of packages. And all the Contents-*.gz files are missing.

Peter Pentchev posted the solution in debian-devel[1]. It's a
configuration of reprepro, conf/distributions file needs a line
containing "Contents:" only. Now the files are generated and apt-file
can deal with the content of the packages.

[1] https://lists.debian.org/debian-devel/2024/12/msg00384.html

--
kind regards
Frank

-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE86z15c6qwvuAkhy+zDIN/uu9BloFAmdjJxIACgkQzDIN/uu9 Blpp3Av/dfy9fLwqqfuzFzxmBi5dY8bnnacDxtrEIHJtn87mjxRXVKVuGpywCux2 yHLWbekLqy9rPEFezqTfThl8XX+wyXQ/svDQJcEi21Lz1Fq9+pb5Tk2gX1hHXzgl S2zxJQCkECTWdExk1jjm/OipRzf0rDRG9To1V5qmDekd/Y4NDuIVhfvYEfPJaUv3 SjZjFubjgb0k8Dl4Nq4KuuR+Gw+nE9IdDAo8V3ldlUxXOOnPAtf6iYeYyge6fGVm V8P0hDFstVjHpDGH1p6wwDaLIJlFugZbPQD+OnqZy44emBLtXdQtRE0Mm5ZrnNru rDcV4adg+DcWanz1P6BAZ59za6/WuCL+1Y5+b2yRlo+jVxmMz3mIQVjjOjhV3XT2 kxTsk5yi1H1q7C7Lo3xY4Ygm3lQjTp3M8G7GKfNyCkCD2eYX+FwbwAp1MK9vw5vA 0Ilykfwi7XWeO+ZkSmRStlcbXl5653vCUSC8BxX+6AgTUdKpU91NBvtdRYFtAFZI
EM2BOzZ1
=xJxX
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 18 Dec 2024, Frank Guthausen wrote:

On Wed, 18 Dec 2024 18:51:58 +0000 (GMT)
Tim Woodall <debianuser@woodall.me.uk> wrote:

On Wed, 18 Dec 2024, Frank Guthausen wrote:

The files of my packages are not included in the database of
apt-file, the mandatory ``apt-file update'' does not help to find
files or show files in my packages.

I guess the problems are related.

I'm pretty sure the contents files aren't necessary for a working repo
[...]

I'm not sure which point is going wrong [...]

The repository itself works fine. But I cannot use apt-file to show or
find content of packages. And all the Contents-*.gz files are missing.

I don't know any tool to generate them, and I don't know the workflow.

apt-ftparchive generates them. It's not the easiest thing to use though.

And I see you asked on debian-devel and someone there suggested what the
config setting is for reprepo I think.

Tim.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 18 Dec 2024 19:57:42 +0000 (GMT)
Tim Woodall <debianuser@woodall.me.uk> wrote:

On Wed, 18 Dec 2024, Frank Guthausen wrote:

I don't know any tool to generate them, and I don't know the
workflow.

apt-ftparchive
generates them. It's not the easiest thing to use though.

My first run before reprepro was this, but without configuration and
deeper understanding. I aim not only for solutions, but for a deeper understanding. The tool apt-ftparchive didn't fullfill my ecpectations
and needs. Maybe a lack of patience, I don't know.

And I see you asked on debian-devel and someone there suggested what
the config setting is for reprepo I think.

Yes, the problem is solved. How can the apt-ftparchive tool support the
average admin expectacion and why is this not default? I'm glad someone
pointed me to the reprepro tool, and with Peter's help I've got the
conf which fullfills my dreams. I took care about the missing line
in the Debian wiki page[1]. I got the conf template from there. I
improved the wiki page with our results in a conservative way, hoping
to find agreement about the topic.

[1] https://wiki.debian.org/DebianRepository/SetupWithReprepro

--
kind regards
Frank

-----BEGIN PGP SIGNATURE-----

iQGzBAEBCgAdFiEE86z15c6qwvuAkhy+zDIN/uu9BloFAmdjTfMACgkQzDIN/uu9 BlqIpQwAl+95LcWsrVrgsUKF/bVIgKd2QmB8DMu8DxpyNIGVVSyW0OF8BCcSrqlv 6IFHdtwgr/zTT8lCtv72snnHGnzlPN3F4wnPp3rXbtKtrN//+4xk0PmRmOS8L7s4 h6ZCxXRwZVPp+ifTW5M83Z1MXJ2xjoKro2ni4VKXCZrMjoEzmfcRCOkyvoowStF1 YJgiGjw2lxhTxRqny6zmBq4GyaRZ0M9bhyN5vG/y/qkjjavPooF/ZN5Uajl96RgT 21h4Zy5bTqz2orhBSDE4LZqsJSMxZ+mu5mgr7FbN6y1RuT0Oy7buwV7nsMGu163E VYM2pQoA3UjkNzMwNDjb3oMgCcp34P7Tv+ndKxF6etJChTmAnt+9A74Ap6IjA4TY j07JJs6lJCMgY3+6CC4K7ThlMzSN2De3bmGkIw993/6XbwC0uMrgJbaoCg1W9gf+ pmfECmFAJe2gotJgTMtuJlymioVK8ghoQ0HIqd/mTocsJDRaZXn0bah7+dPLPctA
kEX1NN9i
=pG1y
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 25 Dec 2024 16:40:36 -0000 (UTC), etphonehomefrance@gmail.com wrote:

On 2024-12-18, Frank Guthausen <fg.debian@shimps.de> wrote:

I try to setup a repository with a similar structure as Debian's
original, just smaller. In the Debian repository, e.g. bookworm main[1] there are a lot Contents-*.gz files. My repository is created with
reprepro following the wiki[2]. But I cannot find information about the missing Contents-* files. What is the intended workflow to create them?

The files of my packages are not included in the database of apt-file,
the mandatory ``apt-file update'' does not help to find files or show
files in my packages.

I guess the problems are related.

What can I do to solve those issues?

[1] https://ftp.debian.org/debian/dists/bookworm/main/
[2] https://wiki.debian.org/DebianRepository/SetupWithReprepro

Creating and managing a repository with a structure similar to Debian's requires generating the Contents-*.gz files, which map the files provided by packages to their respective packages. These files are crucial for tools
like apt-file to function correctly. If you are using reprepro to manage
your repository, these files are not generated by default. Here's how you
can address this:

The Contents-*.gz files list all the files included in the packages of a particular section and architecture in the repository. These files allow tools like apt-file to index and search for files provided by packages.

reprepro is a lightweight tool and does not have built-in support for creating Contents-*.gz files. This is why you don't see these files in your repository and why apt-file update doesn't work as expected.

You can generate the Contents-*.gz files manually or use a script to
automate the process. Here’s a general workflow:

Use a script to extract the contents of the data.tar.* (or data.cpio.*) archives from the .deb files in your repository. For example:

for deb in pool/main/*/*.deb; do dpkg-deb -c "$deb" | awk '{print $6}' | sed "s|^|$(basename "$deb" .deb) |" done This will produce lines in the format package file, which map files to their packages.

Organize the output by architecture (e.g., amd64, arm64) and section (e.g., main, contrib).

For each architecture and section, write the organized output into a text file named Contents-<architecture>.gz. For example:

gzip -c > dists/bookworm/main/Contents-amd64.gz Step 4: Automate the Process You can create a shell or Python script to iterate through all architectures and sections in your repository, generating the required files.

After generating the Contents-*.gz files, ensure they are included in the Release file of your repository so that apt-file can detect them:

Update the Release file with the appropriate checksums for the Contents-*.gz files:

reprepro export Run apt update and then apt-file update on the client
system to test the functionality.

Here is a basic example script:

#!/bin/bash REPO_PATH="/path/to/your/repo" DIST="bookworm" SECTIONS=("main" "contrib" "non-free") ARCHITECTURES=("amd64" "arm64")

for SECTION in "${SECTIONS[@]}"; do for ARCH in "${ARCHITECTURES[@]}"; do CONTENTS_FILE="$REPO_PATH/dists/$DIST/$SECTION/Contents-$ARCH"

man reprepro -> conf/distributions -> Contents

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)