• Package Docs From Wiki

    From Ahmad Khalifa@21:1/5 to All on Sun Jan 19 21:10:01 2025
    Hi,

    I'm trying to package FreeBASIC along with its documentation. However,
    the documentation is built from a download of their Wiki [1].

    How to deal with this situation, I could only think of...

    1. Package the generated html in debian/ directly. This would work, but
    I'm not sure it's good idea or not. Not worried about the effort to
    maintain this, but more about whether it's been done before and common
    or not.

    2. Package the source of the html, which is wiki markup. This has all
    the negatives of #1 along with the fact that compiling the wiki markup
    is done using an included BASIC tool which has to be compiled using the compiler being packaged.

    3. Package the documentation compiler tool itself with a maintainer
    script to download/compile the html? The documentation is important, but
    not this important.

    4. Skip documentation - it's just not redistributable.


    Any views or ideas on this? Example projects would be appreciated.


    1. https://www.freebasic.net/wiki/DocToc

    --
    Regards,
    Ahmad

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_Plissonneau_Duqu=C@21:1/5 to All on Mon Jan 20 09:30:02 2025
    Hi Ahmad,

    Le 2025-01-19 21:01, Ahmad Khalifa a écrit :

    Any views or ideas on this? Example projects would be appreciated.

    Are they already distributing an archive of the generated documentation?
    If this is the case, and the license allows it, you could use it as the
    source for your documentation package, though this is not the most
    satisfying way to deal with the issue.

    Otherwise, are the documentation generation scripts already included in
    your freebasic package(s)? And do they download the generated HTML or
    the wiki source from the wiki?

    One possible way to work around that would be to just change the
    generation scripts to use a local "mirror" of the wiki instead of
    downloading it, and include the mirrored source data into your source
    package (ideally as a separate source archive, providing a script or
    d/rules target to generate it, and figuring out a versioning scheme that
    would allow to make it reproducible).

    Cheers,

    --
    Julien Plissonneau Duquène

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Joost van =?utf-8?Q?Baal-Ili=C4=87?@21:1/5 to Ahmad Khalifa on Mon Jan 20 11:00:01 2025
    Hi Ahmad,

    On Sun, Jan 19, 2025 at 08:01:09PM +0000, Ahmad Khalifa wrote:

    I'm trying to package FreeBASIC along with its documentation. However, the documentation is built from a download of their Wiki [1].

    How to deal with this situation, I could only think of...

    1. Package the generated html in debian/ directly. This would work, but I'm not sure it's good idea or not. Not worried about the effort to maintain this, but more about whether it's been done before and common or not.

    2. Package the source of the html, which is wiki markup. This has all the negatives of #1 along with the fact that compiling the wiki markup is done using an included BASIC tool which has to be compiled using the compiler being packaged.

    3. Package the documentation compiler tool itself with a maintainer script
    to download/compile the html? The documentation is important, but not this important.

    4. Skip documentation - it's just not redistributable.


    Any views or ideas on this? Example projects would be appreciated.


    1. https://www.freebasic.net/wiki/DocToc


    I solved a related problem for a non-free package with https://salsa.debian.org/debian/publicfile/-/blob/master/debian/get-publicfile-docs
    : to be called by the user at runtime.

    Not quite sure if such a hack would be suitable in your situation though...

    The solution as suggested by Julien would be more clean.

    Anyway: HTH! Bye,

    Joost

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ahmad Khalifa@21:1/5 to All on Mon Jan 20 21:40:01 2025
    On 20/01/2025 09:34, Joost van Baal-Ilić wrote:
    Hi Ahmad,
    I solved a related problem for a non-free package with https://salsa.debian.org/debian/publicfile/-/blob/master/debian/get-publicfile-docs
    : to be called by the user at runtime.

    Not quite sure if such a hack would be suitable in your situation though...

    This is pretty useful, this will be my plan B. Learned something new
    reading that root/non-root switch :)

    --
    Regards,
    Ahmad

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ahmad Khalifa@21:1/5 to All on Mon Jan 20 21:30:01 2025
    On 20/01/2025 08:21, Julien Plissonneau Duquène wrote:
    Hi Ahmad,

    Le 2025-01-19 21:01, Ahmad Khalifa a écrit :

    Any views or ideas on this? Example projects would be appreciated.

    Are they already distributing an archive of the generated documentation?
    If this is the case, and the license allows it, you could use it as the source for your documentation package, though this is not the most
    satisfying way to deal with the issue.

    Oh, yes they do have a tarball of the html!
    It's not on git so I never considered it, but that solves it.

    Does this make it 1 source package with 2 orig tarballs then, or should
    I split it completely into 2 source packages?
    Perhaps one source won't work on git unless I switch to a plain repo
    based on tarballs.


    Otherwise, are the documentation generation scripts already included in
    your freebasic package(s)? And do they download the generated HTML or
    the wiki source from the wiki?

    Yes, it's a compiled tool written in freebasic that can download the
    wiki source to a cache dir, then use that cache dir to build html/chm/txt/fbhelp formats. It can run on the cache only, but the tool currently requires pcre3 (libpcre.so) so I only managed to make it work
    locally outside sbuild with lots of manual hacks.

    I think the idea above is much cleaner even if I have to track different
    git repos and tarballs.



    --
    Regards,
    Ahmad

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_Plissonneau_Duqu=C@21:1/5 to All on Tue Jan 21 11:30:02 2025
    Hi Ahmad,

    Le 2025-01-20 21:29, Ahmad Khalifa a écrit :

    Does this make it 1 source package with 2 orig tarballs then, or should
    I split it completely into 2 source packages?
    Perhaps one source won't work on git unless I switch to a plain repo
    based on tarballs.

    It will probably be easier to manage that with a separate source
    package, as their builds are completely independant anyway.

    Yes, it's a compiled tool written in freebasic that can download the
    wiki source to a cache dir, then use that cache dir to build html/chm/txt/fbhelp formats. It can run on the cache only, but the tool currently requires pcre3 (libpcre.so) so I only managed to make it work locally outside sbuild with lots of manual hacks.

    I think the idea above is much cleaner even if I have to track
    different git repos and tarballs.

    It's a bit borderline as it's generated from a source code that will not
    be part of the source package, but on the other hand it avoids the
    difficult issue of guessing which revision of each page is relevant to
    which released version while downloading the sources. To make that
    really clean, it would help if upstream could provide versioned archives
    of the source and generator code. Anyway in the meantime the source
    requirement of the DFSG specifically mentions "the program", html
    documentation (excluding embedded scripts) is not a program, it can
    still be manually edited, and could probably be converted back to its
    wiki source form if a tool is written for that, so I believe that it's
    fine to proceed that way.

    Cheers,

    --
    Julien Plissonneau Duquène

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ahmad Khalifa@21:1/5 to All on Tue Jan 21 19:30:01 2025
    On 21/01/2025 10:25, Julien Plissonneau Duquène wrote:
    Le 2025-01-20 21:29, Ahmad Khalifa a écrit :

    Does this make it 1 source package with 2 orig tarballs then, or
    should I split it completely into 2 source packages?
    Perhaps one source won't work on git unless I switch to a plain repo
    based on tarballs.

    It will probably be easier to manage that with a separate source
    package, as their builds are completely independant anyway.

    I'll play around with 2 sources and 2 branches on 1 salsa repo. See if
    that looks good.

    Yes, it's a compiled tool written in freebasic that can download the
    wiki source to a cache dir, then use that cache dir to build html/chm/
    txt/fbhelp formats. It can run on the cache only, but the tool
    currently requires pcre3 (libpcre.so) so I only managed to make it
    work locally outside sbuild with lots of manual hacks.

    I think the idea above is much cleaner even if I have to track
    different git repos and tarballs.

    It's a bit borderline as it's generated from a source code that will not
    be part of the source package, but on the other hand it avoids the
    difficult issue of guessing which revision of each page is relevant to
    which released version while downloading the sources. To make that
    really clean, it would help if upstream could provide versioned archives
    of the source and generator code. Anyway in the meantime the source requirement of the DFSG specifically mentions "the program", html documentation (excluding embedded scripts) is not a program, it can
    still be manually edited, and could probably be converted back to its
    wiki source form if a tool is written for that, so I believe that it's
    fine to proceed that way.

    I always thought of html as a source markup until lintian complained
    about source-is-missing.

    Upstream has several versioned tarballs (on sf.net [1]), but they don't
    package or even document the doc tools. They probably consider it too
    much plumbing for anyone to use. Anyway for it to be compiled on
    testing, it needs old regex api (libpcre), so first thing is maybe to
    patch the tool to use pcre2 regex.

    Thanks for the help.


    1. https://sourceforge.net/projects/fbc/files/FreeBASIC-1.10.1/


    --
    Regards,
    Ahmad

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_Plissonneau_Duqu=C@21:1/5 to All on Tue Jan 21 21:40:01 2025
    Hi Ahmad,

    Le 2025-01-21 19:28, Ahmad Khalifa a écrit :

    I'll play around with 2 sources and 2 branches on 1 salsa repo. See if
    that looks good.

    This is not a good idea, as a separate source package it should have its
    own separate repository. That's much less confusing for some standard
    tools and for other people that may have to work on the package later.

    I always thought of html as a source markup until lintian complained
    about source-is-missing.

    Even when it's generated that lintian error can be irrelevant: sometimes upstream provide generated reports (e.g. benchmarks or compatibility)
    that are helpful but can't be easily reproduced or could be generated by non-free tools even though the produced documents are free.

    Upstream has several versioned tarballs (on sf.net [1]), but they don't package or even document the doc tools. They probably consider it too
    much plumbing for anyone to use. Anyway for it to be compiled on
    testing, it needs old regex api (libpcre), so first thing is maybe to
    patch the tool to use pcre2 regex.

    That's something you would have to explicitly ask upstream to provide
    (e.g. through an issue or feature request), by explaining to them that
    this will make the long-term maintenance of the documentation easier,
    e.g. when features or bugfixes are backported from a newer version into
    a stable version.

    Thank you for working on this.

    Cheers,

    --
    Julien Plissonneau Duquène

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ahmad Khalifa@21:1/5 to All on Wed Jan 22 00:50:01 2025
    On 21/01/2025 20:33, Julien Plissonneau Duquène wrote:
    Hi Ahmad,

    Le 2025-01-21 19:28, Ahmad Khalifa a écrit :

    I'll play around with 2 sources and 2 branches on 1 salsa repo. See if
    that looks good.

    This is not a good idea, as a separate source package it should have its
    own separate repository. That's much less confusing for some standard
    tools and for other people that may have to work on the package later.

    Yeah, working on it just now, I found myself using two separate dirs
    anyway. I didn't want to keep switching branches and stashing changes,
    so agreed, two source, two repos.


    --
    Regards,
    Ahmad

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)