• [gentoo-dev] [PATCH 0/9] Python 3.13 freethreading support

    From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:00:01 2024
    Hello,

    Here's a patchset that adds support for Python 3.13 freethreading
    version.

    Originally, this was dev-lang/python:3.13[-gil]. However, since it is ABI-incompatible at extension level with the regular build of CPython,
    upstream decided to split it into a separate /usr/lib/python3.13t
    install tree, with its own .so suffix, wheel tag and so on. All things considered, it seems to make most sense to treat it separately in Gentoo
    as well.

    So for this, we do:

    1. Introduce dev-lang/python-freethreading that installs
    the freethreading version of CPython (:3.13t).

    2. Introduce 'python3_13t' Python target for it.

    Note that at this point freethreading support is limited. Some packages
    do support it, others may work incidentally. Right now, the stable ABI
    is not supported at all. Hopefully, this will be improved
    in the future (and I really do hope we won't have to repeat this split
    for Python 3.14). In any case, the extra advantage of splitting it
    right now is that it permits us to test freethreading support separately
    and explicitly mark packages that work with that.

    Freethreading is considered experimental upstream. Unlike python3_13
    target, python3_13t is not intended to be unmasked on stable. We are
    also using a separate package name to avoid having to hack PV into
    something like 3.13.0 + 3.13.0-r100, and to avoid having Portage insist
    on installing the freethreading version, as it loves to proactively
    upgrade stuff. In fact, perhaps we should consider splitting all slots
    of Python into separate packages in the future.

    That said, the current implementation is not nice for PYTHON_COMPAT:

    PYTHON_COMPAT=( pypy3 python3_{10..13} python3_13t )

    I'm still considering our options here, but I don't think there's any
    really clean way of avoiding this.


    Michał Górny (9):
    dev-lang/python: Remove old
    dev-lang/python: Remove freethreading support
    profiles: Copy dev-lang/python entries for python-freethreading
    dev-lang/python-freethreading: New package, 3.13.0
    profiles: Add python3_13t (freethreading) target
    install-qa-check.d: Support dev-lang/python-freethreading
    python-utils-r1.eclass: Support python3_13t (freethreading) target
    app-portage/gpyutils: Add python3.13t
    distutils-r1.eclass: Do not use stable ABI wheels in freethreading

    .../gpyutils/files/implementations.txt | 1 +
    dev-lang/python-freethreading/Manifest | 3 ++
    .../python-freethreading/files/pydoc.conf | 6 +++
    .../python-freethreading/files/pydoc.init | 24 ++++++++++
    dev-lang/python-freethreading/metadata.xml | 39 +++++++++++++++
    .../python-freethreading-3.13.0.ebuild} | 47 +++++++------------
    dev-lang/python/Manifest | 3 --
    dev-lang/python/metadata.xml | 4 --
    dev-lang/python/python-3.13.0.ebuild | 15 +++---
    eclass/distutils-r1.eclass | 13 +++--
    eclass/python-utils-r1.eclass | 10 ++--
    eclass/tests/python-utils-r1.sh | 8 ++--
    metadata/install-qa-check.d/60python-site | 2 +-
    profiles/arch/alpha/package.use.mask | 1 +
    profiles/arch/hppa/package.use.mask | 1 +
    profiles/arch/loong/package.use.mask | 1 +
    profiles/arch/m68k/package.mask | 1 +
    profiles/arch/m68k/package.use.mask | 1 +
    profiles/arch/mips/package.use.mask | 1 +
    profiles/arch/s390/package.use.mask | 1 +
    profiles/base/make.defaults | 3 +-
    profiles/base/package.use.force | 7 +--
    profiles/base/package.use.mask | 2 +
    profiles/base/use.stable.mask | 2 +
    profiles/desc/python_single_target.desc | 1 +
    profiles/desc/python_targets.desc | 1 +
    profiles/features/hardened/package.use | 1 +
    profiles/features/musl/package.use.mask | 1 +
    profiles/features/prefix/package.use.mask | 1 +
    profiles/features/selinux/package.use.force | 1 +
    profiles/targets/desktop/gnome/package.use | 1 +
    profiles/targets/desktop/package.use | 2 +
    32 files changed, 143 insertions(+), 62 deletions(-)
    create mode 100644 dev-lang/python-freethreading/Manifest
    create mode 100644 dev-lang/python-freethreading/files/pydoc.conf
    create mode 100644 dev-lang/python-freethreading/files/pydoc.init
    create mode 100644 dev-lang/python-freethreading/metadata.xml
    rename dev-lang/{python/python-3.13.0_rc3.ebuild => python-freethreading/python-freethreading-3.13.0.ebuild} (92%)

    --
    2.47.0

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:00:01 2024
    Remove freethreading support from dev-lang/python in favor of a separate package. These two variations of Python are not ABI compatible and use separate installation trees.

    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    dev-lang/python/metadata.xml | 4 ----
    dev-lang/python/python-3.13.0.ebuild | 15 ++++++++-------
    profiles/base/make.defaults | 3 +--
    profiles/base/package.use.force | 6 ------
    4 files changed, 9 insertions(+), 19 deletions(-)

    diff --git a/dev-lang/python/metadata.xml b/dev-lang/python/metadata.xml
    index 97c6bd965b9f..22ce986e0676 100644
    --- a/dev-lang/python/metadata.xml
    +++ b/dev-lang/python/metadata.xml
    @@ -14,10 +14,6 @@
    to bootstrap pip and setuptools (if disabled, it will
    be only possible to use venv `--without-pip`)
    </flag>
    - <flag name="gil">
    - Build with Global Interpreter Lock. Disable to use
    - the experimental freethreading mode.
    - </flag>
    <flag name="jit">
    Enable experimental Just-In-Time compilation support.
    </flag>
    diff --git a/dev-lang/python/python-3.13.0.ebuild b/dev-lang/python/python-3.13.0.ebuild
    index 1f61895afb04..46dc39bae618 100644
    --- a/dev-lang/python/python-3.13.0.ebuild
    +++ b/dev-lang/python/pytho
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:00:01 2024
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    profiles/arch/alpha/package.use.mask | 1 +
    profiles/arch/hppa/package.use.mask | 1 +
    profiles/arch/loong/package.use.mask | 1 +
    profiles/arch/m68k/package.mask | 1 +
    profiles/arch/m68k/package.use.mask | 1 +
    profiles/arch/mips/package.use.mask | 1 +
    profiles/arch/s390/package.use.mask | 1 +
    profiles/base/package.use.mask | 2 ++
    profiles/features/hardened/package.use | 1 +
    profiles/features/musl/package.use.mask | 1 +
    profiles/features/prefix/package.use.mask | 1 +
    profiles/features/selinux/package.use.force | 1 +
    profiles/targets/desktop/gnome/package.use | 1 +
    profiles/targets/desktop/package.use | 2 ++
    14 files changed, 16 insertions(+)

    diff --git a/profiles/arch/alpha/package.use.mask b/profiles/arch/alpha/package.use.mask
    index 43de7be3517f..9d1db4f4ed3a 100644
    --- a/profiles/arch/alpha/package.use.mask
    +++ b/profiles/arch/alpha/package.use.mask
    @@ -44,6 +44,7 @@ media-video/f
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:10:02 2024
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    profiles/base/package.use.force | 1 +
    profiles/base/use.stable.mask | 2 ++
    profiles/desc/python_single_target.desc | 1 +
    profiles/desc/python_targets.desc | 1 +
    4 files changed, 5 insertions(+)

    diff --git a/profiles/base/package.use.force b/profiles/base/package.use.force index 9d51b4e49703..800efe47743b 100644
    --- a/profiles/base/package.use.force
    +++ b/profiles/base/package.use.force
    @@ -253,6 +253,7 @@ dev-lang/python-exec python_targets_python3_10
    dev-lang/python-exec python_targets_python3_11
    dev-lang/python-exec python_targets_python3_12
    dev-lang/python-exec python_targets_python3_13
    +dev-lang/python-exec python_targets_python3_13t
    dev-lang/python-exec python_targets_pypy3

    # Zac Medico <zmedico@gentoo.org> (2010-10-20)
    diff --git a/profiles/base/use.stable.mask b/profiles/base/use.stable.mask index 9167592b9e10..d1197864f26b 100644
    --- a/profiles/base/use.stable.mask
    +++ b/profiles/base/use.stable.mask
    @@ -16,6 +16,8 @@ llvm_slot_19
    # Pyth
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:10:02 2024
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    metadata/install-qa-check.d/60python-site | 2 +-
    1 file changed, 1 insertion(+), 1 deletion(-)

    diff --git a/metadata/install-qa-check.d/60python-site b/metadata/install-qa-check.d/60python-site
    index facfb54f1b3c..1a4ee0ca9f4e 100644
    --- a/metadata/install-qa-check.d/60python-site
    +++ b/metadata/install-qa-check.d/60python-site
    @@ -53,7 +53,7 @@ python_site_check() {

    # check for packages installing outside site-packages
    case ${CATEGORY}/${PN} in
    - dev-lang/python|dev-python/pypy*|dev-python/python-tests)
    + dev-lang/python|dev-lang/python-freethreading|dev-python/pypy*|dev-python/python-tests)
    ;;
    *)
    while IFS= read -d $'\0' -r f; do
    --
    2.47.0

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:10:02 2024
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    eclass/python-utils-r1.eclass | 10 +++++++---
    eclass/tests/python-utils-r1.sh | 8 ++++----
    2 files changed, 11 insertions(+), 7 deletions(-)

    diff --git a/eclass/python-utils-r1.eclass b/eclass/python-utils-r1.eclass index 796be7ab456a..ffaa0708ed5b 100644
    --- a/eclass/python-utils-r1.eclass
    +++ b/eclass/python-utils-r1.eclass
    @@ -41,6 +41,7 @@ inherit multiprocessing toolchain-funcs
    _PYTHON_ALL_IMPLS=(
    pypy3
    python3_{10..13}
    + python3_13t
    )
    readonly _PYTHON_ALL_IMPLS

    @@ -136,7 +137,7 @@ _python_set_impls() {
    # please keep them in sync with _PYTHON_ALL_IMPLS
    # and _PYTHON_HISTORICAL_IMPLS
    case ${i} in
    - pypy3|python3_9|python3_1[0-3])
    + pypy3|python3_9|python3_1[0-3]|python3_13t)
    ;;
    jython2_7|pypy|pypy1_[89]|pypy2_0|python2_[5-7]|python3_[1-9])
    obsolete+=( "${i}" )
    @@ -232,7 +233,7 @@ _python_impl_matches() {
    return 0
    ;;
    3.8|3.9|3.1[1-3])
    - [[ ${impl} == python${pattern/./_} ]] && return 0
    + [[ ${impl%t} =
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:10:02 2024
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    eclass/distutils-r1.eclass | 13 ++++++++++---
    1 file changed, 10 insertions(+), 3 deletions(-)

    diff --git a/eclass/distutils-r1.eclass b/eclass/distutils-r1.eclass
    index c3a7d112b03f..7ab8dcae3265 100644
    --- a/eclass/distutils-r1.eclass
    +++ b/eclass/distutils-r1.eclass
    @@ -1494,9 +1494,16 @@ distutils-r1_python_compile() {
    # that this relies on the assumption that we're building
    # from the oldest to the newest implementation,
    # and the wheels are forward-compatible.
    - if [[ ( ! ${DISTUTILS_EXT} && ${whl} == *py3-none-any* ) ||
    - ( ${EPYTHON} == python* && ${whl} == *-abi3-* ) ]]
    - then
    + if [[
    + ( ! ${DISTUTILS_EXT} && ${whl} == *py3-none-any* ) ||
    + (
    + ${EPYTHON} == python* &&
    + # freethreading does not support stable ABI
    + # at the moment
    + ${EPYTHON} != *t &&
    + ${whl} == *-abi3-*
    + )
    + ]]; then
    distutils_wheel_install "${BUILD_DIR}/install" "${whl}"
    return
    fi
    --
    2.47.0

    --- SoupGate-W
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:10:02 2024
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    app-portage/gpyutils/files/implementations.txt | 1 +
    1 file changed, 1 insertion(+)

    diff --git a/app-portage/gpyutils/files/implementations.txt b/app-portage/gpyutils/files/implementations.txt
    index 45705e04e3e1..95b321ec9437 100644
    --- a/app-portage/gpyutils/files/implementations.txt
    +++ b/app-portage/gpyutils/files/implementations.txt
    @@ -23,6 +23,7 @@ python3_10 old 3.10
    python3_11 old 3.11
    python3_12 current 3.12
    python3_13 experimental 3.13
    +python3_13t experimental 3.13t

    pypy1_8 2.7-pypy-1.8 dead p1.8
    pypy1_9 2.7-pypy-1.9 dead p1.9
    --
    2.47.0

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:00:01 2024
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    dev-lang/python/Manifest | 3 -
    dev-lang/python/python-3.13.0_rc3.ebuild | 665 -----------------------
    2 files changed, 668 deletions(-)
    delete mode 100644 dev-lang/python/python-3.13.0_rc3.ebuild

    diff --git a/dev-lang/python/Manifest b/dev-lang/python/Manifest
    index 96902bb9783e..daddb0dad1f4 100644
    --- a/dev-lang/python/Manifest
    +++ b/dev-lang/python/Manifest
    @@ -10,8 +10,6 @@ DIST Python-3.12.7.tar.xz 20444032 BLAKE2B eed8744261cab3b401963ec5187a8b814adb9
    DIST Python-3.12.7.tar.xz.asc 963 BLAKE2B 8c7db3d1971d93a10c611a2e6dd3679d0b331b48df87b5fe410b089061e48753d98af67084d4f051a31ae803fce3aa3c7af7cba9c692640f50068bd1e46f40e6 SHA512
    4d0a7a0da2c81888e93d8ff89ca2ed4bb85a7b0bd00f0d54a14e2c201affac2677ef3984d2b5aea253e624f6465548d9032eefdd9033c1eb3864d82cfd8d3df0
    DIST Python-3.13.0.tar.xz 22532980 BLAKE2B 4a4e397199402de8c9fe8e4d63443ed972cbe8a4f541743908d699c7987127d159d600fd14145b6a946e8671426279d230c9155fdc5863c1e49a945eedcbf21b SHA512
    44a143c9b96b55b01885ec020c33642
  • From =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?@21:1/5 to All on Wed Oct 9 14:00:01 2024
    A separate package installing the freethreading version
    of dev-lang/python.

    Closes: https://bugs.gentoo.org/940507
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    ---
    dev-lang/python-freethreading/Manifest | 3 +
    .../python-freethreading/files/pydoc.conf | 6 +
    .../python-freethreading/files/pydoc.init | 24 +
    dev-lang/python-freethreading/metadata.xml | 39 ++
    .../python-freethreading-3.13.0.ebuild | 654 ++++++++++++++++++
    5 files changed, 726 insertions(+)
    create mode 100644 dev-lang/python-freethreading/Manifest
    create mode 100644 dev-lang/python-freethreading/files/pydoc.conf
    create mode 100644 dev-lang/python-freethreading/files/pydoc.init
    create mode 100644 dev-lang/python-freethreading/metadata.xml
    create mode 100644 dev-lang/python-freethreading/python-freethreading-3.13.0.ebuild

    diff --git a/dev-lang/python-freethreading/Manifest b/dev-lang/python-freethreading/Manifest
    new file mode 100644
    index 000000000000..733309cd531a
    --- /dev/null
    +++ b/dev-lang/python-freethreading/Manifest
    @@ -0,0 +1,3 @@
    +DIST Python-3.13.0.tar.xz 22532980 BLAKE2B 4a4e397199402de8c9fe8e4d63443ed972cbe8a4f541743908d699c7987127d1