• Re: [gentoo-dev] Slotting PyPy

    From =?UTF-8?Q?Micha=C5=82_G=C3=B3rny?=@21:1/5 to All on Thu Oct 17 15:00:01 2024
    On Thu, 2024-10-17 at 08:47 -0400, Jérôme Carretero wrote:
    On Thu, 2024-10-17 at 14:12 +0200, Michał Górny wrote:
    Hello,

    Given that you've expressed your preference for dev-lang/python
    remaining slotted, I'd like to open another can of worms: should we
    slot
    PyPy consistently with it?  Some history and then my ideas below.


    FWIW I'm all for it.

    One reason is that I have current /usr/lib/pypy3.9 but still residues
    of some dev-python packages that somehow haven't been updated in /usr/lib/pypy3.8/site-packages and older folders, when pypy itself of
    the corresponding version isn't even there.
    It would be nice for the USE magic to work with pypy so such things
    can't happen.

    Unless I'm missing something, I don't think that's going to help with
    your case.

    --
    Best regards,
    Michał Górny


    -----BEGIN PGP SIGNATURE-----

    iQFGBAABCgAwFiEEx2qEUJQJjSjMiybFY5ra4jKeJA4FAmcRCh0SHG1nb3JueUBn ZW50b28ub3JnAAoJEGOa2uIyniQO5rIH/AmEIedenLRbakwO07mUK/IZ7JYHyRDe wNoQba5qirmxTIqCUCI3B0H2JiLbYA0QqaA/oK73zlLddiIVmmJtqsx3hWsICJO8 cVheJezO9lXtSDsWHLda6BWPYuwOJa3TXnN2HiSlzM40Ey4jgmnr2wZIXZOzp1Il RR35V+KJdWlmMkuSId+YFC/iUZMPA84Yktk2iUaXc61PRmRne70rZiaLFbpINn4U 9IYbis7tzcEfpkDlobWZtxV+jvM1tSHM2jwr0WKGpLj6SLqAt50jlt8WBrMN9Mzf WcELvucL+ABYv7XAE31q/KjoFLYOCRil0b6zKmSalfmiQhJZeecqX+g=
    =y/nz
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Micha=C5=82_G=C3=B3rny?=@21:1/5 to All on Thu Oct 17 14:20:01 2024
    Hello,

    Given that you've expressed your preference for dev-lang/python
    remaining slotted, I'd like to open another can of worms: should we slot
    PyPy consistently with it? Some history and then my ideas below.


    PyPy versioning
    ===============

    PyPy has basically two versions:

    sys.version_info
    sys.version_info(major=3, minor=10, micro=14, releaselevel='final', serial=0) >>>> sys.pypy_version_info
    sys.pypy_version_info(major=7, minor=3, micro=17, releaselevel='final', serial=0)

    The former indicates the CPython version it is compatible with,
    and the stdlib version from CPython it used. The latter is the actual
    PyPy release.

    PyPy is doing synchronous releases for all Python versions it supports
    at the time, with PyPy release matching between them. For example,
    7.3.16 release involved the following tarballs:

    pypy3.10-v7.3.16-src.tar.bz2
    pypy3.9-v7.3.16-src.tar.bz2
    pypy2.7-v7.3.16-src.tar.bz2

    Nowadays PyPy2.7 and PyPy3.10 are supported and released, and there
    should be a PyPy3.11 release soon.


    History and overview of Gentoo packages
    =======================================

    Initially, we had just PyPy 2.x packaged, as dev-python/pypy, e.g.:

    dev-python/pypy-7.3.17

    corresponds to:

    pypy2.7-v7.3.17-src.tar.bz2

    Then we added PyPy 3.x as a dev-python/pypy3 package, e.g.:

    dev-python/pypy3-7.3.16

    would correspond to:

    pypy3.10-v7.3.16-src.tar.bz2

    We have decided to support only a single PyPy 3.x slot, for two reasons:
    1) PyPy is quite experimental, and 2) upstream is usually far behind
    CPython on releases (PyPy is at 3.10 still, while CPython just released
    3.13).

    For the relatively short transitions when two PyPy 3.x versions were
    released, we combined revisions and subslots, e.g.:

    dev-python/pypy3-7.3.16:0/pypy39-...
    dev-python/pypy3-7.3.16-r100:0/pypy310-...

    would correspond to, respectively:

    pypy3.9-v7.3.16-src.tar.bz2
    pypy3.10-v7.3.16-src.tar.bz2

    The eclasses would depend on 'dev-python/pypy3:=' to bind to a specific subslot, and soon afterwards we'd just be providing the next PyPy 3.x
    versions without the older.

    Then, as we decided to keep older CPython versions around without eclass support (3.8 and 3.9), it started to make sense to also keep old PyPy
    3.x versions around -- also without eclass support. So I've split the
    packages even further, into:

    dev-python/pypy3_9-7.3.16
    dev-python/pypy3_10-7.3.16

    that correspond to, respectively:

    pypy3.9-v7.3.16-src.tar.bz2
    pypy3.10-v7.3.16-src.tar.bz2

    And dev-python/pypy3 remained as a subslotted virtual that would pull
    whichever PyPy 3.x version we support at the time.


    Slotting again
    ==============

    A possible goal for the future would be to recombine all these packages
    into one, e.g.:

    dev-lang/pypy-2.7.7.3.16:pypy27/...
    dev-lang/pypy-3.9.7.3.16:pypy39/...
    dev-lang/pypy-3.10.7.3.16:pypy310/...

    corresponding to:

    pypy2.7-v7.3.16-src.tar.bz2
    pypy3.9-v7.3.16-src.tar.bz2
    pypy3.10-v7.3.16-src.tar.bz2

    So the PV would combine slot and release version, slot would indicate
    PyPy 3.x slot and subslot would indicate the ABI (changes rarely).

    The ebuilds could now depend e.g. on:

    >=dev-lang/pypy-3.10:=

    This would ensure that only slots newer than 3.10 are acceptable,
    and that packages are rebuilt (as they are right now) once we introduce
    3.11 slot. Then, after the transitional period we'd update it to:

    >=dev-lang/pypy-3.11:=

    and so on.


    Backwards compatibility and transition
    ======================================

    For the time being, we'd have to maintain dev-python/pypy3
    as a "virtual" for compatibility. The eclasses would be updated, so
    that newly built packages depend on and bind to the slots of the new
    package.

    Once PyPy3.11 is released, dev-python/pypy3 would gain a new subslot,
    and the := deps will be triggering the rebuild. At the same time,
    the rebuild will result in the packages updating to depend on the new
    package. Some time after this transition, we'd be able to last rite dev-python/pypy3.


    Summary
    =======

    So the rough idea is that we'd be replacing dev-python/pypy, dev-
    python/pypy3, dev-python/pypy3_9, dev-python/pypy3_10 with a single
    slotted package. At least the second package would have to preserved
    for some time, for backwards compatibility; the others are less
    significant since they are not used in eclass-generated dependencies.

    I suppose we'd also be then aiming to combine dev-python/pypy*-exe dev-python/pypy*-exe-bin to some degree, but that's a minor point, since
    they are implementation details.

    WDYT?

    --
    Best regards,
    Michał Górny


    -----BEGIN PGP SIGNATURE-----

    iQFGBAABCgAwFiEEx2qEUJQJjSjMiybFY5ra4jKeJA4FAmcQ/xASHG1nb3JueUBn ZW50b28ub3JnAAoJEGOa2uIyniQO96MH/iqYbrangTYSJAXaxLL69XIU6NMfUOHe BpAbyKx2EyQ2ES2tB4s0jg15ZruatvvhryENUlTI+dfDlbmnJxJjydZ1t+07uZK/ g3/9Tm5cL/4sNZaQaan1rwgpr6v9eoSX416e/KJmYfFJ+EXzVevXbv2LIIbCwvuK lPZfW5hoQBWdgBv3sutOINTT96A3FD8MZTkoTuMNnzFH3elQ1duiaI2V1kH0HrZ4 8g2/FJ4sO33stc9ddKIFlZvcNWxYsUzkjQCzMAAxMvhLk0BYo8Cy6pV0yKHDb8Cz 6y7iEnBKyGTDBoSNgZO9LoB62ATmryaS1xSDqYfZ7kU+DQYa6rlOP1I=
    =iKbi
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)