• python C-api and thread

    From aotto1968@21:1/5 to All on Mon Aug 5 23:19:14 2024
    hi,

    Is it possible to run two completely independent Python interpreters in one process, each using a thread?

    By independent, I mean that no data is shared between the interpreters and thus the C API can be used without any other
    "lock/GIL" etc.

    mfg

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to All on Tue Aug 6 00:32:49 2024
    On Mon, 5 Aug 2024 23:19:14 +0200, aotto1968 wrote:

    Is it possible to run two completely independent Python interpreters in
    one process, each using a thread?

    By independent, I mean that no data is shared between the interpreters
    and thus the C API can be used without any other "lock/GIL" etc.

    Seems like yes <https://docs.python.org/3/c-api/init.html#c.Py_NewInterpreterFromConfig>.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From aotto1968@21:1/5 to Lawrence D'Oliveiro on Tue Aug 6 08:01:41 2024
    On 06.08.24 02:32, Lawrence D'Oliveiro wrote:
    On Mon, 5 Aug 2024 23:19:14 +0200, aotto1968 wrote:

    Is it possible to run two completely independent Python interpreters in
    one process, each using a thread?

    By independent, I mean that no data is shared between the interpreters
    and thus the C API can be used without any other "lock/GIL" etc.

    Seems like yes <https://docs.python.org/3/c-api/init.html#c.Py_NewInterpreterFromConfig>.

    → I think that could be a solution.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From aotto1968@21:1/5 to Grant Edwards on Tue Aug 6 08:11:39 2024
    On 06.08.24 04:34, Grant Edwards wrote:
    On 2024-08-05, aotto1968 via Python-list <python-list@python.org> wrote:

    Is it possible to run two completely independent Python interpreters
    in one process, each using a thread?

    By independent, I mean that no data is shared between the
    interpreters and thus the C API can be used without any other
    "lock/GIL" etc.

    No, not using any OS I've ever seen. The usual definition of "threads"
    is that they share data, and the definition of "processes" is that
    processes don't share data.

    How exactly does what you're trying to do differ from runnig two
    Python interpreters in two processes?

    --
    Grant




    I know but I use a thread like a process because the "conversation" between the threads is done by my
    software. a Thread is usually faster to startup (thread-pool) this mean for high-load this is
    significant faster even than fork.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Barry Scott@21:1/5 to All on Tue Aug 6 21:12:46 2024
    On 6 Aug 2024, at 07:11, aotto1968 via Python-list <python-list@python.org> wrote:

    I know but I use a thread like a process because the "conversation" between the threads is done by my
    software. a Thread is usually faster to startup (thread-pool) this mean for high-load this is
    significant faster even than fork.

    using processes means you are more robust and can survive a process crash. using async, assuming you do enough I/O, will out perform threads.

    Barry

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to All on Tue Aug 6 23:41:45 2024
    On Tue, 6 Aug 2024 08:11:39 +0200, aotto1968 wrote:

    ... a Thread is usually faster to startup (thread-pool) ...

    Process pools exist, too. As I recall, Apache and Android use them.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)