Op 3/12/2024 om 10:41 schreef Roel Schroeven via Python-list:

[...]
When I try the same with bytes as base class though, that doesn't work
(at least in the Python version I'm using, which is CPython 3.11.2
64-bit on Windows 10):

class MyBytes(bytes):
    def __init__(self, data):
        super().__init__(data)
print(MyBytes(b'abcdefghijlkmn'))

This results in an exception:

Traceback (most recent call last):
  File "test_mybytes.py", line 4, in <module>
    print(MyBytes(b'abcdefghijlkmn'))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "test_mybytes.py", line 3, in __init__
    super().__init__(data)
TypeError: object.__init__() takes exactly one argument (the instance
to initialize)

I'm passing two arguments (data and the implicit self), and apparently
that's one too many. Let's try without arguments (i.e. only the
implicit self):

class MyBytes(bytes):
    def __init__(self, data):
        super().__init__()
print(MyBytes(b'abcdefghijlkmn'))

Now it works, and prints b'abcdefghijlkmn'. The same happens with int
as base class, and presumably a number of other classes.

As a follow-up, it looks like this behavior is because bytes and int are immutable. When I try with bytesarray instead of bytes, which works
largely the same but is mutable, things do work as I expect. There's a
hint in the documentation of __new__(): "__new__() is intended mainly to
allow subclasses of immutable types (like int, str, or tuple) to
customize instance creation". But that doesn't tell me why using super().__init__(<custom arguments>) doesn't work for immutable classes.

The documentation for __init__() says " If a base class has an
__init__() method, the derived class’s __init__() method, if any, must explicitly call it to ensure proper initialization of the base class
part of the instance; for example: super().__init__([args...])". So does
that mean that bytes and int not have an __init__() method? Is there a
link between being immutable and not having __init__()?

--
"Man had always assumed that he was more intelligent than dolphins because
he had achieved so much — the wheel, New York, wars and so on — whilst all the dolphins had ever done was muck about in the water having a good time.
But conversely, the dolphins had always believed that they were far more intelligent than man — for precisely the same reasons."
-- Douglas Adams

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

We can use super().__init__() in the __init__() method of a derived
class to initialize its base class. For example:

import string
class MyTemplate(string.Template):
    def __init__(self, template_string):
        super().__init__(template_string)
print(MyTemplate('Hello ${name}').substitute(name="Pedro"))

This works, and prints "Hello Pedro" as expected. Note that I passed template_string in the super().__init__() call, and that is what used to initialize the base class. So far nothing special.

When I try the same with bytes as base class though, that doesn't work
(at least in the Python version I'm using, which is CPython 3.11.2
64-bit on Windows 10):

class MyBytes(bytes):
    def __init__(self, data):
        super().__init__(data)
print(MyBytes(b'abcdefghijlkmn'))

This results in an exception:

Traceback (most recent call last):
  File "test_mybytes.py", line 4, in <module>
    print(MyBytes(b'abcdefghijlkmn'))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "test_mybytes.py", line 3, in __init__
    super().__init__(data)
TypeError: object.__init__() takes exactly one argument (the instance to initialize)

I'm passing two arguments (data and the implicit self), and apparently
that's one too many. Let's try without arguments (i.e. only the implicit
self):

class MyBytes(bytes):
    def __init__(self, data):
        super().__init__()
print(MyBytes(b'abcdefghijlkmn'))

Now it works, and prints b'abcdefghijlkmn'. The same happens with int as
base class, and presumably a number of other classes. That behavior is
beyond my understanding, so I have some questions that might hopefully
lead to a better understanding:

(1) How does that work? How does my argument end up in the code that initializes the instance state?

(2) How can I customize the argument is passed? For example, what if I
want to do something like (supersimple example) super().__init__(data * 2)?

(3) Why is bytes (and int, ...) different? Is it because it's a builtin?
Or implemented in C? Or something else?

--
"Man had always assumed that he was more intelligent than dolphins because
he had achieved so much — the wheel, New York, wars and so on — whilst all the dolphins had ever done was muck about in the water having a good time.
But conversely, the dolphins had always believed that they were far more intelligent than man — for precisely the same reasons."
-- Douglas Adams

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Roel Schroeven <roel@roelschroeven.net> wrote or quoted:

As a follow-up, it looks like this behavior is because bytes and int are >immutable.

Alright, so you've stumbled onto some pretty gnarly behavior with
Python's immutable types. Let's unpack this burrito:

1) Here's the skinny on how it goes down with immutable types
like "bytes" and "int":

These bad boys do their thing in "__new__", not
"__init__". It's like trying to change a burrito after
it's been wrapped - ain't going to happen.

When you hit "MyBytes( b'abcdefghijlkmn' )", it's like this:

a) Python calls up "MyBytes.__new__".

b) That passes the buck to "bytes.__new__", which
whips up the new "bytes" object.

c) Then "MyBytes.__init__" gets a shot, but the horse
has already left the barn.

2) Wanna customize those args?

For immutable types, you got to override `__new__`.
It's like swapping out the ingredients before the
burrito's wrapped:

class MyBytes( bytes ):
def __new__( cls, data ):
return super().__new__( cls, data * 2 )

print( MyBytes( b'abc' )) # Spits out: b'abcabc'

3) Why "bytes" (and "int", ...) are different beasts:

You nailed it in your follow-up. It's all about being immutable.
These types are coded in C for speed, but the real kicker
is they can't change after creation.

On your other questions:

- Yeah, immutable types like "bytes" and "int" pretty much
ghost "__init__". All the magic happens in "__new__".

- There's definitely a connection between being immutable
and ditching "__init__". It's like setting your GPS
before you start driving - once you're moving, you can't
change the route.

- That "__init__" doc you quoted? It's legit for mutable
types, but for immutable ones, it's like trying to add
avocado to a sealed burrito - not going to work.

To wrap it up:

- Mutable types: Override "__init__" to set things up.

- Immutable types: "__new__" is where the party's at.

This setup lets Python keep its promises about immutability while
still letting you customize through subclassing. It's like having
your artisanal, gluten-free burrito and eating it too.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Um9lbCBTY2hyb2V2ZW4gPHJvZWxAcm9lbHNjaHJvZXZlbi5uZXQ+IHdyb3RlOg0KPiBBcyBhIGZv bGxvdy11cCwgaXQgbG9va3MgbGlrZSB0aGlzIGJlaGF2aW9yIGlzIGJlY2F1c2UgYnl0ZXMgYW5k IGludCBhcmUgaW1tdXRhYmxlLg0KDQpZZXMuDQoNCj4gQnV0IHRoYXQgZG9lc24ndCB0ZWxsIG1l IHdoeSB1c2luZyBzdXBlcigpLl9faW5pdF9fKDxjdXN0b20gYXJndW1lbnRzPikgZG9lc24ndCB3 b3JrIGZvciBpbW11dGFibGUgY2xhc3Nlcy4NCg0KYnl0ZXMuX19pbml0X18gZG9lcyB3b3JrLCBi dXQgaXQncyBqdXN0IGFuIGluaGVyaXRlZCBvYmplY3QuX19pbml0X18sIHdoaWNoIGRvZXMgbm90 aGluZywgYW5kIHRha2VzIG5vIHBhcmFtZXRlcnMuDQogX19pbml0X18gY2Fubm90IGNoYW5nZSB0 aGUgdmFsdWUgb2YgdGhlIGJ5dGVzIG9iamVjdDsgdGhlIHZhbHVlIGlzIHNldCBieSBieXRlcy5f X25ld19fIGFuZCBjYW5ub3QgY2hhbmdlIGFmdGVyIHRoYXQuDQoNCkJlc3Qgbm90IHRvIGRlZmlu ZSBhbiBfX2luaXRfXyBtZXRob2QgYXQgYWxsLCBqdXN0IHVzZSBfX25ld19fLg0KDQpTb21ldGhp bmcgbGlrZToNCg0KY2xhc3MgQnl0ZXNTdWJjbGFzcyhieXRlcyk6DQogICAgZGVmIF9fbmV3X18o Y2xzLCB3aGF0ZXZlciwgYXJndW1lbnRzLCB5b3UsIGxpa2UpOg0KICAgICAgICBieXRlc3ZhbHVl ID0gY29tcHV0ZSh3aGF0ZXZlciwgYXJndW1lbnRzLCB5b3UsIGxpa2UpDQogICAgICAgIG9iID0g Ynl0ZXMuX19uZXdfXyhjbHMsIGJ5dGVzdmFsdWUpDQogICAgICAgIG9iLnNvbWVfb3RoZXJfYXR0 ID0gY29tcHV0ZV9zb21ldGhpbmdfZWxzZSh3aGF0ZXZlciwgYXJndW1lbnRzLCB5b3UsIGxpa2Up DQogICAgICAgIHJldHVybiBvYg0KDQpyZWdhcmRzLCANCkFuZGVycw0KDQo=

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Op 3/12/2024 om 13:55 schreef Anders Munch via Python-list:

Roel Schroeven <roel@roelschroeven.net> wrote:

As a follow-up, it looks like this behavior is because bytes and int are immutable.

Yes.

OK.

But that doesn't tell me why using super().__init__(<custom arguments>) doesn't work for immutable classes.

bytes.__init__ does work, but it's just an inherited object.__init__, which does nothing, and takes no parameters.
__init__ cannot change the value of the bytes object; the value is set by bytes.__new__ and cannot change after that.

I see now why __init__, being a regular method, can't change an object's
value (or attributes in general) if that object is immutable. I'm not
sure why I didn't think of that before.

It's not entirely clear to me though how bytes.__new__ *can* set an
object's value. Isn't __new__ also a regular function? Are these
immutable classes special cases in the language that can't be recreated
in the same way with user-defined classes? Not that that's something I
want to do, and it's also not terribly important to me, but I'm trying
to better understand what's going on.

Best not to define an __init__ method at all, just use __new__.

Something like:

class BytesSubclass(bytes):
def __new__(cls, whatever, arguments, you, like):
bytesvalue = compute(whatever, arguments, you, like)
ob = bytes.__new__(cls, bytesvalue)
ob.some_other_att = compute_something_else(whatever, arguments, you, like)
return ob

Thanks, that works perfectly. That's also more important than
understanding all the nitty-gritty details (I feel a basic understanding
is important, but not necessarily always all the low-level details).

--
"There is no cause so noble that it will not attract fuggheads."
-- Larry Niven

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 4/12/24 3:24 am, Roel Schroeven wrote:

It's not entirely clear to me though how bytes.__new__ *can* set an
object's value. Isn't __new__ also a regular function?

Yes, but the __new__ methods of the builtin immutable objects (int,
str, bytes, etc.) are implemented in C, and so are able to do things
that Python methods cannot.

--
Greg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Op 4/12/2024 om 0:14 schreef Greg Ewing via Python-list:

On 4/12/24 3:24 am, Roel Schroeven wrote:

It's not entirely clear to me though how bytes.__new__ *can* set an
object's value. Isn't __new__ also a regular function?

Yes, but the __new__ methods of the builtin immutable objects (int,
str, bytes, etc.) are implemented in C, and so are able to do things
that Python methods cannot.

Aha, yes, that's what I already suspected, but I wasn't sure. Thanks for confirming that.

All clear now. Thanks to Anders and Greg for explaining this to me.

"In the old days, writers used to sit in front of a typewriter and stare out of the window. Nowadays, because of the marvels of convergent technology, the thing
you type on and the window you stare out of are now the same thing.”
-- Douglas Adams

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)