Change in Flag enum behaviour in v3.11rc2

[philthompson10]

Bug report

The default behaviour of Flag enums has chenged between Python v3.10 and v3.11

Your environment

Python v3.11rc2 on macOS

The following script...

from enum import Flag

class MethodHint(Flag):
    HiddenText = 0x10
    DigitsOnly = 0x01
    LettersOnly = 0x02
    OnlyMask = 0x0f

print(MethodHint.HiddenText|MethodHint.OnlyMask)

...raises a ValueError in v3.11 and prints...

MethodHint.HiddenText|OnlyMask|LettersOnly|DigitsOnly

in v3.10.

While it's not particularly sensible to bitwise-or a mask it is a significant change of behaviour without a deprecation period.

It will be common to have a mask value that covers more bits than currently defined when allowing for future expansion of the flags.

[ethanfurman]

There were some significant changes to Enum. They were originally planned for 3.10, but then pushed back to 3.11.

There are a few ways to adjust the above code:

1. use an IntEnum instead (it seems the actual values are important)
2. specify the boundary; i.e. class MethodHint(Enum, boundary=CONFORM):
3. use auto() for the values and reorder the members

The result of (1) would be 31.

The result of (2) would be MethodHint.HiddenText|DigitsOnly|LettersOnly.

(3) would be:

>>> from enum import Flag, auto

>>> class MethodHint(Flag):
...     DigitsOnly = auto()
...     LettersOnly = auto()
...     OnlyMask = DigitsOnly|LettersOnly
...     HiddenText = auto()
... 
>>> print(MethodHint.HiddenText|MethodHint.OnlyMask)
MethodHint.DigitsOnly|LettersOnly|HiddenText

[philthompson10]

On 16/09/2022 16:14, Ethan Furman wrote: There were some significant changes to Enum. They were originally planned for 3.10, but then pushed back to 3.11. There are a few ways to adjust the above code: 1. use an `IntEnum` instead (it seems the actual values are important) 2. specify the `boundary`; i.e. `class MethodHint(Enum, boundary=KEEP):` 3. use `auto()` for the values and reorder the members The result of (1) would be `31`. The result of (2) would be `MethodHint.HiddenText|DigitsOnly|LettersOnly|12`. (3) would be: >>> from enum import Flag, auto >>> class MethodHint(Flag): ... DigitsOnly = auto() ... LettersOnly = auto() ... OnlyMask = DigitsOnly|LettersOnly ... HiddenText = auto() ... >>> print(MethodHint.HiddenText|MethodHint.OnlyMask) MethodHint.DigitsOnly|LettersOnly|HiddenText

My point is that it’s an incompatible change without a deprecation period. Should the default boundary be KEEP for the next version? Phil

[ethanfurman]

@pablogsal -- Thoughts?

[gpshead]

The enum.Flag documentation has never stated this was forbidden and existing code clearly depends on it working so IMNSHO we need to restore the 3.10 enum.Flag behavior. Even if we let this ship in 3.11.0 and just document it as a known issue in the release, it is worthy of fixing before 3.11.1.

3.10 docs: https://docs.python.org/3.10/library/enum.html#flag
3.11 docs: https://docs.python.org/3.11/library/enum.html#enum.Flag

Neither of them proscribe specifics about what the Flag values must be, it just says that operations like | always work and produce a valid result of the same type. The new behavior violates that.

[ethanfurman]

Agreed on restoring the behavior and having the deprecation period, I was unsure if it should go into 3.11.0. I'll get a patch created for 3.11.1.

[ethanfurman]

Actually, won't need a deprecation period, since the proper default for Flag is CONFORM as that matches previous behavior.

Thank you, everyone.

[ethanfurman]

Phil wrote:

Should the default boundary be KEEP for the next version?

I edited my post: KEEP should be CONFORM (no spurious 12 that way).

[ethanfurman]

STRICT was/is the correct boundary for Flag -- some underlying issues were preventing it from working correctly. Those have now been fixed, and pre-3.11 behavior maintained.