Implement PEP 649 and PEP 749

[JelleZijlstra]

PEP-649 has been accepted and should be implemented in Python 3.14. Let's use this issue to track the implementation:

• Decide on or clarify the adoption strategy (https://discuss.python.org/t/pep-649-deferred-evaluation-of-annotations-tentatively-accepted/21331/44)
• Add new __annotate__ attributes gh-119180: PEP 649: Add __annotate__ attributes #119209
• Implement core interpreter changes (e.g., new symbol table functionality) gh-119180: PEP 649 compiler changes #119361
• Implement new Python-level APIs, like the new format argument to inspect.get_annotations gh-119180: Add annotationlib module to support PEP 649 #119891
  • (PEP-649 specifies that inspect.AnnotationsFormat should be a "global enum". Is that desirable? TBD. https://github.com/python/cpython/pull/119361/files#r1614753031)
• Document the new semantics gh-119180: Documentation for PEP 649 and 749 #122235
  • Add higher-level documentation, e.g. a migration guide away from from __future__ import annotations; an introduction to annotationlib; an update to Larry's annotations HOWTO
• Make staticmethod and classmethod not force evaluation of annotations gh-119180: Lazily wrap annotations on classmethod and staticmethod #119864
• Adapt standard library code that uses annotations to work excellently with PEP 649
  • dataclasses gh-119180: Add annotationlib module to support PEP 649 #119891
  • typing.py: TypedDict, NamedTuple, try to remove dependency on inspect. (If it can't be removed, get rid of the awkward dance we do for typing.Protocol to import inspect.getattr_static lazily.)
  • singledispatch gh-119180: Add annotationlib module to support PEP 649 #119891
    • Also make it only evaluate the relevant parameter, gh-119180: Add annotationlib module to support PEP 649 #119891 (comment)
  • functools.update_wrapper gh-119180: Add annotationlib module to support PEP 649 #119891, PEP 649: functools.update_wrapper implementation #124342
• Add PEP 649-like functionality to TypeAliasType, TypeVar, etc. (pep649-typevar branch in my fork)
• Make sure the ecosystem is ready by testing third-party libraries like Pydantic
  • typing_extensions: needs changes similar to those in typing.py (Support my PEP 649 branch typing_extensions#412)
  • beartype: had various unrelated problems on 3.13 ([Future] Python 3.13 conformance beartype/beartype#387); a few additional tests fail on the PR branch due to direct manipulation of __dict__
  • typeguard: some unrelated 3.13 problems (Support Python 3.13 agronholm/typeguard#460); no test failures related to PEP 649
  • pyanalyze: some unrelated 3.13 issues and a few changes necessary due to direct __dict__ access (Support Python 3.13 quora/pyanalyze#773)
  • pydantic: failed to build for 3.14 so far
• Optimize the implementation. Ideas:
  • Avoid creation of function objects and store only code objects instead PEP 649: Avoid creation of function objects for __annotate__ #124157
  • Avoid creation of AST nodes and use of eval() for ForwardRefs that are just names gh-119180: Avoid going through AST and eval() when possible in annotationlib #124337

Things to revisit:

• Should FORWARDREF be the default? Some improvements for Python 3.14 / PEP 649 support beartype/beartype#440 (comment)
• Add VALUE_WITH_FAKE_GLOBALS format
• Name of __annotate__ parameter
• Should setting __annotations__ invalidate __annotate__?
• Rename SOURCE to STRING?

I am planning to work on the interpreter core first.

cc @larryhastings @carljm @samuelcolvin

Linked PRs

• gh-119180: PEP 649: Add __annotate__ attributes #119209
• gh-119180: Add LOAD_COMMON_CONSTANT opcode #119321
• gh-119180: PEP 649 compiler changes #119361
• gh-119180: Update the magic number #119397
• gh-119180: Lazily wrap annotations on classmethod and staticmethod #119864
• gh-119180: Add annotationlib module to support PEP 649 #119891
• gh-119180: Fix annotations lookup on classes with custom metaclasses #120719
• gh-119180: Alternative approach to metaclass annotations: never use the dict #120816
• gh-119180: Use type descriptors to access annotations (PEP 749) #122074
• gh-119180: Improvements to ForwardRef.evaluate #122210
• gh-119180: Add evaluate functions for type params and type aliases #122212
• gh-119180: Documentation for PEP 649 and 749 #122235
• gh-119180: annotationlib: Fix __all__, other minor fixes #122365
• gh-119180: Add myself as CODEOWNER for annotationlib #122366
• gh-119180: Fix annotationlib.ForwardRef.evaluate with no globals #124326
• gh-119180: Avoid going through AST and eval() when possible in annotationlib #124337
• gh-119180: Add discussion of annotations to the 3.14 What's New #124393
• gh-119180: Add VALUE_WITH_FAKE_GLOBALS format to annotationlib #124415
• gh-119180: Rename parameter to __annotate__ functions #124461
• gh-119180: Make FORWARDREF format look at __annotations__ first #124479
• gh-119180: Disallow instantiation of ConstEvaluator objects #124561
• gh-119180: Rename SOURCE format to STRING #124620
• gh-119180: No longer set __annotations__ in __main__ #124634
• gh-119180: Set the name of the param to __annotate__ to "format" #124730

[Jens-Dittrich]

Hi everyone,

just wanted to share a (condensed) example where I had to use Forward references outside of type annotations, but in specific parameters of generics as below:

from typing import TypeVar, Generic


BaseClassType = TypeVar("BaseClassType")


class GenericClass(Generic[BaseClassType]):
    def a_method(self) -> BaseClassType:
        pass


class ConcreteClass(GenericClass["ConcreteFriendClass"]):
    pass


GenericClassType = TypeVar("GenericClassType", bound=GenericClass)


class GenericFriendClass(Generic[GenericClassType]):
    def a_method(self) -> GenericClassType:
        pass


class ConcreteFriendClass(GenericFriendClass[ConcreteClass]):
    pass


instance = ConcreteClass()
friend = ConcreteFriendClass()

res = instance.a_method()

Reason is, that ConcreteClass and ConcreteFriendClass are somehow coupled to each other from a business perspective. Not sure, the design is really great or could be improved, but what I want you guys to point you to is the statement of definition of Class ConcreteClass where the friend's class name is specified as forward reference. PyCharm is already able to correctly determine the type of variable res to be of ConcreteFriedClass - so it is at least ForwardReference-ish.

Question now, is this tbh quite corner case also in scope of the ticket, in that case I am glad to contribute above a test case. Just remove the Forward Reference in that line and it shall work. Or is it out of scope, maybe by a flaw of my own understanding or postponed to a later release etc.

EDIT:
I just happened to realise that I can retrieve the type parameter at runtime in 3.12 and it turns out to be a Forward Reference

from typing import get_args, ForwardRef
from types import get_original_bases

assert ForwardRef("ConcreteFriendClass") == get_args(get_original_bases(ConcreteClass)[0])[0]

/EDIT

Happy to have a conversation on that.

Regards, Jens

[JelleZijlstra]

@Jens-Dittrich your use case is already solved in Python 3.12, where PEP-695 provides a syntax for generic bounds that is lazily evaluated. If you have further questions, please post on https://discuss.python.org instead so we can keep this issue focused on implementing the Python 3.14 changes.