Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[@overload]: What is the effect of the *implementation* type-signature? #1840

Open
alanhdu opened this issue Aug 14, 2024 · 3 comments
Open
Labels
topic: documentation Documentation-related issues and PRs

Comments

@alanhdu
Copy link

alanhdu commented Aug 14, 2024

One thing that I've found underspecified when using @overload is what the type signature of the actual function should be.

For example, let's say I have a function like:

from typing import overload, Literal

@overload
def f(x: int, z: Literal[True]) -> str: ...
@overload
def f(x: int, z: Literal[False] = ...) -> int: ...
def f(x: int, z: bool = False) -> str | int:
    if z:
        return "hello"
    return 1

What is the effect of the last line? In particular:

  • The documentation and examples often just use no annotations for the implementation signature (def f(x, z):). Is that the right thing to do (I sometimes get Overloaded implementation is not consistent with signature of overload 1 errors if I leave out the type annotations)?
  • If I do specify types on the last, do I also have to add it to @overload list -- in other words:
@overload
def f(x: int, z: Literal[True]) -> str: ...
@overload
def f(x: int, z: Literal[False] = ...) -> int: ...
@overload
def f(x: int, z: bool = ...) -> str | int   # is this overload necessary?
def f(x: int, z: bool = False) -> str | int:
    if z:
        return "hello"
    return 1

Both pyright and mypy seem to interpret things differently with and without it (e.g. see this play link).

I understand that @overload is a complicated feature, but I'm hoping this is a small corner we can start with.

@alanhdu alanhdu added the topic: documentation Documentation-related issues and PRs label Aug 14, 2024
@hauntsaninja
Copy link
Collaborator

hauntsaninja commented Aug 15, 2024

The type signature of the implementation is never visible to callers. It is used for two things:

  • To type check the body the implementation
  • As a consistency check against the overloads (this is the "Overloaded implementation is not consistent with signature" error). Omitting type annotations should never trigger this, but things like omitting defaults (as in your example) or changing parameter names or kinds could

Regarding adding a third overload, I'm going to switch your example to using str for clarity:

@overload
def f(x: int, z: Literal["apple"]) -> str: ...
@overload
def f(x: int, z: Literal["banana"] = ...) -> int: ...
@overload
def f(x: int, z: str = ...) -> str | int  # what is the effect of this overload
def f(x: int, z: str = ...): ...

If you do not include the overload, type checkers will starting complaining on:

def main(z: str):
    f(0, z)

because they cannot guarantee the string is "apple" or "banana".

If you include the overload, type checkers will match that third overload on unknown strings.

bool makes things more interesting because there are no values other than True or False, so it would be valid for a hypothetical type checker to explode the bool automatically, making the third overload redundant. This is an area of overload resolution that would need to be specified to ensure all type checkers have that extra logic.

(Finally and more tangentially, I've found myself at times wanting an unsound literal feature (particularly so in typeshed). Where if the type checker knows the value of the literal, it uses the specific overloads, otherwise it aggregates information across the overloads)

@alanhdu
Copy link
Author

alanhdu commented Aug 15, 2024

Do you mind if I file a PR to the docs (and maybe add a conformance test or two)? I think what you said makes sense, but just to check my understanding:

  • The implementation type-signature is never seen by callers (so if it has extremely broad types like setting everythin to Any, that has no effect on other callers)
  • That means you need to add an @overload for the "broadest type signature" as well -- just adding it for the implementation signature is not enough (this seems unfortunately redundant to me, but not the end of the world either).
  • For now, there's no special-casing of bool as only having 2 variants.

I think it's really the first point that I didn't understand. I assumed that the implementation type-signature was "automatically" counted as one of the overloads.

@erictraut
Copy link
Collaborator

@alanhdu, overloads are generally underspecified in the typing spec today. Earlier this week, I posted a proposed update to the "Overloads" chapter of the spec that attempts to fill in many of these gaps. It covers all of the points that you mention above (and many additional details). I welcome feedback on the proposed change. For example, if you think there are ways it could be clearer, please post comments to the thread or the PR.

If and when that update is ratified and incorporated into the typing spec, we'll make sure there are corresponding conformance tests. The current conformance tests for overloads are pretty anemic, reflecting the fact that the spec doesn't provide much guidance here.

The typing documentation site includes several different categories of documentation. My proposed update is to the typing specification, which is primarily targeted at type checker authors. In your points above, the wording might be more appropriate for the "Guides" style of documentation, which is primarily targeted at consumers of type checkers. I think that additional clarity about overloads within the "Guides" documentation would be a great contribution if you're interested in submitting a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: documentation Documentation-related issues and PRs
Projects
None yet
Development

No branches or pull requests

3 participants