Allow Ellipsis in Concatenate; cleanup ParamSpec literals

[ilevkivskyi]

Fixes #14761
Fixes #15318
Fixes #14656
Fixes #13518

I noticed there is a bunch of inconsistencies in semanal/typeanal for ParamSpecs, so I decided do a small cleanup. Using this opportunity I also allow Concatenate[int, ...] (with literal Ellipsis), and reduce verbosity of some errors.

cc @A5rocks

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

steam.py (https://github.com/Gobot1234/steam.py)
- steam/utils.py:239: error: Unexpected "..."  [misc]
- steam/utils.py:239: error: The last parameter to Concatenate needs to be a ParamSpec  [valid-type]
- steam/utils.py:239: error: The first argument to Callable must be a list of types, parameter specification, or "..."  [valid-type]
- steam/utils.py:239: note: See https://mypy.readthedocs.io/en/stable/kinds_of_types.html#callable-types-and-lambdas
- steam/utils.py:239: error: TypeVar "bound" must be a type  [misc]
- steam/utils.py:240: error: Unexpected "..."  [misc]
- steam/utils.py:240: error: The last parameter to Concatenate needs to be a ParamSpec  [valid-type]
- steam/utils.py:240: error: The first argument to Callable must be a list of types, parameter specification, or "..."  [valid-type]
- steam/utils.py:240: note: See https://mypy.readthedocs.io/en/stable/kinds_of_types.html#callable-types-and-lambdas
- steam/utils.py:240: error: TypeVar "bound" must be a type  [misc]
+ steam/http.py:173: error: Value of type variable "F_wait" of function cannot be "Callable[[HTTPClient], Coroutine[Any, Any, None]]"  [type-var]
+ steam/http.py:188: error: Value of type variable "F_wait" of function cannot be "Callable[[HTTPClient], Coroutine[Any, Any, str | None]]"  [type-var]
+ steam/gateway.py:612: error: Value of type variable "F_wait" of function cannot be "Callable[[SteamWebSocket], Coroutine[Any, Any, str]]"  [type-var]
+ steam/state.py:1692: error: Value of type variable "F_no_wait" of "call_once" cannot be "Callable[[ConnectionState], Coroutine[Any, Any, None]]"  [type-var]
+ steam/ext/tf2/state.py:152: error: Value of type variable "F_no_wait" of "call_once" cannot be "Callable[[GCState], Coroutine[Any, Any, None]]"  [type-var]

Tanjun (https://github.com/FasterSpeeding/Tanjun)
- tanjun/abc.py:147: error: Unexpected "..."  [misc]
- tanjun/abc.py:156: error: Unexpected "..."  [misc]
- tanjun/abc.py:172: error: Unexpected "..."  [misc]
- tanjun/abc.py:178: error: Unexpected "..."  [misc]
- tanjun/abc.py:189: error: Unexpected "..."  [misc]
- tanjun/abc.py:196: error: Unexpected "..."  [misc]
- tanjun/abc.py:207: error: Unexpected "..."  [misc]
- tanjun/abc.py:226: error: Unexpected "..."  [misc]
- tanjun/abc.py:241: error: Unexpected "..."  [misc]
- tanjun/abc.py:252: error: Unexpected "..."  [misc]
- tanjun/parsing.py:94: error: Unexpected "..."  [misc]
- tanjun/commands/slash.py:102: error: Unexpected "..."  [misc]
- tanjun/clients.py:152: error: Unexpected "..."  [misc]
- tanjun/annotations.py:277: error: Unexpected "..."  [misc]

sublime_debugger (https://github.com/daveleroy/sublime_debugger)
- modules/dap/session.py:105: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[Any], Any]"; expected "Callable[[VarArg(Iterable[ExceptionBreakpointsFilter]), KwArg(Iterable[ExceptionBreakpointsFilter])], Any]"  [arg-type]
- modules/views/breakpoints.py:28: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[Any], None]"; expected "Callable[[VarArg(Iterable[ExceptionBreakpointsFilter]), KwArg(Iterable[ExceptionBreakpointsFilter])], Any]"  [arg-type]
- modules/views/sources.py:21: error: Cannot determine type of "on_session_sources_updated"  [has-type]
- modules/views/sources.py:22: error: Cannot determine type of "on_session_removed"  [has-type]
- modules/views/modules.py:23: error: Cannot determine type of "on_session_modules_updated"  [has-type]
- modules/views/modules.py:24: error: Cannot determine type of "on_session_removed"  [has-type]
- modules/views/debugger.py:23: error: Cannot infer type of lambda  [misc]
- modules/views/debugger.py:23: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[Any, Any], Any]"; expected "Callable[[], Any]"  [arg-type]
- modules/views/debugger.py:24: error: Cannot determine type of "on_session_active"  [has-type]
- modules/views/debugger.py:25: error: Cannot determine type of "on_session_added"  [has-type]
- modules/views/debugger.py:27: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[], Any]"; expected "Callable[[VarArg(None), KwArg(None)], Any]"  [arg-type]
- modules/views/debugger.py:69: error: Cannot infer type of lambda  [misc]
- modules/views/debugger.py:69: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[Any, Any], Any]"; expected "Callable[[], Any]"  [arg-type]
- modules/views/debugger.py:70: error: Cannot determine type of "on_session_removed"  [has-type]
- modules/views/debugger.py:71: error: Cannot infer type of lambda  [misc]
- modules/views/debugger.py:71: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[], Any]"; expected "Callable[[VarArg(None), KwArg(None)], Any]"  [arg-type]
- modules/output_panel.py:21: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[], Any]"; expected "Callable[[VarArg(None), KwArg(None)], Any]"  [arg-type]
- modules/views/variables.py:33: error: Cannot determine type of "on_session_variables_updated"  [has-type]
- modules/views/variables.py:34: error: Cannot determine type of "on_session_removed"  [has-type]
- modules/debugger.py:76: error: Incompatible types in assignment (expression has type "Event[Session]", base class "Debugger" defined the type as "Event[[Session]]")  [assignment]
- modules/debugger.py:77: error: Incompatible types in assignment (expression has type "Event[Session]", base class "Debugger" defined the type as "Event[[Session]]")  [assignment]
- modules/debugger.py:78: error: Incompatible types in assignment (expression has type "Event[Session]", base class "Debugger" defined the type as "Event[[Session]]")  [assignment]
- modules/debugger.py:80: error: Incompatible types in assignment (expression has type "Event[Session]", base class "Debugger" defined the type as "Event[[Session]]")  [assignment]
- modules/debugger.py:81: error: Incompatible types in assignment (expression has type "Event[Session]", base class "Debugger" defined the type as "Event[[Session]]")  [assignment]
- modules/debugger.py:82: error: Incompatible types in assignment (expression has type "Event[Session]", base class "Debugger" defined the type as "Event[[Session]]")  [assignment]
- modules/debugger.py:83: error: Incompatible types in assignment (expression has type "Event[Session]", base class "Debugger" defined the type as "Event[[Session]]")  [assignment]
- modules/debugger.py:84: error: Bracketed expression "[...]" is not valid as a type  [valid-type]
- modules/debugger.py:84: note: Did you mean "List[...]"?
- modules/debugger.py:85: error: Type application has too many types (1 expected)  [misc]
- modules/debugger.py:112: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[str], Future[None]]"; expected "Callable[[VarArg(str), KwArg(str)], Any]"  [arg-type]
- modules/debugger.py:113: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[SourceLocation], Any]"; expected "Callable[[VarArg(SourceLocation), KwArg(SourceLocation)], Any]"  [arg-type]
- modules/debugger.py:127: error: Argument 1 to "add" of "Event" has incompatible type "Callable[[], Any]"; expected "Callable[[VarArg(None), KwArg(None)], Any]"  [arg-type]
- modules/debugger.py:141: error: Cannot determine type of "on_session_active"  [has-type]
- modules/debugger.py:363: error: Cannot determine type of "on_session_active"  [has-type]
- modules/debugger.py:370: error: Cannot determine type of "on_session_modules_updated"  [has-type]
- modules/debugger.py:373: error: Cannot determine type of "on_session_sources_updated"  [has-type]
- modules/debugger.py:376: error: Cannot determine type of "on_session_variables_updated"  [has-type]
- modules/debugger.py:379: error: Cannot determine type of "on_session_threads_updated"  [has-type]
- modules/debugger.py:387: error: Cannot determine type of "on_session_added"  [has-type]
- modules/debugger.py:388: error: Cannot determine type of "on_session_active"  [has-type]
- modules/debugger.py:394: error: Cannot determine type of "on_session_removed"  [has-type]
- modules/debugger.py:406: error: Cannot determine type of "on_session_active"  [has-type]
- modules/debugger.py:498: error: Cannot determine type of "on_session_active"  [has-type]

discord.py (https://github.com/Rapptz/discord.py)
- discord/ext/commands/bot.py:86: error: Bracketed expression "[...]" is not valid as a type  [valid-type]
- discord/ext/commands/bot.py:86: note: Did you mean "List[...]"?

[ilevkivskyi]

OK, I checked the mypy_primer, all fixes look real. The few new errors are because some people expect Callable[Concatenate[int, ...], int] to behave in Any-like manner (similar to as Callable[..., int] does). But we don't allow this (at least for now, there are few requests to allow this, maybe we can do this in the future).

[tmke8]

some people expect Callable[Concatenate[int, ...], int] to behave in Any-like manner (similar to as Callable[..., int] does). But we don't allow this (at least for now, there are few requests to allow this, maybe we can do this in the future).

pyright recently decided to support this, so that's maybe why people are using it:

• Add support for using ellipsis as last argument to Concatenate microsoft/pyright#5715

[A5rocks]

I'm in favor of ... having "Any-like" behavior cause that's useful!

(I kinda thought that was already in agreement and just that nobody updated mypy xd; like in my mind the mypy issue was solely about syntactically representing a construct that could be constructed other ways earlier)

Cause the PEP's vague about ellipsis but does say this (in context of Callable[..., <something>]):

an ellipsis in place of parameter types is specified to mean that we do no validation on arguments

[ilevkivskyi]

I kinda thought that was already in agreement and just that nobody updated mypy xd; like in my mind the mypy issue was solely about syntactically representing a construct that could be constructed other ways earlier

No, one can already "manually", define a callable like

def foo(x: int, *args: Any, **kwargs: Any) -> None: ...

and the question is like should we accept e.g. Callable[[int], None] as a subtype of it? It is not an obvious question, because normally Any affects just the type, but if we allow such subtyping, it would affect also argument kinds:

• Arguments in favour are e.g. is that plain ... behave this way, and also tuple[int, ...] means "ignore the length".
• Argument against is e.g. that often when people put such signature in a callback argument their intention is often "I don't care about types, but the callback should not fail if I dump a bunch of arguments there".

About the latter one, in such cases people should just use object instead of Any. But the problem is that 10 years after PEP 484 people still often confuse Any and object.

In any case, it should be a separate PR.

[ilevkivskyi]

This PR seems low risk, so I will be merging it unless there are objections.