[BUG] Failure during flytekit remote launch w/ dataclass flytekit.core.type_engine.TypeTransformerFailedError: Type of Val 'XXXX' is not an instance of <class 'XXXX'>

[CalvinLeather]

Describe the bug

Originally reported in https://flyte-org.slack.com/archives/CREL4QVAQ/p1661266231263149, also encountered by our team. When launching a remote execution of a task that takes a dataclass as input, the type is incorrectly inferred. This results in a failure in flytekit before the task is launched. This error looks like:

flytekit.core.type_engine.TypeTransformerFailedError: Type of Val '<Level2proxyparametersSchema(many=False)>' is not an instance of <class 'types.Level2proxyparametersSchema'>

We also found something that may be a related bug... if we do not pass default_project and default_domain when creating the remote, we get a related error (missing project/domain) when actually launching the launchplan/worfklow.

The workaround is to use type_hints to pass in the class for the dataclass explicitly.

This may be related to #2502, but is happening for dataclasses.

Expected behavior

The workflow/launch (which exists) launches and the dataclass provided is accepted.

Additional context to reproduce

No response

Screenshots

Screenshot of related issue from not setting default_project/domain in remote init

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[CalvinLeather]

Some discussion of this happened in slack

[eapolinario]

@pingsutw , can we close this?

[pingsutw]

yes, closed it.