Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Cannot return StructuredDataset which is a field in a dataclass #6117

Open
2 tasks done
gvogel-hh opened this issue Dec 19, 2024 · 1 comment
Open
2 tasks done
Assignees
Labels
backlogged For internal use. Reserved for contributor team workflow. bug Something isn't working flytekit FlyteKit Python related issue

Comments

@gvogel-hh
Copy link

Describe the bug

The following code fails in the extract task with AttributeError: 'StructuredDataset' object has no attribute 'to_flyte_idl'.

@dataclass
class Data:
    f: StructuredDataset


@task
def create_data() -> Data:
    return Data(f=StructuredDataset(dataframe=pd.DataFrame({"a": [5]})))


@task
def extract(d: Data) -> StructuredDataset:
    return d.f


@workflow
def example_wf() -> None:
    d = create_data()
    f = extract(d=d)

It is possible to work around the issue by returning StructuredDataset(dataframe=d.f.open(pd.DataFrame).all()) instead.

Expected behavior

The field value should be returned by the task.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@gvogel-hh gvogel-hh added bug Something isn't working untriaged This issues has not yet been looked at by the Maintainers labels Dec 19, 2024
@eapolinario eapolinario added flytekit FlyteKit Python related issue backlogged For internal use. Reserved for contributor team workflow. and removed untriaged This issues has not yet been looked at by the Maintainers labels Dec 19, 2024
@eapolinario
Copy link
Contributor

Let's confirm it repros in flytekit 1.14.0.

@davidmirror-ops davidmirror-ops moved this from Backlog to In progress in Flyte Issues/PRs maintenance Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backlogged For internal use. Reserved for contributor team workflow. bug Something isn't working flytekit FlyteKit Python related issue
Projects
Status: In progress
Development

No branches or pull requests

3 participants