Fix the type of optional[int] in nested dataclass

[pingsutw]

Signed-off-by: Kevin Su [email protected]

TL;DR

The value of optional[int] in nested dataclass is converted to float. #1135 doesn't handle optional[int] in the nested dataclasses

In this pr, forced cast the value of optional[int] in the dataclass or nested databases to an integer.

Type

• Bug Fix
• Feature
• Plugin

Are all requirements met?

• Code completed
• Smoke tested
• Unit tests added
• Code documentation added
• Any pending items have an associated Issue

Complete description

The code to reproduce

from dataclasses import dataclass
from typing import Optional
from dataclasses_json import dataclass_json
from flytekit import task, workflow, dynamic


@dataclass_json
@dataclass
class Args(object):
    max_iter: int = 0
    init_eval: int = 0
    max_eval: int = 0


@dataclass_json
@dataclass
class Schema(object):
    kwargs: Optional[Args] = None

@task
def t1(schema: Schema):
    print(schema)


@workflow
def wf(schema: Schema = Schema(Args(1, 2, 3))):
    t1(schema=schema)


if __name__ == "__main__":
    wf(schema=Schema(Args(1, 2, 3)))

Tracking Issue

flyteorg/flyte#2839

Follow-up issue

NA

[codecov]

Codecov Report

Merging #1148 (d378c7f) into master (46a6129) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #1148   +/-   ##
=======================================
  Coverage   68.43%   68.44%           
=======================================
  Files         288      288           
  Lines       26032    26036    +4     
  Branches     2915     2916    +1     
=======================================
+ Hits        17816    17820    +4     
  Misses       7736     7736           
  Partials      480      480           

Impacted Files	Coverage Δ
flytekit/core/type_engine.py	59.34% <100.00%> (+0.09%)	⬆️
tests/flytekit/unit/core/test_type_engine.py	98.39% <100.00%> (+<0.01%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[wild-endeavor]

Does this work if the user doesn't do Optional, but instead does typing.Union[int, None].

Also unrelated, can we add a unit test to document what happens in the case of typing.Union[int, float] (where the value is like 5 or something, something that's an int?

[pingsutw]

Marshmellow doesn't support union type. marshmallow-code/marshmallow#1191
Therefore, we can't use typing.Union[int, float] in dataclass. btw, Union[None, int] is also an optional type, but Marshmallow does not support it.

will also update the doc of the dataclass example in flytesnack.