Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update log level for structured dataset #1394

Merged
merged 6 commits into from
Jan 5, 2023
Merged

Update log level for structured dataset #1394

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1a7a147
Update log level for structured dataset
pingsutw Dec 26, 2022
c8192ae
update to debug
pingsutw Dec 28, 2022
18ce68c
Merge branch 'master' of github.com:flyteorg/flytekit into debug-1
pingsutw Dec 28, 2022
cb24e93
Merge branch 'master' into debug-1
pingsutw Dec 29, 2022
4c87cca
Update
pingsutw Jan 4, 2023
6eda385
Merge branch 'master' into debug-1
wild-endeavor Jan 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions flytekit/types/structured/structured_dataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -483,17 +483,18 @@ def register_for_protocol(

if (default_format_for_type or default_for_type) and h.supported_format != GENERIC_FORMAT:
if h.python_type in cls.DEFAULT_FORMATS and not override:
logger.warning(
f"Not using handler {h} with format {h.supported_format} as default for {h.python_type}, {cls.DEFAULT_FORMATS[h.python_type]} already specified."
)
if cls.DEFAULT_FORMATS[h.python_type] != h.supported_format:
logger.debug(
f"Not using handler {h} with format {h.supported_format} as default for {h.python_type}, {cls.DEFAULT_FORMATS[h.python_type]} already specified."
)
else:
logger.debug(
f"Setting format {h.supported_format} for dataframes of type {h.python_type} from handler {h}"
)
cls.DEFAULT_FORMATS[h.python_type] = h.supported_format
if default_storage_for_type or default_for_type:
if h.protocol in cls.DEFAULT_PROTOCOLS and not override:
logger.warning(
logger.debug(
f"Not using handler {h} with storage protocol {h.protocol} as default for {h.python_type}, {cls.DEFAULT_PROTOCOLS[h.python_type]} already specified."
)
else:
Expand Down