Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Azure Workload Identity for fsspec in flytekit #1813

Merged
merged 1 commit into from
Sep 1, 2023
Merged

Enable Azure Workload Identity for fsspec in flytekit #1813

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions flytekit/core/data_persistence.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,9 @@ def get_filesystem(
if anonymous:
kwargs["token"] = _ANON
return fsspec.filesystem(protocol, **kwargs) # type: ignore
elif protocol == "abfs":
kwargs["anon"] = False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this the correct behavior for abfs? anonymous is always false? is there not an anonymous construct for abfs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It really looks like that the purpose of anon = True is to disable using Azure DefaultCredentials in adlfs. Even with always anon = False you can still connect via storage account SAS token or storage account key.

Would you suggest to set: kwargs["anon"] = anonymous?

return fsspec.filesystem(protocol, **kwargs) # type: ignore

# Preserve old behavior of returning None for file systems that don't have an explicit anonymous option.
if anonymous:
Expand Down