feat(ingest): 🔒 - Add AWS IAM Roles Support to the Glue Source

[afranzi]

This PR aims to add IAM Roles support to the existing Glue Source. This could allow importing the Glue Metadata cross AWS accounts by chaining roles or by allowing the current instance to assume an external role.

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable)

[afranzi]

@shirshanka, I'm not sure why the tests are failing since they are unrelated to my PR 🤔

[hsheth2]

Code LGTM

The CI failure is due to mypy complaining about a typing issue:

> Task :metadata-ingestion:lint
src/datahub/ingestion/source/glue.py:100: error: Argument 3 to "reduce" has incompatible type "None"; expected "Dict[Any, Any]"
Found 1 error in 1 file (checked 138 source files)

[afranzi]

Code LGTM

The CI failure is due to mypy complaining about a typing issue:

> Task :metadata-ingestion:lint
src/datahub/ingestion/source/glue.py:100: error: Argument 3 to "reduce" has incompatible type "None"; expected "Dict[Any, Any]"
Found 1 error in 1 file (checked 138 source files)

But, with an credentials: Optional[dict] = None, shouldn't be possible to specify None? 🤔

[hsheth2]

Seems like your change should fix it 🎉

[afranzi]

Seems like your change should fix it 🎉

Yeap, fixed! :)

[shirshanka]

LGTM!