Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ingest): add fix to tableau connector when table has None fields #4445

Merged

Conversation

nandacamargo
Copy link
Contributor

Checklist

Co-authored-by: Ludmila Ferreira [email protected]

Hello!

We are using Datahub for ingesting metadata from Tableau. We were facing an issue when ingesting some Tableau data sources. The execution was crashing in the middle when data was coming with None fields, as shown in the following stack trace:

File "/usr/local/lib/python3.8/site-packages/datahub/ingestion/source/tableau.py", line 570, in emit_datasource
    upstream_tables = self._create_upstream_table_lineage(datasource, project)
File "/usr/local/lib/python3.8/site-packages/datahub/ingestion/source/tableau.py", line 257, in _create_upstream_table_lineage
    table_urn = make_table_urn(
File "/usr/local/lib/python3.8/site-packages/datahub/ingestion/source/tableau_common.py", line 379, in make_table_urn
    final_name = full_name.replace("[", "").replace("]", "")

AttributeError: 'NoneType' object has no attribute 'replace'

This happens when table object has this structure:

{'id': '1ca95eea-fab3-643b-110e-e91af4b0ea74', 'name': None, 'schema': None, 'fullName': None, 'connectionType': None, 'description': None, 'columns': [{'name': None, 'remoteType': 'WSTR'}, {'name': None, 'remoteType': 'I8'}, {'name': None, 'remoteType': 'DATE'}, {'name': None, 'remoteType': 'DATE'}, {'name': None, 'remoteType': 'R8'}, {'name': None, 'remoteType': 'I8'}, {'name': None, 'remoteType': 'R8'}, {'name': None, 'remoteType': 'I8'}]}

This fix is to treat scenarios like this one mentioned above.

Thanks a lot!

Copy link
Contributor

@treff7es treff7es left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, thanks for the fix

@github-actions
Copy link

github-actions bot commented Mar 18, 2022

Unit Test Results (build & test)

  92 files  ±0    92 suites  ±0   12m 40s ⏱️ - 22m 13s
674 tests ±0  615 ✔️ ±0  59 💤 ±0  0 ±0 

Results for commit 85e2afd. ± Comparison against base commit 0a522e5.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

Unit Test Results (metadata ingestion)

       5 files  ±0         5 suites  ±0   48m 15s ⏱️ + 1m 33s
   363 tests ±0     363 ✔️ ±0    0 💤 ±0  0 ±0 
1 662 runs  ±0  1 624 ✔️  - 7  38 💤 +7  0 ±0 

Results for commit 85e2afd. ± Comparison against base commit 0a522e5.

Copy link
Collaborator

@jjoyce0510 jjoyce0510 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you!

@treff7es treff7es merged commit e290e6e into datahub-project:master Mar 22, 2022
maggiehays pushed a commit to maggiehays/datahub that referenced this pull request Aug 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants