fix(ingest): tableau - fix chart custom properties None key error, up…

…date docs (#4931)
datahub-project · May 25, 2022 · 6f553d4 · 6f553d4
1 parent 65cf005
commit 6f553d4
Show file tree

Hide file tree

Showing 3 changed files with 26 additions and 7 deletions.
diff --git a/metadata-ingestion/docs/sources/tableau/tableau.md b/metadata-ingestion/docs/sources/tableau/tableau.md
@@ -20,15 +20,15 @@ This ingestion source maps the following Source System Concepts to DataHub Conce
 
 | Source Concept | DataHub Concept | Notes |
 | -- | -- | -- |
-| `Tableau` | [Data Platform](../../metamodel/entities/dataPlatform.md) | |
-| Embedded DataSource | [Dataset](../../metamodel/entities/dataset.md) | |
-| Published DataSource | [Dataset](../../metamodel/entities/dataset.md) | |
-| Custom SQL Table | [Dataset](../../metamodel/entities/dataset.md) | |
+| `"Tableau"` | [Data Platform](../../metamodel/entities/dataPlatform.md) | |
+| Embedded DataSource | [Dataset](../../metamodel/entities/dataset.md) | SubType `"Embedded Data Source"` |
+| Published DataSource | [Dataset](../../metamodel/entities/dataset.md) | SubType `"Published Data Source"` |
+| Custom SQL Table | [Dataset](../../metamodel/entities/dataset.md) | SubTypes `"View"`, `"Custom SQL"` |
 | Embedded or External Tables | [Dataset](../../metamodel/entities/dataset.md) | |
 | Sheet | [Chart](../../metamodel/entities/chart.md) | |
 | Dashboard | [Dashboard](../../metamodel/entities/dashboard.md) | |
 | User | [User (a.k.a CorpUser)](../../metamodel/entities/corpuser.md) | |
-| Workbook | [Container](../../metamodel/entities/container.md) | | 
+| Workbook | [Container](../../metamodel/entities/container.md) | SubType `"Workbook"` | 
 | Tag | [Tag](../../metamodel/entities/tag.md) | | 
 
 
@@ -410,4 +410,6 @@ Lineage is emitted as received from Tableau's metadata API for
 
 ### Why are only some workbooks ingested from the specified project?
 
-This happens when the Tableau API returns NODE_LIMIT_EXCEEDED error and returns partial results with message "Showing partial results. , The request exceeded the ‘n’ node limit. Use pagination, additional filtering, or both in the query to adjust results." To resolve this, reduce the page size using the `workbooks_page_size` config param (Defaults to 10).
+This may happen when the Tableau API returns NODE_LIMIT_EXCEEDED error in response to metadata query and returns partial results with message "Showing partial results. , The request exceeded the ‘n’ node limit. Use pagination, additional filtering, or both in the query to adjust results." To resolve this, consider 
+- reducing the page size using the `workbooks_page_size` config param in datahub recipe (Defaults to 10).
+- increasing tableau configuration [metadata query node limit](https://help.tableau.com/current/server/en-us/cli_configuration-set_tsm.htm#metadata_nodelimit) to higher value. 
diff --git a/metadata-ingestion/src/datahub/ingestion/source/tableau.py b/metadata-ingestion/src/datahub/ingestion/source/tableau.py
@@ -901,11 +901,13 @@ def emit_sheets_as_charts(self, workbook: Dict) -> Iterable[MetadataWorkUnit]:
                 sheet_external_url = None
             fields = {}
             for field in sheet.get("datasourceFields", ""):
+                name = get_field_value_in_sheet(field, "name")
                 description = make_description_from_params(
                     get_field_value_in_sheet(field, "description"),
                     get_field_value_in_sheet(field, "formula"),
                 )
-                fields[get_field_value_in_sheet(field, "name")] = description
+                if name:
+                    fields[name] = description
 
             # datasource urn
             datasource_urn = []

diff --git a/metadata-ingestion/tests/integration/tableau/setup/workbooksConnection_all.json b/metadata-ingestion/tests/integration/tableau/setup/workbooksConnection_all.json
@@ -29,6 +29,21 @@
                                 }
                             ],
                             "datasourceFields": [
+                                {
+                                    "__typename": "ColumnField",
+                                    "id": "xxxdc8ee-30e5-644b-cf76-48a3dea79cda",
+                                    "name": null,
+                                    "description": null,
+                                    "datasource": {
+                                        "__typename": "EmbeddedDatasource",
+                                        "id": "801c95e3-b07e-7bfe-3789-a561c7beccd3",
+                                        "name": "Marketo"
+                                    },
+                                    "dataCategory": "NOMINAL",
+                                    "role": "DIMENSION",
+                                    "dataType": "STRING",
+                                    "aggregation": null
+                                },
                                 {
                                     "__typename": "ColumnField",
                                     "id": "277dc8ee-30e5-644b-cf76-48a3dea79cda",