Add looker metadata ingestion examples

Intended as examples we used for a POC, not a fully featured and reliable metadata ingestion
datahub-project · Mar 30, 2021 · 7eceaca · 7eceaca
1 parent a5b206a
commit 7eceaca
Show file tree

Hide file tree

Showing 6 changed files with 797 additions and 0 deletions.
diff --git a/metadata-ingestion/examples/looker/dashboard_ingestion/README.md b/metadata-ingestion/examples/looker/dashboard_ingestion/README.md
@@ -0,0 +1,14 @@
+## looker_dashboard_ingestion.py
+This tool helps ingest looker dashboard/looker lineage into datahub. 
+
+The script assumes you already have run lookml_ingestion.py to scrape view and explore definitions into datahub.
+
+
+## Steps:
+- Use a version of python >= 3.7
+- Make a virtual environment
+- pip install -r requirements.txt
+- Set env vars: LOOKERSDK_CLIENT_ID, LOOKERSDK_CLIENT_SECRET, LOOKERSDK_BASE_URL
+- Configure extra kafka conf in looker_dashboard_ingestion.py
+
+python looker_dashboard_ingestion.py