Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: Ingestion Source Docs Template #4275

Merged
merged 22 commits into from
Mar 30, 2022
Merged

docs: Ingestion Source Docs Template #4275

merged 22 commits into from
Mar 30, 2022

Conversation

maggiehays
Copy link
Collaborator

@maggiehays maggiehays commented Mar 1, 2022 •
edited
Loading

This is meant to be a new template for Metadata Ingestion Sources; main improvements include:

  • Support Status indicator
  • Area for a plain-language overview of integration; what it's meant to do & where metadata is extracted from
  • Concept mapping between ingestion source & DataHub Metadata Model
  • Combining multiple ingestion jobs under a single source (ie. LookML & Looker Dashboards would live within a single file)
  • Standardized view of supported capabilities -- we need to figure out how to auto-populate this
  • Troubleshooting section to address common issues over time

Open Questions

  • How & where do we define Support Status?
  • How do we auto-generate capability support for each source? How do we avoid drift?
  • Which capabilities do we want to explicitly call out? How do we maintain this over time?

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

@github-actions
Copy link

github-actions bot commented Mar 1, 2022 •
edited
Loading

Unit Test Results (metadata ingestion)

       5 files         5 suites   1h 2m 16s ⏱️
   388 tests    388 ✔️   0 💤 0 ❌
1 787 runs  1 749 ✔️ 38 💤 0 ❌

Results for commit f7db645.

♻️ This comment has been updated with latest results.

@maggiehays maggiehays marked this pull request as ready for review March 1, 2022 17:05
@github-actions
Copy link

github-actions bot commented Mar 30, 2022 •
edited
Loading

Unit Test Results (build & test)

  96 files  +3    96 suites  +3   14m 37s ⏱️ +24s
686 tests +6  627 ✔️ +6  59 💤 ±0  0 ❌ ±0 

Results for commit f7db645. ± Comparison against base commit 14df3cf.

♻️ This comment has been updated with latest results.

@dexter-mh-lee dexter-mh-lee merged commit 0be0689 into datahub-project:master Mar 30, 2022
maggiehays added a commit to maggiehays/datahub that referenced this pull request Aug 1, 2022
@maggiehays
docs: Ingestion Source Docs Template (datahub-project#4275)
2e91672 
* testing img.shield for status

* update to hyperlink

* changing link format

* adding status options

* updating prerequisities and quickstart

* update to ingestion docs

* updating template with collapse details

* adding linebreak between pip install commands

* Removed incomplete sentence

* typo fix

* pushing current changes

* testing logos in markdown table

* markdown table fix

* markdown table fix

* adding in additional logos

* transposing markdown table

* settling on final table format

* adding commented-out source template to sidebar.js

* moving reference sidebar and adding trailing comma

* fixing docs build
@maggiehays maggiehays deleted the ingestion-source-template branch August 29, 2022 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anshbansal anshbansal anshbansal left review comments

@pedro93 pedro93 pedro93 approved these changes

@dexter-mh-lee dexter-mh-lee dexter-mh-lee approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@maggiehays @dexter-mh-lee @pedro93 @anshbansal