Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ingest): add Python libs for Urns #4172

Merged
merged 3 commits into from
Feb 23, 2022

Conversation

tc350981
Copy link
Contributor

@tc350981 tc350981 commented Feb 17, 2022

In the data model we define Urn as String which make it hard to manipulate. This PR creates python lib for Urn, DatasetUrn, DataPlatformUrn classes which is meant to help interact with Urn easier.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

@github-actions
Copy link

github-actions bot commented Feb 17, 2022

Unit Test Results (metadata ingestion)

       5 files         5 suites   44m 15s ⏱️
   345 tests    345 ✔️   0 💤 0
1 572 runs  1 534 ✔️ 38 💤 0

Results for commit d3bf51c.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Feb 17, 2022

Unit Test Results (build & test)

  70 files  ±0    70 suites  ±0   11m 11s ⏱️ -51s
611 tests +2  552 ✔️ +2  59 💤 ±0  0 ±0 

Results for commit d3bf51c. ± Comparison against base commit c89a1f2.

♻️ This comment has been updated with latest results.

Copy link
Contributor

@shirshanka shirshanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great work and will make working with Urns much nicer. Thanks!
Have a few suggestions for you.

Also, how do you think we should transition from the make_x_urn pattern in mce_builder.py, since all connectors use that pathway today.

@shirshanka shirshanka added the pending-submitter-response Issue/request has been reviewed but requires a response from the submitter label Feb 20, 2022
@tc350981
Copy link
Contributor Author

This is great work and will make working with Urns much nicer. Thanks! Have a few suggestions for you.

Also, how do you think we should transition from the make_x_urn pattern in mce_builder.py, since all connectors use that pathway today.

maybe we could do it in phases, like:

  1. change the implementation of make_x_urn to use x Urn class
  2. Use Urn classes in new code
  3. Soft deprecate the make_x_urn util function

Copy link
Contributor

@shirshanka shirshanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@shirshanka shirshanka merged commit aa3363b into datahub-project:master Feb 23, 2022
@shirshanka shirshanka removed the pending-submitter-response Issue/request has been reviewed but requires a response from the submitter label Mar 31, 2022
maggiehays pushed a commit to maggiehays/datahub that referenced this pull request Aug 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants