Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(fieldpaths): prevent duplicate field paths #10590

Merged

Conversation

david-leifker
Copy link
Collaborator

@david-leifker david-leifker commented May 24, 2024

Adding read/write mutator to protect against invalid field path columns. The duplicate columns are dropped. This prevents a UI bug from an ingestion error.

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub


private static boolean changeTypeFilter(BatchItem item) {
return !ChangeType.DELETE.equals(item.getChangeType())
&& !ChangeType.PATCH.equals(item.getChangeType());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the entity-registry config PATCH is marked as a supported type.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the idea here is that it can't occur for patches because patches require a keyed operation which prevents duplicates, but @david-leifker please verify.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PATCH is not needed since mutation occurs after PATCH change type has been converted to UPSERT

@david-leifker david-leifker merged commit 250a22c into datahub-project:master May 28, 2024
62 of 63 checks passed
aabharti-visa pushed a commit to aabharti-visa/datahub that referenced this pull request May 30, 2024
sleeperdeep pushed a commit to sleeperdeep/datahub that referenced this pull request Jun 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants