Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(policies): Make policies searchable by privilege, type, status or editable fields #9877

Merged
merged 7 commits into from
Feb 26, 2024

Conversation

pedro93
Copy link
Collaborator

@pedro93 pedro93 commented Feb 16, 2024

Adds searchable annotations to DataHubPolicyInfo fields: privilege, type, status and editable fields so that we can use these fields for filtering in graphQL queries + a filter parameter when listing policies.

For backfilling pre-existing policies, adds a system update step to RESTATE existing policies. This is enabled by default so the mae processor updates documents in ES for all pre-existing DataHub policies.

Tested it works by checking elasticsearch directly to see new properties are available:
Screenshot 2024-02-19 at 11 40 41

And that using the graphQL command I am able to filter by the new features:
Screenshot 2024-02-26 at 10 28 40

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the devops PR or Issue related to DataHub backend & deployment label Feb 16, 2024
@pedro93 pedro93 changed the title feat(policies): Make policies searchable by privilege, type, status or editable fields [WIP] feat(policies): Make policies searchable by privilege, type, status or editable fields-- Feb 16, 2024
@Searchable = {
"/*": {
"fieldType": "KEYWORD",
"addToFilters": true
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What doesaddToFilters allow us to do?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://datahubproject.io/docs/next/metadata-modeling/extending-the-metadata-model/#searchable - This is to allow the array values to be used as filters from what I understand.

@pedro93 pedro93 changed the title [WIP] feat(policies): Make policies searchable by privilege, type, status or editable fields-- feat(policies): Make policies searchable by privilege, type, status or editable fields-- Feb 19, 2024
@pedro93 pedro93 changed the title feat(policies): Make policies searchable by privilege, type, status or editable fields-- feat(policies): Make policies searchable by privilege, type, status or editable fields Feb 19, 2024
@pedro93 pedro93 merged commit 21ecb71 into master Feb 26, 2024
59 checks passed
@pedro93 pedro93 deleted the feature/prd-1021 branch February 26, 2024 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops PR or Issue related to DataHub backend & deployment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants