DataHub v0.8.32

Release Highlights

User Experience

We're excited to announce View-based RBAC Policies! You can now create and apply view-only permissions to your DataHub end-users, providing more robust access controls.

We've also included some small (but impactful!) improvements to UX, including:

• Display recent search terms when beginning the search flow
• Consistently displaying entity subtypes for dbt, Looker, Kafka, & more. Think: Kafka entities are displayed as "topics" instead of "datasets"

Ingestion Highlights

• New! Protobuf ingestion (shoutout to @leifker for this Community-led contribution!)
• Initial work to support a "Notebook" entity (shoutout to @tc350981 for spearheading this work!!)
• Stateful ingestion for dbt is now supported
• Ongoing improvements to our Tableau ingestion source from @nandacamargo & @cuong-pham
• Improvements to handling database aliases for Redshift ingestion
• Improvements to S3 source:
  • Add containers for datasets
  • Support platform_instance
  • Support for folder level datasets
  • Increased flexibility to specify dataset paths
• Ingestion Fixes:
  • Snowflake Usage - log warning instead of error out & other error handling
  • Snowflake allow/deny patterns
  • Examples of allow/deny patterns added to docs

Full Commit Log

• #4570 @gabe-lyons fix(search): handle commas in search queries in the UI
• #4557 @daha fix: replace direct and indirect references to linkedin with datahub-project
• #4569 @dexter-mh-lee fix(policy): Add view entity page priv to all entity types
• #4567@anshbansal fix(bigquery): missing dependency
• #4548 @mayurinehate fix(tableau): gracefully stop ingestion if tableau sign in not successful
• #4564 @dexter-mh-lee fix(docs): fix logo links on ingestion docs
• #4396 @Abhiram98 feat(ingestion): schema, table filtering for redshift-usage
• #4554 @darapuk (fix): Update path generated when creating LookML URL
• #4549 @maggiehays docs: add sumup logo
• #4560 @mhw docs: Fix PostgreSQL typo in features.md
• #4562 @gabe-lyons feat(lineage): show fully qualified dataset name on expansion
• #4561 @anshbansal fix: dependencies for usage sources
• #3782 @CorentinDuhamel feat(ingest): indent sql queries for usage sources
• #4551 @pedro93 fix(rollback) Removes status & key aspects from affected aspects count during rollback
• #4538 @dexter-mh-lee fix(policy): Remove all from the resource type choices
• #4544 @anshbansal fix(ingest): snowflake-usage - log warning instead of error out
• #4542 @mattmatravers build(ui): allow custom nodeDistBaseUrl
• #4545 @mayurinehate fix(kafka-connect): add platform for default case in jdbc connector, update tests for platform instance map
• #4547 @anshbansal chore: update pull request template
• #4537 @RyanHolstien fix(dataPlatformInstance): add data platform instance to entity registry
• #4375 @mayurinehate fix(kafka-connect): fix lineage for postgres-like 3-level hierarchy d…
• #4492 @RyanHolstien fix(cli): delete - handle case insensitive entity types
• #4130 @sgomezvillamor feat(ingest): glue - adds platform instance capability
• #4456 @mohdsiddique feat(stateful dbt): add stateful ingestion capability in dbt source
• #4482 @pedro93 feat(platform): adds side-effect report for rollbacks
• #4275 @maggiehays docs: Ingestion Source Docs Template
• #4470 @mayurinehate feat(tableau): emit lineage edge from embedded datasource to upstream…
• #4535 @pppsunil feat(ingestion): Support pluggable Schema Registry for Kafka Source
• #4369 @eburairu feat(ui): Add new loading pattern logo
• #4493 @leifker feat(integration): protobuf - additional annotations and features
• #4435 @zhoxie-cisco perf(docker): datahub-gms - add jetty configuration xml
• #4532 @anshbansal doc: add example of profiling in default example
• #4533 @anshbansal doc: clarify CLI releases
• #4528 @daha fix(doc): Change to forward slash-separated strings, as in the example
• #4315 @daha fix(docs): Minor fixes to pip install commands
• #4521 @anshbansal doc: update docker docs for mentioning Python CLI
• #4523 @kevinhu fix(ingest): mssql - support database_alias
• #4501 @arunvasudevan feat(ingest): kafka-connect - support mapping for multiple DB instances
• #4477 @jjoyce0510 feat(metadata service): Introducing Platform Events
• #4526 @jjoyce0510 Adding has container
• #4494 @cuong-pham fix(ingest): make tableau ingestion more resilient to error
• #4519 @anshbansal doc(ingestion): add examples of running in docker and Kubernetes
• #4485 @andres-lowrie docs(metadata-ingestion): callout props in para
• #4511 @RyanHolstien Oss/urn validation
• #4525 @dexter-mh-lee feat(policy): Add tooltip and view button
• #4507 @mayurinehate feat(assertion): update python example, assertion entity doc
• #4513 @kevinhu feat(ingestion): detect and disable telemetry in CI
• #4516 @dexter-mh-lee feat(policy): Add domain based and view based policies
• #4520 @rslanka Fix: Snowflake Table to View lineage
• #4510 @anshbansal feat(ingest): Add config to improve user exp for initial ingestion and fix docs
• #4517 @anshbansal feat(ingest): option for number of workunits in preview
• #4503 @treff7es feat(ingest): athena - set Athena location as upstream
• #4490 @MugdhaHardikar-GSLab feat(s3): add s3 source
• #4468 @tc350981 feat(notebook): graphqul related logic change for notebook
• #4401 @anshbansal doc: update instructions for updating DataHub on quickstart
• #4500 @ShubhamThakre SecretBuilderModal -> name field validation updated
• #4505 @anshbansal docs: add example of database and schema allow/deny patterns
• #4504 @anshbansal fix(snowflake): allow/deny patterns
• #4496 @shirshanka feat(ingest): dbt,looker,sql_common,kafka - moving sources to produce display names and subtypes more consistently
• #4480 @anshbansal feat(snowflake): stop querying for usage data when no mix/max dates
• #4483 @anshbansal fix(snowflake-usage): do not ingest for stage as a dataaset
• #4497 @shirshanka moving to dockerhub for actions container
• #4475 @darapuk fix: Update GroupProfile to read from properties over deprecated info aspect
• #4489 @anshbansal fix(ingestion): pin Jinja2 to version < 3.1.0
• #4484 @anshbansal fix(ingestion): stop CLI build failures
• #4467 @anshbansal doc: add caveats to snowflake doc
• #4481 @anshbansal fix(snowflake): don't recommend accountadmin role for snowflake
• #4479 @anshbansal fix: change log level to debug
• #4476 @dexter-mh-lee Add recent searches filtering
• #4469 @tc350981 feat(ingest): add python utility classes for NotebookUrn, CorpuserUrn and CorpGroupUrn
• #4439 @ShubhamThakre Feature/modal-validation-and-UI-fixes-updates
• #4398 @kevinhu feat(ingest): simplify event IDs for function invocations
• #4474 @sgomezvillamor chore: acryl-data 0.6.12
• #4473 @treff7es fix(redshift) Properly handling database alias in redshift usage and redshift lineage generation
• #4471 @gabe-lyons enabling ml tabs
• #4460 @eclaassen-pb fix: java dependency vulnerabilities
• #4453 @treff7es feat(ingest) data-lake: Add s3 properties metadata when ingesting s3 files
• #4464 @anshbansal fix: change for repository change
• #4466 @anshbansal fix(snowflake-usage): add more error handling
• #4445 @nandacamargo fix(ingest): add fix to tableau connector when table has None fields
• #4457 @mayurinehate docs(hive): update recipe with example to specify kerberos auth
• #4313 @pedro-iatzky fix(ingest): bigquery - fix ingestion of external tables
• #4462 @gabe-lyons adding final transport options
• #4223 @tc350981 feat(notebook): add data models for Notebook entity
• #4450 @pedro93 feat(frontend) Adds multiple group claim support
• #4442 @anshbansal feat(ingestion): snowflake, bigquery - enhancements to log and bugfix
• #4333 @anshbansal doc: add guide for ui tabs
• #4443 @jjoyce0510 Fixing privilege option display bug
• #4451 @gabe-lyons fix tableau connector when it cannot connect to URI
• #4237 @tc350981 (docs) add RFC file to introduce Notebook entity data model
• #4446 @kevinneville fix: Replace old repository link with new link
• #4447 @cuong-pham getting database directly from upstream tables incase there are multiple databases in upstreamDatabases
• #4436 @treff7es Passing entity properly on deletion
• #4433 @rslanka Fix bug in the SchemaField type computation for AVRO logical types.

DataHub v0.8.31

Bugfix release to prevent failing reindexing of system metadata index in elasticsearch

Full Commit Log

• #4440 @pedro93 fix(cli) Makes filtered search deletes include BOTH removed and non-removed
• #4444 @pedro93 fix(cli) Adds elasticsearch mapping
• #4432 @leifker feat(protobuf): Gradle protobuf example project

Datahub v0.8.30

V0.8.30

Release Highlights

• Fix for OIDC encryption bug from v0.8.29
• Adds platform instance id to the container id generation, and support for migrating the old container ids to the new ones via the datahub migrate CLI.

Notable UI-Based Features

• Showing recent searches in autocomplete.

What's Changed

• fix(ui): some small ui fixes for lineage by @gabe-lyons in #4381
• fix(docs): change cabify link by @maaaikoool in #4373
• Fixed Bug: Alpha slider doesn’t move, only the color slider is movabe in tag color picker by @Ankit-Keshari-Vituity in #4359
• feat(GE): add option to disable sql parsing, use default parser by @mayurinehate in #4377
• fix(removed): Make sure removed entities do not appear on recommendations by @dexter-mh-lee in #4353
• fix(browse): fix browse double click issue by @gabe-lyons in #4382
• fix(oidc): Update group membership each login (and make group extraction disabled by default) by @jjoyce0510 in #4380
• feat(ingestion): add java protobuf schema ingestion by @leifker in #4178
• Docs/update docs by @RyanHolstien in #4393
• Revert "Fixed Bug: Alpha slider doesn’t move, only the color slider is movabe in tag color picker" by @gabe-lyons in #4390
• feat(ingestion): improve logging, docs for bigquery, snowflake, redshift by @anshbansal in #4344
• fix(ingest) Azure AD: support nested groups (#4367) by @cccs-eric in #4368
• fix: add missing logo by @anshbansal in #4386
• feat(spark-lineage): add support to custom env and platform_instance by @MugdhaHardikar-GSLab in #4208
• fix(containers) - configure domain resolver for containers by @aditya-radhakrishnan in #4404
• feat(*): Support setting owner type when assigning ownership by @jjoyce0510 in #4354
• fix: telemetry failure should not cause CLI failure by @anshbansal in #4406
• feat(autocomplete): Show recent searches + improved autocomplete by @jjoyce0510 in #4400
• fix(ingestion): Fix mypy error stateful committable & restore mypy version. by @rslanka in #4408
• build(markupsafe): update markupsafe pinning for Airflow compatibility by @set5think in #4388
• feat(search): Add flag to enable caching on search service by @dexter-mh-lee in #4335
• fix(query_combiner): add try block to handle queries of type str by @WaStCo in #4397
• fix(ingestion): read all tables from redshift by @Abhiram98 in #4345
• fix(ingestion): Invoke SqlLineageSQLParser's implementation in a separate process by @rslanka in #4391
• fix(ingest): handle endpoints without 200 response in openapi by @JorgenEvens in #4332
• feat(ingestion): Add the ability to query the latest timeseries aspect value via the get_cli. by @rslanka in #4395
• Refactoring the quries into a single one to get the search results on Home Page by @Ankit-Keshari-Vituity in #4372
• feat(lineage): hide soft deleted nodes in lineage & adds banner in entity page by @gabe-lyons in #4410
• fix(lineage): Move lineage registry to entity-registry module by @dexter-mh-lee in #4412
• feat(cli) Changes rollback behaviour to apply soft deletes by default by @pedro93 in #4358
• fix(looker): various looker fixes by @gabe-lyons in #4394
• fix(oidc): Fixing OIDC encryption bug in v0.8.29 by @jjoyce0510 in #4418
• feat(oidc): Adding support for extracting single string groups claim by @jjoyce0510 in #4419
• fix: change log levels to debug by @anshbansal in #4411
• tests(cypress): reduce cypress flakiness by retrying login on failure by @gabe-lyons in #4423
• fix(ingest): extract redshift platform correctly from sqlalchemy uri by @mayurinehate in #4421
• build: Fix line endings for Windows check-out by @mattmatravers in #4370
• feat(gql): make gql layer resistant to unresolvable relationships by @gabe-lyons in #4424
• fix(ingestion) containers: Adding platform instance to container keys by @treff7es in #4279
• fix: don't set None default by @anshbansal in #4422
• Flexible search on soft delete by @pedro93 in #4405
• fix(no-code metadata models in ui): fixes bug with rendering renderSpec aspects by @gabe-lyons in #4430

New Contributors

• @set5think made their first contribution in #4388
• @Abhiram98 made their first contribution in #4345
• @JorgenEvens made their first contribution in #4332
• @mattmatravers made their first contribution in #4370

Full Changelog: v0.8.29...v0.8.30

DataHub v0.8.29

v0.8.29

NOTICE

This version is affected by an OIDC (SSO) related issue with the following stack trace:

datahub-datahub-frontend-8d7f7cf6f-xvjwm datahub-frontend Caused by: java.security.InvalidKeyException: Invalid AES key length: 30 bytes
datahub-datahub-frontend-8d7f7cf6f-xvjwm datahub-frontend 	at com.sun.crypto.provider.AESCrypt.init(AESCrypt.java:87)

DataHub core team is working to address this. For now, we recommend staying on 0.8.28 if you are using OIDC actively!

Release Highlights

Fix for MAE & MCE consumer healthcheck
Upgrade to Java 11 and Gradle 6

Full Commit Log

• #4360 @maaaikoool Add cabify as adopter
• #4365 @dexter-mh-lee fix(vulnerabilities): Fix vulnerabilities in datahub-frontend
• #4361 @jjoyce0510 fix(ui): Supporting unknown data platform type
• #4363 @rslanka feat(ingest): Add memory leak detection capability to the datahub cli command.
• #4366 @RyanHolstien fix(metadata-jobs): fix root context for springboot
• #4340 @leifker feat(build): upgrade to gradle 6 for toolchain to support java 11
• #4357 @anshbansal feat: change quickstart to use head tag for actions
• #4356 @treff7es fix(ingest): bigquery - Fixing missing attribute error if credential was not set
• #4319 @vcs9 feat(ingest): mysql - add database_alias functionality
• #4352 @dexter-mh-lee fix(ci): fix model generation workflow
• #4351 @jjoyce0510 fix(frontend): Fix common OIDC issues
• #4111 @treff7es fix(ingest) bigquery-usage: Adding credential support for bigquery usage
• #4343 @Ankit-Keshari-Vituity Fixed the Small Project issue
• #4350 @MugdhaHardikar-GSLab fix(config-parsing): add support for variable expansion for in variables in between string
• #4330 @anshbansal fix(hive): clean protocol for hive source
• #4338 @maggiehays doc(platforms) adding PowerBI logo to docs website'
• #4342 @anshbansal feat(quickstart): restart actions pod in case of failures
• #4347 @mayurinehate fix(GE): fix dependencies for GE DataHubValidationAction, logic for s…
• #4349 @gabe-lyons query for custom properties on containers
• #4341 @anshbansal fix(doc): remove duplicate entry for permission
• #9 @shirshanka fix(ci): fix datahub jar publish action
• #8 @shirshanka feat(ci): fix jar publish action
• #7 @czbernard fix(ci): fixing tag computation for docker image build
• #6 @shirshanka feat(ci): adding dockerfile and action for datahub-airflow image
• #5 @shirshanka fix(ci): pin python version to 3.9.9 for release action
• #4 @dexter-mh-lee fix(ci): docker-ingestion - update acryl workflow
• #3 @shirshanka fix(pypi): fixing package metadata to reflect source and changelog correctly
• #2 @shirshanka fix(ci): check for tagged reference to kick off pypi push

DataHub v0.8.28

Release Highlights

Notable UI-Based Features

Quickly view, search, and filter the downstream dependencies of any Entity! By using the Impact Analysis Lineage view, you can now see the full set of downstream entities that may be impacted by a change to a given entity. You can also search, filter, and export the list of entities to CSV; try it for yourself here.

View Dataset- and Column-Level Data Validation outcomes in DataHub. We now support surfacing outcomes from Great Expectations validations in Dataset Entities! Easily view the full history of validation outcomes to understand the trustworthiness of your data.

User Groups, Policies, and Tags have a new look!

• The User Group page has a new look, allowing you to assign an email address, Slack Channel, Group Owner, and more. Easily add/remove Group Members from the UI - test it out here.
• We refreshed the Policies Page, allowing you to see Policy membership and status at a glance.
• The Tag Details page has been overhauled! You can now edit the definition, assigned owners, and tag color via the UI (try it here).

Notable Metadata Model & Ingestion-Based Features

First Milestone: Column-Level Lineage is complete! The Metadata Model now supports “fine-grained” lineage for Datasets; see documentation here for details, including adding fine-grained lineage to a dataset or a datajob.

Define Dataset-to-Dataset lineage via YAML. As demonstrated in the February 2022 Town Hall, you can now set Dataset-level lineage via YAML. This is great for teams that have more bespoke lineage needs that cannot be auto-extracted by the current set of supported ingestion sources.

Track all changes to entities using the Timeline API. This unified timeline of changes to entities in the metadata graph provides a robust picture of how your metadata has evolved over time. Upcoming work will support surfacing this detail via the DataHub UI. See the overview from Town Hall here.

Miscellaneous Metadata Ingestion Updates:

• Incubating: PowerBI Ingestion Source
• BigQuery Profiling: ability to disable profiling by partition
• Tableau improvements: Workbooks are now modeled as “Containers”

What's Changed

• doc(adoption): adds Adevinta as DataHub adopter by @sgomezvillamor in #4227
• fix(policies): Remove multiple privileges for GENERATE_PERSONAL_ACCESS_TOKEN by @jjoyce0510 in #4239
• Added Drawer to show the tag profile data by @Ankit-Keshari-Vituity in #4132
• fix(ui) Misc styling fixes (truncated filter values, updating tag color on clickaway) by @jjoyce0510 in #4246
• feat(ingest): switch telemetry endpoint to Mixpanel by @kevinhu in #4238
• docs: Add Udemy & Adevinta logos by @maggiehays in #4247
• fix(ingest) kafka-connect: Pass the env variable as part of making dataset by @arunvasudevan in #4244
• feat(lineage): Column level lineage model by @rslanka in #4248
• feat(docker): add multiplatform docker support for arm64 (m1) by @zhaofengnian18 in #4221
• fix(ingest): fix tableau sheets external url ingestion by @maaaikoool in #4231
• fix(docs): fix broken link by @anshbansal in #4242
• fix(docs): fix reference to the credential step in okta guide by @bskim45 in #4243
• feat(ingest): add ability to provide lineage described from within a file by @eddyv in #4116
• Add support for url_prefix in elastisearch source by @pppsunil in #4214
• feat(search): supporting chinese glossaryterm full text retrieval(#3914) by @Huyueeer in #3956
• feat(platform): Schema version history timeline. by @rslanka in #4252
• fix(mae-comsumer): wrong aspect name in usage event transformer by @zhoxie-cisco in #4249
• feat(ml): Add searchable annotation for features in feature table by @dexter-mh-lee in #4216
• fix(ingest): fix telemetry profile emission by @kevinhu in #4253
• bug(logo): add platform to chart relationship query by @RyanHolstien in #4255
• chore(docs): cleanup location of guide, gitignore generated from git by @anshbansal in #4256
• feat(ingest): Spark-free data lake ingestion by @kevinhu in #4131
• Openapi new auth by @vlavorini in #4086
• feat(docs) add fine-grained lineage docs and examples by @ksrinath in #4260
• feat(ingest): add option to copy URN, fix graphql docs by @anshbansal in #4209
• chore(managed ingestion): add variables for default val, update vals by @anshbansal in #4186
• docs(ui): Adding guide for adding users to DataHub. by @jjoyce0510 in #4262
• fix(docs): doc build failing by @anshbansal in #4267
• fix(doc) fix spelling mistake in dataset doc by @ksrinath in #4264
• feat(ingest): add lineage_client_project_id field to the BigQuery config by @vcs9 in #4138
• feat(graphql): Adding resolved users and groups to policies by @jjoyce0510 in #4272
• bug(schema_version_history): fix semantic version ordering by @RyanHolstien in #4271
• fix(recs ui): fixing tag color in recommendations by @jjoyce0510 in #4274
• fix(ingestion): Fix snowflake lineage + logging & reporting improvements. by @rslanka in #4276
• fix(docs): Error in running airflow locally by @buggythepirate in #4259
• feat: powerbi source plugin by @mohdsiddique in #4201
• test(timeline): fix smoke test by @RyanHolstien in #4285
• fix(ingest) - always display CLI version by @aditya-radhakrishnan in #4282
• feat(lineage) Bigquery: Supporting v2 audit metadata on Bigquery by @treff7es in #4233
• fix(recipe-parsing): fix recipe config parsing for $ by @MugdhaHardikar-GSLab in #4258
• docs: add details to update users when using helm by @anshbansal in #4268
• feat(model): adds PRE in the FabricType enum by @sgomezvillamor in #4226
• fix(ingestion): Fix snowflake view upstream lineages to eliminate false edges. by @rslanka in #4284
• fix(docs): fix frontend docs to replace port 9001 -> 9002 by @gabe-lyons in #4280
• fix(docker): use exec form to start container main process by @gmcoringa in #4245
• fix(ingestion): revert positional arg change by @anshbansal in #4266
• feat(profiling) - Bigquery: Ability to disable partition profiling by @treff7es in #4228
• fix(ingest): clarify s3/s3a requirements and platform defaults by @kevinhu in #4263
• Remove <> from add-users.md doc by @jjoyce0510 in #4293
• fix(ingestion): Fix bigquery stateful ingestion checkpoint reconstruction. by @rslanka in #4295
• fix(telemetry): telemetry fail should not cause the CLI to fail by @anshbansal in #4302
• fix(search): Update urn tokenizer to tokenize on periods and slashes by @dexter-mh-lee in #4085
• Fixed the UI issue: Height issue of editor and the spacing issue between logo and description by @Ankit-Keshari-Vituity in #4300
• fix(vulnerabilities): Fix new vulnerabilities by upgrading libraries by @dexter-mh-lee in #4297
• docs(readmes): Update module READMEs to reflect the current state of the world by @jjoyce0510 in #4294
• Resign the policy tab by @Ankit-Keshari-Vituity in #4232
• refactor(extractor): Move extractors to entity-registry by @dexter-mh-lee in #4307
• refactor(ui) Minor policies styling improvements. by @jjoyce0510 in #4309
• feat(ui): Introducing New group profile by @jjoyce0510 i...

DataHub Release Candidate v0.8.28 (rc1)

DataHub v0.8.28 Release Candidate 1

What's Changed

• doc(adoption): adds Adevinta as DataHub adopter by @sgomezvillamor in #4227
• fix(policies): Remove multiple privileges for GENERATE_PERSONAL_ACCESS_TOKEN by @jjoyce0510 in #4239
• Added Drawer to show the tag profile data by @Ankit-Keshari-Vituity in #4132
• fix(ui) Misc styling fixes (truncated filter values, updating tag color on clickaway) by @jjoyce0510 in #4246
• feat(ingest): switch telemetry endpoint to Mixpanel by @kevinhu in #4238
• docs: Add Udemy & Adevinta logos by @maggiehays in #4247
• fix(ingest) kafka-connect: Pass the env variable as part of making dataset by @arunvasudevan in #4244
• feat(lineage): Column level lineage model by @rslanka in #4248
• feat(docker): add multiplatform docker support for arm64 (m1) by @zhaofengnian18 in #4221
• fix(ingest): fix tableau sheets external url ingestion by @maaaikoool in #4231
• fix(docs): fix broken link by @anshbansal in #4242
• fix(docs): fix reference to the credential step in okta guide by @bskim45 in #4243
• feat(ingest): add ability to provide lineage described from within a file by @eddyv in #4116
• Add support for url_prefix in elastisearch source by @pppsunil in #4214
• feat(search): supporting chinese glossaryterm full text retrieval(#3914) by @Huyueeer in #3956
• feat(platform): Schema version history timeline. by @rslanka in #4252
• fix(mae-comsumer): wrong aspect name in usage event transformer by @zhoxie-cisco in #4249
• feat(ml): Add searchable annotation for features in feature table by @dexter-mh-lee in #4216
• fix(ingest): fix telemetry profile emission by @kevinhu in #4253
• bug(logo): add platform to chart relationship query by @RyanHolstien in #4255
• chore(docs): cleanup location of guide, gitignore generated from git by @anshbansal in #4256
• feat(ingest): Spark-free data lake ingestion by @kevinhu in #4131
• Openapi new auth by @vlavorini in #4086
• feat(docs) add fine-grained lineage docs and examples by @ksrinath in #4260
• feat(ingest): add option to copy URN, fix graphql docs by @anshbansal in #4209
• chore(managed ingestion): add variables for default val, update vals by @anshbansal in #4186
• docs(ui): Adding guide for adding users to DataHub. by @jjoyce0510 in #4262
• fix(docs): doc build failing by @anshbansal in #4267
• fix(doc) fix spelling mistake in dataset doc by @ksrinath in #4264
• feat(ingest): add lineage_client_project_id field to the BigQuery config by @vcs9 in #4138
• feat(graphql): Adding resolved users and groups to policies by @jjoyce0510 in #4272
• bug(schema_version_history): fix semantic version ordering by @RyanHolstien in #4271
• fix(recs ui): fixing tag color in recommendations by @jjoyce0510 in #4274
• fix(ingestion): Fix snowflake lineage + logging & reporting improvements. by @rslanka in #4276
• fix(docs): Error in running airflow locally by @buggythepirate in #4259
• feat: powerbi source plugin by @mohdsiddique in #4201
• test(timeline): fix smoke test by @RyanHolstien in #4285
• fix(ingest) - always display CLI version by @aditya-radhakrishnan in #4282
• feat(lineage) Bigquery: Supporting v2 audit metadata on Bigquery by @treff7es in #4233
• fix(recipe-parsing): fix recipe config parsing for $ by @MugdhaHardikar-GSLab in #4258
• docs: add details to update users when using helm by @anshbansal in #4268
• feat(model): adds PRE in the FabricType enum by @sgomezvillamor in #4226
• fix(ingestion): Fix snowflake view upstream lineages to eliminate false edges. by @rslanka in #4284
• fix(docs): fix frontend docs to replace port 9001 -> 9002 by @gabe-lyons in #4280
• fix(docker): use exec form to start container main process by @gmcoringa in #4245
• fix(ingestion): revert positional arg change by @anshbansal in #4266
• feat(profiling) - Bigquery: Ability to disable partition profiling by @treff7es in #4228
• fix(ingest): clarify s3/s3a requirements and platform defaults by @kevinhu in #4263
• Remove <> from add-users.md doc by @jjoyce0510 in #4293
• fix(ingestion): Fix bigquery stateful ingestion checkpoint reconstruction. by @rslanka in #4295
• fix(telemetry): telemetry fail should not cause the CLI to fail by @anshbansal in #4302
• fix(search): Update urn tokenizer to tokenize on periods and slashes by @dexter-mh-lee in #4085
• Fixed the UI issue: Height issue of editor and the spacing issue between logo and description by @Ankit-Keshari-Vituity in #4300
• fix(vulnerabilities): Fix new vulnerabilities by upgrading libraries by @dexter-mh-lee in #4297
• docs(readmes): Update module READMEs to reflect the current state of the world by @jjoyce0510 in #4294
• Resign the policy tab by @Ankit-Keshari-Vituity in #4232
• refactor(extractor): Move extractors to entity-registry by @dexter-mh-lee in #4307
• refactor(ui) Minor policies styling improvements. by @jjoyce0510 in #4309
• feat(ui): Introducing New group profile by @jjoyce0510 in #4308
• refactor(ui): Simplify process of adding user.props (w/ docs) by @jjoyce0510 in #4296
• feat (ingest) Kafka-connect: Adding Auth to Kafka Connect API by @arunvasudevan in #4298
• fix(doc): Add warning on using AWS glue schema registry by @dexter-mh-lee in #4306
• fix(ingestion) Removing python restriction by @treff7es in #4312
• fix(ingest) bigquery: Remove unneeded warning by @treff7es in #4317
• doc: improve doc on adding source by @anshbansal in #4316
• fix: revert changes to OpenApi casing by @anshbansal in #4291
• feat(assertions): Adding Assertions Entity & Great Expectations BETA by @jjoyce0510 in #4305
• feat(tableau): emit workbook as container entity in tableau source, some minor fixes in tableau source by @mayurinehate in #4261
• fix(ui) Misc UI fixes & styling improvements. by @jjoyce0510 in #4311
• fix(tags) - map tags to globalTags for entities by @aditya-radhakrishnan in #4310
• fix(quickstart): Pin actions pod + add volume mount for datahub-frontend by @jjoyce0510 in #4318
• fix(ui): Correct display name for users in UI by @jjoyce0510 in #4323
• feat(Impact Analysis): Support impact analysis to check all downstreams of given entity by @dexter-mh-lee in #4322

New Contributors

• @zhaofengnian18 made their first contribution in #4221
• @bskim45 made their first contribution in #4243
• @eddyv made their first contribution in #4116
• @Huyueeer made their first contribution in #3956
• @vcs9 made their first contribution in #4138
• @mohdsiddique made their first contribution in #4201
• @gmcoringa made their first contribution in #4245

Full Changelog: v0.8.27...v0.8.28rc1

Release Candidate v0.8.28

Release Candidate for Version 0.8.28.

What's Changed

• doc(adoption): adds Adevinta as DataHub adopter by @sgomezvillamor in #4227
• fix(policies): Remove multiple privileges for GENERATE_PERSONAL_ACCESS_TOKEN by @jjoyce0510 in #4239
• Added Drawer to show the tag profile data by @Ankit-Keshari-Vituity in #4132
• fix(ui) Misc styling fixes (truncated filter values, updating tag color on clickaway) by @jjoyce0510 in #4246
• feat(ingest): switch telemetry endpoint to Mixpanel by @kevinhu in #4238
• docs: Add Udemy & Adevinta logos by @maggiehays in #4247
• fix(ingest) kafka-connect: Pass the env variable as part of making dataset by @arunvasudevan in #4244
• feat(lineage): Column level lineage model by @rslanka in #4248
• feat(docker): add multiplatform docker support for arm64 (m1) by @zhaofengnian18 in #4221
• fix(ingest): fix tableau sheets external url ingestion by @maaaikoool in #4231
• fix(docs): fix broken link by @anshbansal in #4242
• fix(docs): fix reference to the credential step in okta guide by @bskim45 in #4243
• feat(ingest): add ability to provide lineage described from within a file by @eddyv in #4116
• Add support for url_prefix in elastisearch source by @pppsunil in #4214
• feat(search): supporting chinese glossaryterm full text retrieval(#3914) by @Huyueeer in #3956
• feat(platform): Schema version history timeline. by @rslanka in #4252
• fix(mae-comsumer): wrong aspect name in usage event transformer by @zhoxie-cisco in #4249
• feat(ml): Add searchable annotation for features in feature table by @dexter-mh-lee in #4216
• fix(ingest): fix telemetry profile emission by @kevinhu in #4253
• bug(logo): add platform to chart relationship query by @RyanHolstien in #4255
• chore(docs): cleanup location of guide, gitignore generated from git by @anshbansal in #4256
• feat(ingest): Spark-free data lake ingestion by @kevinhu in #4131
• Openapi new auth by @vlavorini in #4086
• feat(docs) add fine-grained lineage docs and examples by @ksrinath in #4260
• feat(ingest): add option to copy URN, fix graphql docs by @anshbansal in #4209
• chore(managed ingestion): add variables for default val, update vals by @anshbansal in #4186
• docs(ui): Adding guide for adding users to DataHub. by @jjoyce0510 in #4262
• fix(docs): doc build failing by @anshbansal in #4267
• fix(doc) fix spelling mistake in dataset doc by @ksrinath in #4264
• feat(ingest): add lineage_client_project_id field to the BigQuery config by @vcs9 in #4138
• feat(graphql): Adding resolved users and groups to policies by @jjoyce0510 in #4272
• bug(schema_version_history): fix semantic version ordering by @RyanHolstien in #4271
• fix(recs ui): fixing tag color in recommendations by @jjoyce0510 in #4274
• fix(ingestion): Fix snowflake lineage + logging & reporting improvements. by @rslanka in #4276
• fix(docs): Error in running airflow locally by @buggythepirate in #4259
• feat: powerbi source plugin by @mohdsiddique in #4201
• test(timeline): fix smoke test by @RyanHolstien in #4285
• fix(ingest) - always display CLI version by @aditya-radhakrishnan in #4282
• feat(lineage) Bigquery: Supporting v2 audit metadata on Bigquery by @treff7es in #4233
• fix(recipe-parsing): fix recipe config parsing for $ by @MugdhaHardikar-GSLab in #4258
• docs: add details to update users when using helm by @anshbansal in #4268
• feat(model): adds PRE in the FabricType enum by @sgomezvillamor in #4226
• fix(ingestion): Fix snowflake view upstream lineages to eliminate false edges. by @rslanka in #4284
• fix(docs): fix frontend docs to replace port 9001 -> 9002 by @gabe-lyons in #4280
• fix(docker): use exec form to start container main process by @gmcoringa in #4245
• fix(ingestion): revert positional arg change by @anshbansal in #4266
• feat(profiling) - Bigquery: Ability to disable partition profiling by @treff7es in #4228
• fix(ingest): clarify s3/s3a requirements and platform defaults by @kevinhu in #4263
• Remove <> from add-users.md doc by @jjoyce0510 in #4293
• fix(ingestion): Fix bigquery stateful ingestion checkpoint reconstruction. by @rslanka in #4295
• fix(telemetry): telemetry fail should not cause the CLI to fail by @anshbansal in #4302
• fix(search): Update urn tokenizer to tokenize on periods and slashes by @dexter-mh-lee in #4085
• Fixed the UI issue: Height issue of editor and the spacing issue between logo and description by @Ankit-Keshari-Vituity in #4300
• fix(vulnerabilities): Fix new vulnerabilities by upgrading libraries by @dexter-mh-lee in #4297
• docs(readmes): Update module READMEs to reflect the current state of the world by @jjoyce0510 in #4294
• Resign the policy tab by @Ankit-Keshari-Vituity in #4232
• refactor(extractor): Move extractors to entity-registry by @dexter-mh-lee in #4307
• refactor(ui) Minor policies styling improvements. by @jjoyce0510 in #4309
• feat(ui): Introducing New group profile by @jjoyce0510 in #4308
• refactor(ui): Simplify process of adding user.props (w/ docs) by @jjoyce0510 in #4296
• feat (ingest) Kafka-connect: Adding Auth to Kafka Connect API by @arunvasudevan in #4298
• fix(doc): Add warning on using AWS glue schema registry by @dexter-mh-lee in #4306
• fix(ingestion) Removing python restriction by @treff7es in #4312
• fix(ingest) bigquery: Remove unneeded warning by @treff7es in #4317
• doc: improve doc on adding source by @anshbansal in #4316
• fix: revert changes to OpenApi casing by @anshbansal in #4291
• feat(assertions): Adding Assertions Entity & Great Expectations BETA by @jjoyce0510 in #4305
• feat(tableau): emit workbook as container entity in tableau source, some minor fixes in tableau source by @mayurinehate in #4261
• fix(ui) Misc UI fixes & styling improvements. by @jjoyce0510 in #4311
• fix(tags) - map tags to globalTags for entities by @aditya-radhakrishnan in #4310
• fix(quickstart): Pin actions pod + add volume mount for datahub-frontend by @jjoyce0510 in #4318
• fix(ui): Correct display name for users in UI by @jjoyce0510 in #4323
• feat(Impact Analysis): Support impact analysis to check all downstreams of given entity by @dexter-mh-lee in #4322

New Contributors

• @zhaofengnian18 made their first contribution in #4221
• @bskim45 made their first contribution in #4243
• @eddyv made their first contribution in #4116
• @Huyueeer made their first contribution in #3956
• @vcs9 made their first contribution in #4138
• @mohdsiddique made their first contribution in #4201
• @gmcoringa made their first contribution in #4245

Full Changelog: v0.8.27...RC-v0.8.28

DataHub v0.8.27

Release Highlights

Notable UI-Based Features

• The User Page has a new look! You can now quickly filter & search for entities owned by a User, update/edit the user profile, and see details of which Groups the User belongs to. See it in action here.

• Search for Entities by Owner - Easily filter search results by User/Group Owner

• Edit existing Glossary Terms - you can now edit/update Glossary Term descriptions via the UI. Future work will allow creating Terms from the UI as well - stay tuned!

• Improved Metadata Analytics - keep tabs on your DataHub entities across Domains, Platforms, Glossary Terms, Environments, & more. Check out the new & improved Analytics tab!

Notable Metadata Model & Ingestion-Based Features

• ClickHouse integration is now incubating! This is a 100% Community-led integration - huge shoutout to @ne1r0n & @havramar for pushing initial code & moving this work through!

• Kafka Stateful Ingestion - shoutout to @claudio-benfatto for building this out!

• Extract Airflow Task Description - big thanks to @guidoturtu for the contrib!

• BigQuery: profile latest Partition/Shard - We know that Data Profiling can be computationally expensive for partitioned/sharded BQ instances. We now support profiling only the latest partition/shard to minimize processing load.

Notable Docs Updates

• NEW! Tips for Searching within DataHub - Ever wondered how to make the most of Searching within DataHub? Check out this doc put together by @xiphl

• Improvements to Metadata Model Docs - This is a huge win for the Community - we’re taking a big step toward providing auto-generated & curated docs related to the Metadata Model - take a look here.

What's Changed

• feat(deprecation): Entity Deprecation Backend by @jjoyce0510 in #4073
• Fixed auto complete pr coments by @Ankit-Keshari-Vituity in #4072
• fix(ingestion): enforce correct behaviour for commit policy by @claudio-benfatto in #4092
• fix(aggregate): Fix NPE in aggregate api by @dexter-mh-lee in #4095
• add Haibo corp by @wangqinghuan in #4082
• fix(ingestion): Add psutil dependency required for stateful ingestion reporting. by @rslanka in #4099
• docs(kafka): add example for using domains, change for clarity by @anshbansal in #4100
• feat(ui): Add display name & title to editable corp user properties. by @jjoyce0510 in #4097
• fix(ingestion): Enhance BigQuery source logging. by @rslanka in #4101
• fix(glossary terms): fix add glossary term flow by @gabe-lyons in #4106
• (docs) Add Zynga & Tableau logos by @maggiehays in #4109
• fix(ingestion): Add sql lineage to redshift-usage plugin by @dexter-mh-lee in #4103
• feat(ui): Add svg datahub satellite loading logo by @eburairu in #4067
• fix(ingestion): resolve oracle issue with large view definitions by @hsheth2 in #4027
• fix(ingest): ignore Postgres information_schema tables by default by @kevinhu in #4069
• fix(ingest) - close event loops in Okta source and add additional debug logging by @aditya-radhakrishnan in #4077
• chore(ingest): remove unused groupby_unsorted utility by @hsheth2 in #4011
• fix(docs): fixing metadata model doc generation script and updating png by @swaroopjagadish in #4120
• fix(ci): fix formatting in doc generation action yaml by @swaroopjagadish in #4121
• fix(ci): fix formatting for action yaml by @swaroopjagadish in #4122
• feat(Tags/Terms): Backend support for tag & term mutations by @jjoyce0510 in #4096
• docs(backup): add doc for taking backup by @anshbansal in #3917
• fix(docs): make intro to metadata ingestion easier for beginners by @anshbansal in #4039
• fix(ingest) Athena: db filter was not applied by @treff7es in #4127
• fix(ui) - move book logo to right of glossary term by @aditya-radhakrishnan in #4125
• fix(docs) Fix doc on modelDocUpload by @daha in #4112
• fix(cypress): force clicks on tag mutation test by @gabe-lyons in #4102
• feat(ingest) Athena: Getting table properties for Athena datasets by @treff7es in #4123
• fix(logging): Fix Restli Logging Filter to print full stack trace on error by @dexter-mh-lee in #4136
• docs : markdown fixes for db retention table by @satyamkrishna in #4133
• docs : markdown fixes for db retention table by @satyamkrishna in #4148
• feat(ingestion): Kafka stateful ingestion by @claudio-benfatto in #4028
• fix(docs): update graphql docs to reference new graphql file by @gabe-lyons in #4139
• Feature/oss/update to v2 endpoints by @RyanHolstien in #4128
• fix(cli): add timeout for telemetry calls by @anshbansal in #4135
• chore(cli): update default cli version pinned in the UI based ingestion by @anshbansal in #4150
• fix(docs): fix example of delta lake by @anshbansal in #4149
• fix(ui): Fix cutoff profiling axis labels by @jjoyce0510 in #4154
• feat(ingest): Glue - Support for domains and containers by @treff7es in #4110
• feat(ui): Host platform images on datahub-web-react by @ngamanda in #4118
• bug(seedData): adds a key to the root user seed data and fixes corner case check for missing key aspects by @RyanHolstien in #4162
• UI Fix: Modal close on Enter press, autofocus on modal, added split panel, alignment of button by @Ankit-Keshari-Vituity in #4155
• feat(ui): Edit glossary term descriptions via UI by @jjoyce0510 in #4156
• Update querying-entities.md -> Documentation Error by @buggythepirate in #4157
• refactor(metadata-io/test): common ElasticsearchContainer and ability to override from environment. by @stephenp-gr in #4152
• feat(ingestion): Add support for snowflake view lineage. by @rslanka in #4163
• Update the doc to including options to include Views by @cuong-pham in #4164
• fix(ingest): Use lower-case dataset names in the dataset urns for all SQL-styled datasets. by @rslanka in #4140
• chore(ingestion): upgrade mypy by @hsheth2 in #4141
• ci(ingestion): fix airflow 1 deps for tox by @hsheth2 in #4083
• fix(ingest) Glue: Removing sqlalchemy dependency from glue by @treff7es in #4168
• fix(ingest) Athena: Generating propert containers for Athena by @treff7es in #4167
• Feature/users and groups UI updated as per new design by @ShubhamThakre in #4134
• chore(docs): various cleanup for docs-website by @hsheth2 in #4143
• bugfix(logging): reduce log noise from authentication chain by @RyanHolstien in #4173
• bug(glossaryTermLabels): fix glossary term labels missing and add cypress test by @RyanHolstien in #4171
• fixes(ui): Misc UI fixes + Adding Owners to Search Filters by @jjoyce0510 in #4175
• BugFixes/user-and-groups-minor-ui-fixes by @ShubhamThakre in #4181
• feat(groups): Adding editable group properties in the backend by @jjoyce0510 in #4166
• fix(python build): Pinning markupsafe by @treff7es in #4188
• feat(analytics): Improve analytics page by adding more charts regarding metadata ingested by @dexter-mh-lee in #4176
• docs(model): auto-generated docs and hand-written docs for the metada… by @swaroopjagadish in #4189
• minor fixes(ui): Small UI display fixes by @jjoyce0510 in #4190
• fix(ui): Return empty search response on invalid characters in search by @jjoyce0510 in #4193
• refactor(spark-line...

DataHub v0.8.26

This is a Bugfix release meant to address the issue with adding Glossary Terms to Dataset fields present in version 0.8.25.

Release Highlights

• Fixing bug where Glossary Terms cannot be added to Dataset fields in previous release version.

DataHub v0.8.25

Known Issues

• Adding Glossary Terms to schema fields does not work with this version due to a bug. Upgrade to v0.8.26 for the fix.

Release Highlights

Buckle up, folks! v0.8.25 brings some very exciting (and highly-requested!) updates.

Notable UI-Based Features

• UI-based Ingestion - as demoed in December Town Hall, we now support creating, configuring, scheduling, & executing batch metadata ingestion using the DataHub user interface. This makes getting metadata into DataHub easier by minimizing the overhead required to operate custom integration pipelines.
• Data Domains - DataHub now supports grouping data assets into logical collections called Domains. Domains are curated, top-level folders or categories where related assets can be explicitly grouped. Read the guide here!
• Data Containers are now supported! This is the physical grouping of entities, ex. a Schema is a container of 1 or more Datasets; a Dashboard is a container of 1 or more Charts.

Notable Metadata Model & Ingestion-Based Features

• Data Quality test results are now supported in the DataHub metadata model. This is the first milestone toward surfacing Dataset & Column-level Data Quality results in the UI (read full scope of work here). Future releases will include a Great Expectations integration & UI support - we’re on track to complete this in Q1 as planned.
• Avro files are now supported in the Data Lake File ingestion source
• Ingest metadata from multiple instances of the same platform type. This has been a very common use case within the Community - you can now differentiate multiple instances of the same platform type! If you already have pre-existing entries, use the datahub migrate command to migrate them over to platform instances.
• Ignore users from Top Users calculation
  • feat(ingestion): Adding ability to ignore users from top users calculation by @treff7es in #3735
• BigQuery - Data Profiling on only the latest partition/shard
  • feat(ingestion) bigquery: Profiling only the latest partition/shard on bigquery by @treff7es in #3930

• (feat)(Business Glossary) add tabular schema and new UI for business glossary by @saxo-lalrishav in #3813

Notable Fixes

• Fix to support View in Looker * feat(looker): Adding optional Looker external url base url config by @jjoyce0510 in #3985
• fix(graphql): support group display name in ownership by @thomasplarsson in #3979
• fix(profiling): Enabling profiling for low cardinality number columns by @treff7es in #3990
• fix(ingestion): match default username for Azure OIDC and Azure ingestion source by @iasoon in #3926

DataHub Usage Guides

• docs(domains): Adding a User Guide for Domains by @jjoyce0510 in #4038
• docs(ingest): Adding UI ingestion guide by @jjoyce0510 in #4048

What's Changed

• fix(vulnerability): Upgrade gms base image by @dexter-mh-lee in #3962
• logging(frontend): Improve OIDC debug logs by @jjoyce0510 in #3967
• docs(delete): add curl request example to delete entity by @anshbansal in #3928
• fix(ingestion): match default username for Azure OIDC and Azure ingestion source by @iasoon in #3926
• Feature/dynamic platform icons by @RyanHolstien in #3968
• refactor(ingestion): remove duplicate aspect type by @hsheth2 in #3972
• fix(example): fix typo by @anshbansal in #3907
• fix(ingestion): Restrict python to <=3.9.9 by @treff7es in #3961
• feat(build): remove requirement for git directory for builds by @swaroopjagadish in #3977
• fix(ingestion): tighten conditions for restli json transformation by @hsheth2 in #3973
• fix(ingestion): don't dump variables for config errors by @hsheth2 in #3974
• Bugfix/increase socket timeout by @RyanHolstien in #3982
• feat(ingest): support for Avro data lake files by @kevinhu in #3913
• fix(build): exclude old log4j core by @RickardCardell in #3966
• fix(quickstart): Pin Quickstart version to v0.8.23. by @jjoyce0510 in #3983
• feat(looker): Adding optional Looker external url base url config by @jjoyce0510 in #3985
• fix(graphql): support group display name in ownership by @thomasplarsson in #3979
• fix(quickstart): Assign correct mysql-setup container for M1 and remove "head" default version. by @jjoyce0510 in #3987
• feat(embedded search results): support custom endpoints in embedded search result by @gabe-lyons in #3986
• fix(docker): datahub-gms - build in native, copy to target by @swaroopjagadish in #3992
• fix(ci): moving defaults back to head now that docker builds are green by @swaroopjagadish in #3993
• feat(ui): UI-based ingestion (as featured in Dec Townhall) by @jjoyce0510 in #3975
• quickstart: Adding UI ingestion to quickstart YAML by @jjoyce0510 in #3994
• feat(domains): Adding backend for Asset Domains (p1) by @jjoyce0510 in #3952
• Bug: a bug fix to bigquery_to_datahub.yml file by @dipeshmaurya in #3988
• fix(ingest): check if feature data type is present by @maaaikoool in #3932
• feat(platform-instance): a simple client-only change to support platf… by @swaroopjagadish in #3996
• docs(metadata-model): Adding to Metadata model docs by @jjoyce0510 in #3998
• Add Stash Logo & new Source Icons by @maggiehays in #4002
• feat(domains): UI for Asset Domains (p2) by @jjoyce0510 in #3995
• docs: add missing back tick for metadata-ingestion/README.md by @nickwu241 in #4003
• Bugfix/add missing classes by @RyanHolstien in #4000
• fix(superset): fix connection for redshift by @anshbansal in #3944
• fix(setup): fix setup for M1 by @anshbansal in #3958
• docs:add Optum logo by @maggiehays in #4005
• Refining Metadata Model docs further by @jjoyce0510 in #4001
• fix(docker): Alpine based multiplatform docker build for kafka-setup by @treff7es in #3991
• Bugfix/graph concurrency issue by @RyanHolstien in #4007
• feat(ingest): Add additional snowflake auth by @MikeSchlosser16 in #4009
• fix(ci): Reverting unnecessary domain test changes by @jjoyce0510 in #4013
• fix(metrics): Add metrics for mcl hooks by @dexter-mh-lee in #4008
• feat(platform) - Update FabricType enum to represent more fabrics by @aditya-radhakrishnan in #3997
• feat(ingest): emit flags and stats for profiling telemetry by @kevinhu in #3969
• fix(formatting): fix linting lib version requirement by @anshbansal in #3939
• fix(docs): fix business glossary docs by @anshbansal in #3916
• fix(profiling): Enabling profiling for low cardinality number columns by @treff7es in #3990
• fix(docs): update gms link by @lhvubtqn in #3927
• fix(ingest): lint fix a few files by @swaroopjagadish in #4016
• fix(ingest): adding platform instance urn to data platform instance aspects by @swaroopjagadish in #4015
• feat(ingest): use trino python client for sqlalchemy, supports python… by @mayurinehate in #3888
• fix(spark-lineage): select mock server port dynamically for unit test by @MugdhaHardikar-GSLab in #4018
• (feat)(Business Glossary) add tabular schema and new UI for business glossary by @saxo-lalrishav in #3813
• Test/add concurrency issue smoke test by @RyanHolstien in #4014
• feat(glossary-terms): Index glossary term custom properties by @jjoyce0510 in #3960
• feat(ingestion): Adding ability to ignore users from top users...