v0.8.39

Release Highlights

Known Issues

When using stand-alone MAE consumers (mae-consumer-job) this release will not work; this has been resolved in v0.8.40.

User Experience

• NEW: support for surfacing outcomes of dbt Tests in dataset entity pages (see it in action here)
• NEW: Improved navigation of dbt resources: dbt models and their associated warehouse tables are now merged into a unified entity (see it here). This will automatically be enabled for all newly ingested entities. To view this for entities you have already ingested, you will need to run a restore indices job.
• Improvement to Impact Analysis: When looking at the Lineage tab, you can now easily toggle between “Upstream” and “Downstream” entities (try it out here)

Developer Experience

• NEW: Java Kafka Emitter – Use this when you want to decouple your metadata producer from the uptime of your datahub metadata server by utilizing Kafka as a highly available message bus

Metadata Ingestion

• NEW: Make bulk edits to your metadata via CSV (read more)
• Snowflake ingestion improvements: configure profiling to run only if they have been updated within the prior N days
• Managed ingestion update: removed need for sink block

What's Changed

• fix(ui-ingestion): update looker ingestion warning banner by @aditya-radhakrishnan in #5142
• chore: Bump Default UI Ingestion Version 0.8.38 by @jjoyce0510 in #5145
• feat(schema): support rendering schemas with . in field names by @gabe-lyons in #5141
• feat(dbt): Platform instances for target platform by @skrydal in #5129
• feat(ingest): snowflake profile tables only if they have been updates… by @mayurinehate in #5132
• fix(airflow): fixes DeprecationWarning with hook-class-names by @sayakmaity in #5143
• feat(frontend): Parse JWT access token claims by @chen4119 in #5138
• fix(tokens): Using keyword search filters for ListAccessTokensResolver by @jjoyce0510 in #5154
• feat(ui) Update the max text length of Terms/Term Groups by @chriscollins3456 in #5162
• docs(policies): add info about Manage User Credentials by @aditya-radhakrishnan in #5157
• fix(restore-indices): Do not fail on MAE row count diff by @dexter-mh-lee in #5165
• fix(Kafka-setup): Make sure it doesn't fail when the new envs are not set by @dexter-mh-lee in #5168
• chore(deps): Bump Nimbus Jose JWT dependency by @pedro93 in #5158
• fix(recs): Verify that an entity exists before recommending by @jjoyce0510 in #5163
• fix(business glossary): setting properties to be empty if the node has no properties aspect by @gabe-lyons in #5166
• refactor(ui): Misc improvements to Dataset Assertions UI by @jjoyce0510 in #5155
• chore(guava): force version of guava in client jars per #5134 by @RyanHolstien in #5153
• feat(boot): Make Glossary Term Upgrade Async by @jjoyce0510 in #5164
• fix(frontend): Add iam auth jar to frontend by @dexter-mh-lee in #5171
• docs(features): update & clean up Features page by @maggiehays in #5175
• fix(glue): fix glue profiling config option by @kangseonghyun in #5178
• feat(upgrade) Check version when determining to run RestoreGlossaryIndices step by @chriscollins3456 in #5182
• fix(jaas): fixed auth.jaas.enabled option parsing by @alexey-kravtsov in #5179
• feat(ingestion): bigquery - Option to send usage queries as well as Operational metadata by @treff7es in #5151
• feat(build): changes to decrease build time, cancel runs in case of multiple commits by @anshbansal in #5187
• refactor(docs): Update Metadata Events Docs by @jjoyce0510 in #5173
• fix(ingest): If there is no manager for a LDAP user (example: system account) by @bda618 in #5180
• bug(ingest): correct case of sys views for mssql description populati… by @BALyons in #5186
• refactor(configs): Simplify Kafka Topic name configurations + docs by @jjoyce0510 in #5198
• feat(ingest): dbt - adding support for dbt tests by @shirshanka in #5201
• fix(cli): correct handling of env variables by @anshbansal in #5203
• feat(ci): split integration tests to reduce run time by @anshbansal in #5205
• feat(datahub-client): add java kafka emitter by @MugdhaHardikar-GSLab in #5074
• feat(graphql): add metrics capturing for graphql latency by @RyanHolstien in #5200
• test(ingestion): bigquery-usage - Adding tests for bigquery usage filters by @treff7es in #5195
• fix(ui): load monaco-editor as a dependency and not from a third party CDN by @Masterchen09 in #5189
• feat(cli): Add token parameter for sample ingestion by @pedro93 in #5160
• feat(lineage) Update Lineage tab and Impact Analysis feature by @chriscollins3456 in #5121
• fix(ingest): add missing ownership types by @afghori in #5209
• feat(ingestion) ldap: make ldap attrs keys configurable by @atulsaurav in #4682
• Remove unnecessary space from application.yml of GMS by @mmmeeedddsss in #5216
• fix(upgrade): fix upgrade when s3 path has = by @RyanHolstien in #5220
• feat(docs) Add and update docs for the new Glossary experience by @chriscollins3456 in #5211
• feat(glossary) Add empty state for the Business Glossary home page by @chriscollins3456 in #5217
• feat(bootstrap): add bootstrap step to clear out unknown aspect rows from the database by @RyanHolstien in #5148
• feat(ingest): adds csv enricher ingestion source by @aditya-radhakrishnan in #5221
• fix(build): pin confluent kafka dependency by @anshbansal in #5224
• fix(ingest): databricks - ingest structs correctly through hive by @shirshanka in #5223
• feat(dbt): add sibling association logic to associate dbt elements with their target systems by @gabe-lyons in #5190
• feat(tableau): use pagination for all connection queries by @mayurinehate in #5204
• Handling 404 page not found by @Ankit-Keshari-Vituity in #5227
• refactor(UI): Refactor Dataset Health Status by @jjoyce0510 in #5222
• fix(dbt-test): Inconsistency in assertions by @Santhin in #5214
• feat(ingest): remove need for sink block in UI based ingestion by @anshbansal in #5208
• fix(ingest): bigquery - Grouping date named tables at bigquery by @treff7es in #5230
• Add check for 0 rows when profiling datasets from s3 by @Jiafi in #5219
• [bug fix]: disabled create buttons by @xiphl in #5234
• fix(ingest): bigquery - Handling gracefully sql parser error in bq lineage by @treff7es in #5238
• fix(ingest): do not dump password by @anshbansal in #5235
• feat(ingest): dbt - improving dbt_meta mapping by @shirshanka in #5237
• fix(siblings): Force merged urn to viewed entity urn by @jjoyce0510 in #5247
• refactor(docs): Move CLI docs to root level by @jjoyce0510 in #5246

New Contributors

• @skrydal made their first contribution in #5129
• @sayakmaity made their first contribution in #5143
• @kangseonghyun made their first contribution in #5178
• @alexey-kravtsov made their first contribution in #5179
• @bda618 made their first contribution in #5180
• @BALyons made their first contribution in #5186
• @afghori made their first contribution in #5209
• @Santhin made their first contribution in #5214

Full Changelog: v0.8.38...v0.8.39