v0.8.34

Release Highlights

Developer Experience

• DataHub Actions Framework is LIVE! The Actions Framework makes responding to real-time changes in your Metadata Graph easy, enabling you to seamlessly integrate DataHub into a broader events-based architecture. Check out the repo here
• This release also introduces OpenAPI endpoints to post, get, and delete entities. Check out the usage guide here
• Metadata Ingestion Source docs have a new look! We now have code-generated documentation to apply consistency in format and contents

User Experience

• New! The Dataset Schema page now supports a “Blame View” to quickly understand how a field has evolved over semantic schema versions. You can find more info about how we compute versions here​​.

Ingestion Improvements

• New! Now incubating the Apache Pulsar source
• Update to Feast connector to support v0.18
• Ongoing improvements to Snowflake external table support
• Improvements to handling BigQuery audit log SQL queries
• Miscellaneous Tableau fixes for lineage, browse path, non-embedded datasets

What's Changed

• fix(cypress) - enable retries for failed tests to minimize flaking by @aditya-radhakrishnan in #4680
• Deprecate an entity by @Ankit-Keshari-Vituity in #4633
• fix(timeline): enhance schema field name change and removal support by @RyanHolstien in #4603
• fix(cli): rest emitter should override config and env variables by @anshbansal in #4622
• fix(docs): elasticsearch secret reference by @felixb in #4314
• fix(mcl-processor): Remove unnecessary log.info by @dexter-mh-lee in #4686
• fix(datahub-client): avoid parallel execution of metadat-io:test by @MugdhaHardikar-GSLab in #4685
• docs(metadata-models-custom): add example script to show producing cu… by @shirshanka in #4681
• fix(gms): Ensure Ordering by version when fetching next version by @arunvasudevan in #4696
• fix(docker): Fix issue #4683 by @jjoyce0510 in #4697
• feat(vulnerability): Upgrade spring libraries to latest version by @dexter-mh-lee in #4698
• refactor(gms): EbeanAspectDao - make the orderBy clause explicitly ascending in getNextVersions by @jjoyce0510 in #4699
• feat(gms): Entity change events v1 (Platform Event) by @jjoyce0510 in #4687
• Redesign the login page by @Ankit-Keshari-Vituity in #4684
• fix(snowflake): remove extra lineage edges in reports, change badly named config variable by @anshbansal in #4595
• fix(bigquery): error due to not handling data properly by @anshbansal in #4702
• fix(looker): Fix for Pydantic validation error for Looker TransportOptions on python 3.8 by @treff7es in #4705
• fix(ingest) bigquery: Moving bigquery temporary credential deletion to atexit by @treff7es in #4701
• fix(lineage): Fix lineage entity drawer height UI bug by @chriscollins3456 in #4707
• feat(ingest) - update identity sources to add flags for masking sensitive work units by @aditya-radhakrishnan in #4711
• fix(snowflake): deprecate config, update examples by @anshbansal in #4644
• fix(glue): delete CatalogId parameter from get_jobs api call by @BoyuanZhangDE in #4646
• fix(ui): Show deprecate button only for specific entity pages. by @jjoyce0510 in #4712
• feat(ml): show custom properties for MLFeatureTable in UI by @maaaikoool in #4706
• fix(glue): fix error for custom connector if ignore_unsupported_conne… by @mayurinehate in #4667
• feat(ingest): add decimal128 custom type for mysql by @kevinhu in #4624
• fix(policy): Use search to fetch all policies by @dexter-mh-lee in #4713
• fix(transformers): add snapshot aspects from dataset into base_transf… by @shirshanka in #4719
• Revert "fix(policy): Use search to fetch all policies" by @dexter-mh-lee in #4725
• minor fix(metadata-ingestion): Add new schemas to python codegen by @jjoyce0510 in #4726
• fix(ui): Display warning in UI when metadata service auth is disabled. by @jjoyce0510 in #4728
• fix(timelineCli): fix naming for timeline cli by @RyanHolstien in #4729
• fix(entity header): Fixes two issues in the EntityHeader - update UI and remove link by @chriscollins3456 in #4720
• Revert "fix(timelineCli): fix naming for timeline cli (#4729)" by @jjoyce0510 in #4731
• feat(cli): suppress stacktrace printing on configuration errors by @shirshanka in #4718
• fix(cli): align default sink env variables across ingest and other cl… by @shirshanka in #4739
• feat(ingest) dbt: Dbt query tag mapping and match template by @treff7es in #4744
• fix(cli): telemetry - make config file processing more robust by @shirshanka in #4738
• feat(react theming): stop homepage flicker for env-var based logos by @gabe-lyons in #4730
• feat(Cassandra): add Cassandra implementation of EntityService by @xdl in #3286
• fix(policies): Re-revert the policies fix + ingest documents directly to search by @dexter-mh-lee in #4733
• feat(cli): Eagerly load datahub actions CLI commands by @jjoyce0510 in #4748
• fix(ingest) bigquery: Fix BigQuery Datetime/Timestamp type column partition table profile bug by @sebkim in #4658
• docs: add missing PR numbers by @anshbansal in #4742
• fix(azure_ad): silently discard other Azure AD object types (#4693) by @cccs-eric in #4704
• fix(datahub-frontend): OIDC discovery URL will not have NONE as auth_methods_supported by @chen4119 in #4710
• fix(docs): fix links by @daha in #4703
• feat(ingest): add Feast repository source by @danilopeixoto in #4094
• feat(soft deletes): rephrasing soft delete banner by @gabe-lyons in #4753
• feat(ebeans): Add metrics to track connection pool by @dexter-mh-lee in #4755
• fix(AWS) When using aws_profile, grab temporary credentials from the session. by @Jiafi in #4751
• feat(metadata-ingestion): Custom endpoint url and proxies in S3. by @pawel3275 in #4708
• fix(tableau): miscellaneous tableau fixes for lineage, browse path, non-embedded datasets by @mayurinehate in #4724
• doc: add warning for JDK by @anshbansal in #4761
• fix(ui): fix expandedName for dataset by @mayurinehate in #4762
• fix(ui): Users and Groups UI bug fixes by @ShubhamThakre in #4746
• fix(azure_ad): make redirect and graph_url optional parameters and update docs by @aditya-radhakrishnan in #4754
• docs(glue): clarify that table regex patterns should be fully-qualified by @aditya-radhakrishnan in #4747
• fix(ml models): fix features tab by @gabe-lyons in #4769
• fix(lint): lib upgrade caused by @anshbansal in #4773
• fix(lineage) Filter dataset -> dataset lineage edges if data is transformed by @chriscollins3456 in #4732
• fix(build): Fix breaking changes from GE 0.15.3 that are affecting our Python3.6 smoke_tests by @rslanka in #4779
• fix(ingestion): Fixing how we eagerly import DataHub actions by @jjoyce0510 in #4784
• fix(ingest): fwk - datahub_api should be initialized by datahub-rest … by @shirshanka in #4786
• feat(ingestion) Ingest Tags from s3 bucket on an AWS Glue job and S3 Data Lake Ingest Job by @Jiafi in #4689
• fix(snowflake): improve debug log for external tables by @anshbansal in #4772
• feat(snowflake): add option to disable checking role grants by @anshbansal in #4760
• fix(m1): tweak m1 preflight by @anshbansal in #4771
• feat(ingestion): add Pulsar source by @vanmeete in #4721
• fix(mae consumer): Fixes delete logic in MAE consumer by @pedro93 in #4790
• feat(analytics): display glossary term percentage coverage by @aditya-radhakrishnan in #4782
• refactor(gms): Removing unused source field by @jjoyce0510 in #4781
• feat(versionedDataset): adds a versionStamp to timeline response & adds versionStamp param to dataset graphql by @RyanHolstien in #4727
• fix(s3): improved handling for corner cases by @mayurinehate in #4774
• fix(ingest): databricks - hive ingestion should not fail on table com… by @shirshanka in #4787
• fix(ui ingest): Unschedule all sources on ingestion source refresh, fix delete not being enforced by @jjoyce0510 in #4792
• feat(tracking) Configure whether mixpanel is enabled with env variable by @chriscollins3456 in #4768
• feat(ingest): docs - overhaul source connector docs to make it code driven by @shirshanka in #4798
• fix(docs): Fixing outdated control-center doc on policies.md by @jjoyce0510 in #4799
• fix(ui): update default preview component with new ui design by @ShubhamThakre in #4783
• feat(operation): display the reported time for last updated in the UI by @aditya-radhakrishnan in #4800
• feat(blame) - add schema history blame UI by @aditya-radhakrishnan in #4793
• fix(ingestion): Fix schema field type for avro logical types by @rslanka in #4801
• Create sample_pii_glossary.yml by @mitchelllovessoftware123 in #4795
• fix(ingestion): Fix presto_on_hive tests. by @rslanka in #4802
• fix(bigquery): improve handling of extracted audit log sql queries by @vgaidass in #4735
• fix(snowflake): get external tables when there is default namespace by @anshbansal in #4803
• fix(snowflake): passing connect args should not cause failures by @anshbansal in #4764
• fix(scrolling) Fixes scrolling and weird heights for embeddedListSearch across entities by @chriscollins3456 in #4805
• fix(ui): update default preview card description text by @ShubhamThakre in #4796
• fix(ui): preview card UI design update by @ShubhamThakre in #4808
• fix(blame): make view blame prior to button work properly by @aditya-radhakrishnan in #4810
• fix(docgen): fix failure count incrementing during doc generation by @shirshanka in #4806
• fix(search) Fixes a UI issue so results and filters are always separated by @chriscollins3456 in #4811
• feat(openapi): initial post,get, and delete endpoints for entities by @RyanHolstien in #4775
• feat(protobuf) Adding deprecation support for datasets and fields by @leifker in #4634

New Contributors

• @felixb made their first contribution in #4314
• @chriscollins3456 made their first contribution in #4707
• @sebkim made their first contribution in #4658
• @chen4119 made their first contribution in #4710
• @Jiafi made their first contribution in #4751
• @pawel3275 made their first contribution in #4708
• @vanmeete made their first contribution in #4721
• @mitchelllovessoftware123 made their first contribution in #4795
• @vgaidass made their first contribution in #4735

Full Changelog: v0.8.33...v0.8.34