1.5.14-release

Ignore Bot on Domain restriction (#19032)

(cherry picked from commit b01707e54df153208a65f168244331d74513e1d6)

1.6.1-release

1.6.1

• Fix tags listing for explore page

1.6.0

Backward Incompatible Changes

Ingestion Workflow Status

We are updating how we compute the success percentage. Previously, we took into account for partial success the results
of the Source (e.g., the tables we were able to properly retrieve from Snowflake, Redshift, etc.). This means that we had
an error threshold in there were if up to 90% of the tables were successfully ingested, we would still consider the
workflow as successful. However, any errors when sending the information to OpenMetadata would be considered as a failure.

Now, we're changing this behavior to consider the success rate of all the steps involved in the workflow. The UI will
then show more Partial Success statuses rather than Failed, properly reflecting the real state of the workflow.

Database Metadata & Lineage Workflow

With 1.6 Release we are moving the View Lineage & Stored Procedure Lineage computation from metadata workflow to lineage workflow.

This means that we are removing the overrideViewLineage property from the DatabaseServiceMetadataPipeline schema which will be moved to the DatabaseServiceQueryLineagePipeline schema.

Profiler & Auto Classification Workflow

We are creating a new Auto Classification workflow that will take care of managing the sample data and PII classification,
which was previously done by the Profiler workflow. This change will allow us to have a more modular and scalable system.

The Profiler workflow will now only focus on the profiling part of the data, while the Auto Classification will take care
of the rest.

This means that we are removing these properties from the DatabaseServiceProfilerPipeline schema:

• generateSampleData
• processPiiSensitive
• confidence
  which will be moved to the new DatabaseServiceAutoClassificationPipeline schema.

What you will need to do:

• If you are using the EXTERNAL ingestion for the profiler (YAML configuration), you will need to update your configuration,
  removing these properties as well.
• If you still want to use the Auto PII Classification and sampling features, you can create the new workflow
  from the UI.

RBAC Policy Updates for EditTags

We have given more granularity to the EditTags policy. Previously, it was a single policy that allowed the user to manage
any kind of tagging to the assets, including adding tags, glossary terms, and Tiers.

Now, we have split this policy to give further control on which kind of tagging the user can manage. The EditTags policy has been
split into:

• EditTags: to add tags.
• EditGlossaryTerms: to add Glossary Terms.
• EditTier: to add Tier tags.

Collate - Metadata Actions for ML Tagging - Deprecation Notice

Since we are introducing the Auto Classification workflow, we are going to remove in 1.7 the ML Tagging action
from the Metadata Actions. That feature will be covered already by the Auto Classification workflow, which even brings
more flexibility allow the on-the-fly usage of the sample data for classification purposes without having to store
it in the database.

Service Spec for the Ingestion Framework

This impacts users who maintain their own connectors for the ingestion framework that are NOT part of the
OpenMetadata python library (openmetadata-ingestion).
Introducing the "connector specifcication class (ServiceSpec)".
The ServiceSpec class serves as the entrypoint for the connector and holds the references for the classes that will be used
to ingest and process the metadata from the source.
You can see postgres for an
implementation example.

Fivetran

The filtering of Fivetran pipelines now supports using their names instead of IDs. This change may affect existing configurations that rely on pipeline IDs for filtering.

DBT Cloud Pipeline Service

We are removing the field jobId which we required to ingest dbt metadata from a specific job, instead of this we added a new field called jobIds which will accept multiple job ids to ingest metadata from multiple jobs.

MicroStrategy

The serviceType for MicroStrategy connector is renamed from Mstr to MicroStrategy.

What's New

Visualizing Your Data Landscape with Entity Relationship (ER) Diagrams! (Collate)

Understanding complex database schemas can be challenging without clear visualization. While OpenMetadata's best-in-class Lineage UI helps track data flow, there are better options for viewing structural relationships between tables. Collate 1.6 introduces ER diagrams as a new feature to let you:

• Visualize table connections through primary and foreign key constraints

• Navigate between data assets to discover relationships

• Modify connections using the built-in UI editor

ER diagrams help you better understand and manage your data architecture by showing how your database tables relate to each other.

Establishing Smooth Data Governance with Automated Glossary Approval Workflows! (Collate)

Organizations often struggle with data governance due to rigid, pre-defined manual workflows. OpenMetadata 1.6 introduces a new, automated data governance framework designed to be customized to each organization's needs.

In Collate 1.6, the Glossary Approval Workflow has been migrated to this new framework. Now, you can create custom approval processes with specific conditions and rules and easily visualize them through intuitive workflow diagrams. You can also create smart approval processes for glossary terms with real-time state changes and task creation to save time and streamline work. 

Data Certification Workflows for Automated Bronze, Silver, & Gold Data Standardization! (Collate)

Collate 1.6 also leverages the new data governance framework for a new Data Certification Workflow, allowing you to define your organization's rules to certify your data as Bronze, Silver, or Gold. Certified assets are a great way to help users discover the right data and inform them which data has been properly curated.

Our vision is to expand our governance framework to allow our users to create their own Custom Governance workflows. We want to enable data teams to implement and automate data governance processes that perfectly fit your organization, promoting data quality and compliance.

Maintaining a Healthy Data Platform with Observability Dashboards! (Collate)

Monitoring data quality and incident management across platforms can be challenging. OpenMetadata has been a pillar for data quality implementations, with its ability to create tests from the UI, native observability alerts, and Incident Manager. It offers data quality insights on a per-table level.

In Collate 1.6, we're introducing platform-wide observability dashboards that allow you to track overall data quality coverage trends and analyze incident response performance across your entire data estate. Quickly identify root causes through enhanced asset and lineage views and enable proactive data quality management across your entire data ecosystem.

Elevating Metric Management with Dedicated Metric Entities

Metrics are essential for data-driven organizations, but OpenMetadata previously lacked dedicated metric management, forcing users to use glossary terms as a workaround.

The new "Metric" entity in OpenMetadata 1.6 provides a purpose-built solution to:

• Document detailed metric calculations and descriptions

• Record calculation formulas and implementation code (Python, Java, SQL, LaTeX)

• Visualize metric lineage from source data to insights

This new addition helps teams better manage, understand, and calculate their business KPIs, for improved data literacy and consistency across data teams. 

Reinforcing Data Security with Search RBAC

OpenMetadata's Roles and Policies enable granular permission control, ensuring appropriate access to metadata across different domains and teams. Some data teams may wish to enable data discovery to search for other tables while still enforcing controls with access requests. Other data teams in more restrictive environments may also wish to control the search experience.

OpenMetadata 1.6 extends Role-Based Access Control (RBAC) to search functionality, allowing administrators to tailor user search experience. This provides personalized search results, with users only seeing assets they have permission to access, as well as stronger data governance by ensuring users only interact with data within their defined roles and responsibilities.

Expanded Connector Ecosystem and Diversity

OpenMetadata's ingestion framework contains 80+ native connectors. These connectors are the foundation of the platform and bring in all the metadata your team needs: technical metadata, lineage, usage, profiling, etc.

We bring new connectors in each release, continuously expanding our coverage. This time, release 1.6 comes with seven new connectors:

1. OpenAPI: Extract rich metadata from OpenAPI specifications, including endpoints and schemas.

2. Sigma: Bringing in your BI dashboard information.

3. Exasol: Gain insights into your Exasol database, now supported thanks to Nicola Coretti's OSS contribution!

And in Collate, we are bringing four ETL, dashboarding and ML tools: Matillion, Azure Data Factory, Stitch, PowerBI Server and Vertex AI!

Streamlining Data Management with Additional Enhancements

Release 1.6 comes with several other notable improvements...

1.6.0-release

Backward Incompatible Changes

Ingestion Workflow Status

We are updating how we compute the success percentage. Previously, we took into account for partial success the results
of the Source (e.g., the tables we were able to properly retrieve from Snowflake, Redshift, etc.). This means that we had
an error threshold in there were if up to 90% of the tables were successfully ingested, we would still consider the
workflow as successful. However, any errors when sending the information to OpenMetadata would be considered as a failure.

Now, we're changing this behavior to consider the success rate of all the steps involved in the workflow. The UI will
then show more Partial Success statuses rather than Failed, properly reflecting the real state of the workflow.

Database Metadata & Lineage Workflow

With 1.6 Release we are moving the View Lineage & Stored Procedure Lineage computation from metadata workflow to lineage workflow.

This means that we are removing the overrideViewLineage property from the DatabaseServiceMetadataPipeline schema which will be moved to the DatabaseServiceQueryLineagePipeline schema.

Profiler & Auto Classification Workflow

We are creating a new Auto Classification workflow that will take care of managing the sample data and PII classification,
which was previously done by the Profiler workflow. This change will allow us to have a more modular and scalable system.

The Profiler workflow will now only focus on the profiling part of the data, while the Auto Classification will take care
of the rest.

This means that we are removing these properties from the DatabaseServiceProfilerPipeline schema:

• generateSampleData
• processPiiSensitive
• confidence
  which will be moved to the new DatabaseServiceAutoClassificationPipeline schema.

What you will need to do:

• If you are using the EXTERNAL ingestion for the profiler (YAML configuration), you will need to update your configuration,
  removing these properties as well.
• If you still want to use the Auto PII Classification and sampling features, you can create the new workflow
  from the UI.

RBAC Policy Updates for EditTags

We have given more granularity to the EditTags policy. Previously, it was a single policy that allowed the user to manage
any kind of tagging to the assets, including adding tags, glossary terms, and Tiers.

Now, we have split this policy to give further control on which kind of tagging the user can manage. The EditTags policy has been
split into:

• EditTags: to add tags.
• EditGlossaryTerms: to add Glossary Terms.
• EditTier: to add Tier tags.

Collate - Metadata Actions for ML Tagging - Deprecation Notice

Since we are introducing the Auto Classification workflow, we are going to remove in 1.7 the ML Tagging action
from the Metadata Actions. That feature will be covered already by the Auto Classification workflow, which even brings
more flexibility allow the on-the-fly usage of the sample data for classification purposes without having to store
it in the database.

Service Spec for the Ingestion Framework

This impacts users who maintain their own connectors for the ingestion framework that are NOT part of the
OpenMetadata python library (openmetadata-ingestion).
Introducing the "connector specifcication class (ServiceSpec)".
The ServiceSpec class serves as the entrypoint for the connector and holds the references for the classes that will be used
to ingest and process the metadata from the source.
You can see postgres for an
implementation example.

Fivetran

The filtering of Fivetran pipelines now supports using their names instead of IDs. This change may affect existing configurations that rely on pipeline IDs for filtering.

DBT Cloud Pipeline Service

We are removing the field jobId which we required to ingest dbt metadata from a specific job, instead of this we added a new field called jobIds which will accept multiple job ids to ingest metadata from multiple jobs.

MicroStrategy

The serviceType for MicroStrategy connector is renamed from Mstr to MicroStrategy.

What's New

Visualizing Your Data Landscape with Entity Relationship (ER) Diagrams! (Collate)

Understanding complex database schemas can be challenging without clear visualization. While OpenMetadata's best-in-class Lineage UI helps track data flow, there are better options for viewing structural relationships between tables. Collate 1.6 introduces ER diagrams as a new feature to let you:

• Visualize table connections through primary and foreign key constraints

• Navigate between data assets to discover relationships

• Modify connections using the built-in UI editor

ER diagrams help you better understand and manage your data architecture by showing how your database tables relate to each other.

Establishing Smooth Data Governance with Automated Glossary Approval Workflows! (Collate)

Organizations often struggle with data governance due to rigid, pre-defined manual workflows. OpenMetadata 1.6 introduces a new, automated data governance framework designed to be customized to each organization's needs.

In Collate 1.6, the Glossary Approval Workflow has been migrated to this new framework. Now, you can create custom approval processes with specific conditions and rules and easily visualize them through intuitive workflow diagrams. You can also create smart approval processes for glossary terms with real-time state changes and task creation to save time and streamline work. 

Data Certification Workflows for Automated Bronze, Silver, & Gold Data Standardization! (Collate)

Collate 1.6 also leverages the new data governance framework for a new Data Certification Workflow, allowing you to define your organization's rules to certify your data as Bronze, Silver, or Gold. Certified assets are a great way to help users discover the right data and inform them which data has been properly curated.

Our vision is to expand our governance framework to allow our users to create their own Custom Governance workflows. We want to enable data teams to implement and automate data governance processes that perfectly fit your organization, promoting data quality and compliance.

Maintaining a Healthy Data Platform with Observability Dashboards! (Collate)

Monitoring data quality and incident management across platforms can be challenging. OpenMetadata has been a pillar for data quality implementations, with its ability to create tests from the UI, native observability alerts, and Incident Manager. It offers data quality insights on a per-table level.

In Collate 1.6, we're introducing platform-wide observability dashboards that allow you to track overall data quality coverage trends and analyze incident response performance across your entire data estate. Quickly identify root causes through enhanced asset and lineage views and enable proactive data quality management across your entire data ecosystem.

Elevating Metric Management with Dedicated Metric Entities

Metrics are essential for data-driven organizations, but OpenMetadata previously lacked dedicated metric management, forcing users to use glossary terms as a workaround.

The new "Metric" entity in OpenMetadata 1.6 provides a purpose-built solution to:

• Document detailed metric calculations and descriptions

• Record calculation formulas and implementation code (Python, Java, SQL, LaTeX)

• Visualize metric lineage from source data to insights

This new addition helps teams better manage, understand, and calculate their business KPIs, for improved data literacy and consistency across data teams. 

Reinforcing Data Security with Search RBAC

OpenMetadata's Roles and Policies enable granular permission control, ensuring appropriate access to metadata across different domains and teams. Some data teams may wish to enable data discovery to search for other tables while still enforcing controls with access requests. Other data teams in more restrictive environments may also wish to control the search experience.

OpenMetadata 1.6 extends Role-Based Access Control (RBAC) to search functionality, allowing administrators to tailor user search experience. This provides personalized search results, with users only seeing assets they have permission to access, as well as stronger data governance by ensuring users only interact with data within their defined roles and responsibilities.

Expanded Connector Ecosystem and Diversity

OpenMetadata's ingestion framework contains 80+ native connectors. These connectors are the foundation of the platform and bring in all the metadata your team needs: technical metadata, lineage, usage, profiling, etc.

We bring new connectors in each release, continuously expanding our coverage. This time, release 1.6 comes with seven new connectors:

1. OpenAPI: Extract rich metadata from OpenAPI specifications, including endpoints and schemas.

2. Sigma: Bringing in your BI dashboard information.

3. Exasol: Gain insights into your Exasol database, now supported thanks to Nicola Coretti's OSS contribution!

And in Collate, we are bringing four ETL, dashboarding and ML tools: Matillion, Azure Data Factory, Stitch, PowerBI Server and Vertex AI!

Streamlining Data Management with Additional Enhancements

Release 1.6 comes with several other notable improvements:

• Asynchronous Export APIs: Enjoy increased efficiency...

1.6.0-rc3-release

What's Changed

• Docs: Adding Import Entity Reference by @RounakDhillon in #18854
• Minor secret utils error log by @TeddyCr in #18856
• MINOR: Mysql Lineage Support Main by @ulixius9 in #18780
• MINOR: Catch exception that arises due to a change in code. by @IceS2 in #18863
• Fixes #18848: "Failed to parse query parameters from JDBC URL" when query is an empty string by @plutaniano in #18850
• Minor: improve custom properties e2e test by @Sachin-chaurasiya in #18861
• MINOR: update lineage column with new design by @Ashish8689 in #18862
• [SSO] Google returns null refresh token on Refresh requests by @mohityadav766 in #18868
• Minor: improve the CSV import encoding for column description and glossaryTerms by @Sachin-chaurasiya in #18870
• Docs: Updating Collate in Pipeline Status Widget by @RounakDhillon in #18869
• MINOR: Reorder domain and owner tags by @sweta1308 in #18872
• Fixes issue #18781: Adding lambda permissions to Athena connector documentation by @Farreeda in #18871
• Minor: fixed connection modal style by @ShaileshParmar11 in #18860
• Minor : Fix #18775 validation on enum custom property by @sonika-shah in #18855
• Minor: Fix typo on the ingestion word by @gyohub in #18881
• MINOR - FQN encoding in ometa_api, TestSuite pipeline creation & serialization of test case results by @pmbrull in #18877
• GEN-2157: Snowflake increased cost issue by @harshsoni2024 in #18825
• MINOR: chore(data-quality): improve messaging for COUNT strategy failure by @sushi30 in #18884
• Minor: Fix TestConnectionDefinitionResourceTest update count by @SumanMaharana in #18893
• MINOR: Snowflake Support Alter Table Swap Lineage by @ulixius9 in #18887
• Fix: limit change after pagination by @pranita09 in #18844
• MINOR: chore(importer): better error message for missing plugins by @sushi30 in #18878
• Use certification icons from public folder by @karanh37 in #18902
• Minor: removing logging on jdbcURL by @harshach in #18885
• Minor: Fix dbt source file issue by @SumanMaharana in #18903
• Fixes: microstrategy test connection by @SumanMaharana in #18889
• Make it clear when to use docker compose down by @harshach in #18911
• Create a relationship to manage ER based on tableConstraints by @harshach in #18892
• Minor: improve the service type display name by @Sachin-chaurasiya in #18895
• Fix: show full fqn for reference tables by @pranita09 in #18858
• Run json2ts from maven by @harshach in #18900
• MINOR - Clean up configs & add auto classification docs by @pmbrull in #18907
• MINOR: fix table/schema name by @TeddyCr in #18899
• DOCS - Use the right partials and images dirs for 1.6 by @pmbrull in #18917
• chore(ui): add gifsicle as it's needed for image-webpack-loader by @chirag-madlani in #18919
• GEN-Activity Feed Filtering by @Siddhanttimeline in #18873
• GEN-2186: Fix Updating sampling config from deep link issue by @ShaileshParmar11 in #18916
• MINOR: Set Certification Field Dynamically by @IceS2 in #18874
• Change lineage positioning algorithm by @karanh37 in #18897
• MINOR: Add Try/Except to store procedure yielding by @ulixius9 in #18883
• Minor: fix indexType definition in searchIndex spec by @harshach in #18912
• Fix Table Column Imports by @mohityadav766 in #18924
• chore(ui): remove deprecated image-webpack-loader by @chirag-madlani in #18923
• Added what new content for 1.6.0 by @Kenil27 in #18914
• Reduce amount of flowable created tables by @IceS2 in #18882
• MINOR: Snowflake UDF Lineage Support - main by @ulixius9 in #18886
• FIX: Combined index on offset and eventTime in changeEvent table by @Siddhanttimeline in #18922
• Fixes GEN-2198: Snowflake unencrypted key support, connection error on storing bytes by @ayush-shah in #18927
• Fix : Knowledge page update Parent patch - search side changes by @sonika-shah in #18760
• MINOR: Fix Glossary Term Consolidation and Flaky Test by @IceS2 in #18920
• MINOR: DB2 Support License by @ulixius9 in #18928
• MINOR: Tableau Capture SQL in Lineage by @ulixius9 in #18925
• Fixes GEN-2199: Allow Fivetran filtering of pipelines using name instead of id by @ayush-shah in #18929
• Docs: Updating Collate Ingestion Doc by @RounakDhillon in #18915
• Docs: Reindexing Doc Updation by @RounakDhillon in #18840
• Docs: Updating Bot Images in Developers by @RounakDhillon in #18830
• Docs: Python Version Correction and Separator removal by @RounakDhillon in #18818
• Docs: Developer Bot Image Version Change by @RounakDhillon in #18931
• Support to add and edit primary constraints in Table Constraints modal by @Ashish8689 in #18905
• MINOR: tableau lineage minor errors by @harshsoni2024 in #18932
• Fix: dbt manifest parsing issue by @SumanMaharana in #18930

New Contributors

• @plutaniano made their first contribution in #18850
• @Farreeda made their first contribution in #18871
• @gyohub made their first contribution in #18881

Full Changelog: 1.6.0-rc2-release...1.6.0-rc3-release

1.6.0-rc2-release

What's Changed

• MINOR: Improve PeriodicBatchEntityTrigger Workflows performance by @IceS2 in #18664
• Fixes GEN-1260: Add Validators while creating table to escape special characters by @ayush-shah in #18456
• Add Queue Size in Configuration by @mohityadav766 in #18672
• GEN-1983 : Feat - Support Tag Asset Page by @sonika-shah in #18505
• Change Defaults Values for Indexing by @mohityadav766 in #18675
• FIX: test summary reindexing by @TeddyCr in #18667
• Minor: Fix Column Level Lineage export by @harshach in #18670
• [Snyk] Security upgrade software.amazon.awssdk:ssm from 2.29.11 to 2.29.13 by @harshach in #18651
• Search Index Optimizations by @mohityadav766 in #18649
• Create new Auto Classification Workflow by @pmbrull in #18610
• Minor: Add SanitizedInput Component for Input Sanitization by @Sachin-chaurasiya in #18674
• fix: overflowing metrics for kpi charts by @Kenil27 in #18683
• MINOR: added domain data in lineage side panel and fix relatedColumn search TableConstraints modal by @Ashish8689 in #18671
• fix: update dashboard data model when sql changes by @aballiet in #18657
• Removed Consumer by @mohityadav766 in #18685
• Docs: Gif updation for data discovery by @RounakDhillon in #18676
• Minor: Fix the maven build on main by @aniketkatkar97 in #18692
• Docs: Search Hierarchy Update by @RounakDhillon in #18693
• MINOR: revert aws sdk version from f0b76c4 by @sushi30 in #18695
• fix #18278 "Pipeline ingested from Airflow are now not limited to 100" by @akashverma0786 in #18689
• Add Checksum package to pom by @mohityadav766 in #18696
• Slack app improvements by @Siddhanttimeline in #18684
• MINOR: Move from Date to LocalDate for thread safe operations by @TeddyCr in #18679
• Docs: Metapilot Updation to Collate AI by @RounakDhillon in #18704
• #18677 - Added Lineage Field back to SearchLineage by @TeddyCr in #18687
• Docs: Collate Image Path Rectification by @RounakDhillon in #18706
• FIX: handle potential null node in DQ failure lineage by @TeddyCr in #18707
• Docs: Custom Metric Updation in profiler by @RounakDhillon in #18708
• MINOR: Update Glossary Term tests by @IceS2 in #18698
• MINOR: Add the possibility to have no Schedule and trigger the workflow manually by @IceS2 in #18640
• MINOR - Fallback for testsuite imports by @pmbrull in #18710
• MINOR: Fix missing check on set entity certification by @IceS2 in #18713
• [GEN-970] feat(data-quality): support multiple runtime parameter types by @sushi30 in #18588
• minor: improve rule card actions dropdown by @Sachin-chaurasiya in #18717
• Minor: supporting PR for UI/UX improvement in DQ by @ShaileshParmar11 in #18663
• MINOR - Fix import by @pmbrull in #18721
• Fixed: GEN-1984 Hide Support field from service form by @ShaileshParmar11 in #18724
• TEST - Add autoClassification for e2e by @pmbrull in #18722
• fix: test case result reindex performance by @TeddyCr in #18723
• MINOR: shown NoDataPlaceholder if no search result found in glossary by @Ashish8689 in #18718
• MINOR: doc(mongodb): mongodb+srv connection scheme by @sushi30 in #18712
• fix: modify fqn to allow quotes with dots by @ayush-shah in #18719
• fix the switch placement and added beta tag by @Ashish8689 in #18726
• NoSchedule as ScheduleType for Slack App by @Siddhanttimeline in #18715
• Docs: Private key note added by @RounakDhillon in #18727
• Docs: Airflow Doc Updation by @RounakDhillon in #18725
• Fix: time series of conversion of y-Axis in test case graph by @ShaileshParmar11 in #18735
• Docs: Version Updation on Python SDK by @RounakDhillon in #18739
• Docs: Airflow Doc Quote Correction by @RounakDhillon in #18741
• GEN-2078: fix minor styling of domain component by @karanh37 in #18738
• MINOR : Add Null check for source config by @mohityadav766 in #18746
• docs: add stich docs by @harshsoni2024 in #18666
• Fixes #18441 - Kafka Connect Ingestion Undecrypted Password by @keshavmohta09 in #18742
• Fix: mstr removed dependency issues by @SumanMaharana in #18732
• Minor: fix Lineage export when there is no column / pipeline edge by @harshach in #18737
• Minor: Fix Athena location path by @SumanMaharana in #18750
• [GEN-2109] feat(mongo): added ssl support by @sushi30 in #18731
• Minor: Search RBAC, handle tier tags by @harshach in #18733
• Minor: skip domain check for bots and admin by @harshach in #18734
• Minor: address dq feedback and move lineage tracing in observability layer by @ShaileshParmar11 in #18751
• GEN-1529: Fix View definition tab, query display issue by @ShaileshParmar11 in #18752
• Add async apis for csv import by @sonika-shah in #18647
• GEN-1530: fix DQ Layout Improvements by @ShaileshParmar11 in #18754
• MINOR: Governance Workflows fixes by @IceS2 in #18744
• playwright: fixed failing incident manager test by @ShaileshParmar11 in #18756
• MINOR: modify some test id in table constraints by @Ashish8689 in #18747
• minor: add whats new for 1.5.12 by @Sachin-chaurasiya in #18743
• Minor : update import csv response message by @sonika-shah in #18768
• feat(ci): GEN-2128 add auto cherry pick workflow by @pellejador in #18748
• fix(ui): GEN-1690 sso token refresh issue for OIDC by @chirag-madlani in #18668
• MINOR: Fix StackTraceError validation error - Superset by @ulixius9 in #18740
• MINOR: Add UDF support for SP - backend by @ulixius9 in #18767
• minor: update whats new for 1.5.12 by @Sachin-chaurasiya in #18773
• GEN-1224: fixed Profiler is not displaying correct time for table profile by @ShaileshParmar11 in #18766
• GEN-2073: Not able to copy full logs from log viewer by @ShaileshParmar11 in #18771
• MINOR: doc: service spec breaking changes by @sushi30 in #18688
• MINOR: add reportExplicitAny = false for basedpyright by @sushi30 in #18777
• MINOR: fix(data-diff): get added columns by @sushi30 in #18694
• Minor: Fix Powerbi refresh token on expire by @SumanMaharana in #18774
• Minor: fix dbt ingest failed runs by @SumanMaharana in https://github.com/open-metadata/OpenMetada...

1.5.13-release

What's Changed

• Minor: microstrategy patch login mode 1.5.12 by @SumanMaharana in #18837
• MINOR: Snowflake UDF Lineage Support by @ulixius9 in #18846

Full Changelog: 1.5.12-release...1.5.13-release

1.5.12-release

What's Changed

• Improvement: Added async apis for csv import.
• Improvement: Skip domain check for bots and admin
• Improvement: MYSQL lineage and usage.
• Minor: Added Lineage Field back to SearchLineage.
• Fix: Database is missing from the search dropdown
• Fix: Bad Representation of owners.
• Fix: The Daily Active Users Summary Card in Data Insights.
• Fix: The processing of URL Encoded Assets in Data Insights.
• Fix: Column Level Lineage export.
• Fix: Store procedure yielding by adding Try/Except.
• Fix: Lineage export when there is no column / pipeline edge.

Full Changelog: 1.5.11-release...1.5.12-release

1.6.0-rc1-release

What's Changed

• Doc: Adding SSL Docs for Messaging & Dashboard by @Prajwal214 in #17602
• remove sorting of tags by @karanh37 in #17604
• Minor: Fixed AUT failure by @ShaileshParmar11 in #17607
• MINOR: Fixed owner not updating from dbt if owners field has an empty list set in db by @OnkarVO7 in #17610
• MINOR: Add pagination to entity history versions by @IceS2 in #17587
• minor: add some spacing for table item action when floating icons are enabled by @Sachin-chaurasiya in #17612
• fix: pass the correct argument to get_reference_by_email for dbt ingestion by @nicor88 in #17613
• fix(e2e): api collection owner propagation by @Sachin-chaurasiya in #17616
• Bump Openmetadata Snapshot version by @mohityadav766 in #17621
• migrate glossary tests to playwright by @karanh37 in #17617
• [MINOR] Fix Email Updates by @mohityadav766 in #17620
• MINOR: Bump Ingestion and Dockerfile versions by @ayush-shah in #17624
• migrated task cypress spec in Activity feed playwright by @Ashish8689 in #17618
• MINOR: Fix delete lineage in case of override lineage enabled by @ulixius9 in #17625
• minor(test): migrate to playwright SchemaSearch & DataInsightReportApplication by @chirag-madlani in #17627
• ci(e2e): speed up playwright with 2 machines by @chirag-madlani in #17630
• Fix: lineage lookup for long FQNs by @TeddyCr in #17633
• Connector: vertexai schema by @harshsoni2024 in #17632
• MINOR - Implement SAP Hana Lineage by @pmbrull in #17615
• fix(ui): pick email and name based on claim values at time of login by @chirag-madlani in #17626
• show tier tag with styles by @karanh37 in #17643
• minor: add resizable layout for glossary page by @Sachin-chaurasiya in #17644
• Minor: Fix Edit Reviewers Permission by @harshach in #17638
• minor(ui): what's new for 1.5.2 by @chirag-madlani in #17647
• Minor: added support for edit assign button in incident manager by @ShaileshParmar11 in #17646
• Minor: fixed styling issue for DQ by @ShaileshParmar11 in #17651
• Explore tree reset by @karanh37 in #17653
• minor: update what's new for 1.5.2 by @Sachin-chaurasiya in #17656
• Fixes #16498 : Support custom properties for Data Products by @sonika-shah in #17641
• fix(e2e): remove data insight report if already present by @chirag-madlani in #17654
• PW: migrate query entity spec to playwright by @ShaileshParmar11 in #17619
• test(e2e): service ingestion and schedule options by @chirag-madlani in #16846
• Minor: remove unnecessary repository calls in SearchRepository by @harshach in #17579
• Docs: Updating broken links and Missing Docs by @Prajwal214 in #17642
• migrated user spec to playwright by @Ashish8689 in #17652
• Doc: Updating Connection Docs for SSL in UI for Messaging & Dashboard by @Prajwal214 in #17592
• DOCS - Prep Collate docs structure by @pmbrull in #17655
• FIX - Docs sync workflow syntax by @pmbrull in #17663
• MINOR: Fix Athena lineage models by @ulixius9 in #17662
• fix some entities not visible in Following widget by @Ashish8689 in #17665
• migrated cypress Bots spec to playwright by @Ashish8689 in #17673
• Upgrade dependencies for release 1.6 by @sureshms in #17676
• test: migrate frequently joined spec to playwright by @Sachin-chaurasiya in #17669
• Fixes: Parse postgres json column fields by @SumanMaharana in #17645
• playwright: migrate incident manager test to playwright by @ShaileshParmar11 in #17657
• Minor: Remove the EntityDataConsumer spec dependency on organization inherited role by @aniketkatkar97 in #17667
• fixes 17648: add tags and description for tableau published data source by @nicor88 in #17678
• Bump webpack from 5.76.0 to 5.94.0 in /openmetadata-ui/src/main/resources/ui by @dependabot in #17622
• Migrate advanced search tests by @karanh37 in #17666
• Clean up code warnings flagged by IDE by @sureshms in #17679
• Docs: Updated the broken links in v.1.5.x by @Prajwal214 in #17693
• Docs: Updated the latest Release v.1.5.2 by @Prajwal214 in #17688
• Docs: Updated Kafka Schema Registry Config Broken Link by @Prajwal214 in #17687
• fix permission cypress due to missing file by @Ashish8689 in #17690
• show display name in suggestions with fqn by @karanh37 in #17691
• Connector: VertexAI UI, docs by @harshsoni2024 in #17649
• MINOR - PII Scanner tests and log levels by @pmbrull in #17686
• MINOR: Fixed dbt cloud run_results.json name by @OnkarVO7 in #17694
• MINOR: [DOCS] Added SAP ERP connector documentation by @OnkarVO7 in #17668
• fix(test): ingestion related playwright by @chirag-madlani in #17674
• minor(ui): migrate AddRoleAndAssignToUser.spec by @chirag-madlani in #17670
• minor(test): migrate persona spec by @chirag-madlani in #17701
• fetch domains before any widget is loaded by @karanh37 in #17695
• fix user spec flaky playwright test by @Ashish8689 in #17684
• minor: disable image upload support in description editor by @Sachin-chaurasiya in #17697
• DOCS - OSS deployment is flagged as Collate False by @pmbrull in #17722
• Fixes some things on the APICollection by @IceS2 in #17704
• minor(ui): align dependency version to fix vulnerabilities by @chirag-madlani in #17729
• supported followed data in Following widget using search api by @Ashish8689 in #17689
• Fix #17640: add links to menus by @nakaken-churadata in #17659
• fix ui freezing due to images in feed changes by @Ashish8689 in #17703
• Docs: Updating the Image Reference for Bots in SDK by @Prajwal214 in #17736
• fix expand all operation on terms page by @karanh37 in #17733
• minor(test): fix ingestion related flaky for aut by @chirag-madlani in #17727
• test: migrate login config to playwright by @Sachin-chaurasiya in #17740
• Minor: remove unused dependency by @harshach in #17709
• make cost analysis as collate only by @chirag-madlani in #17719
• GEN-1333 Add TS validation on DQ and Porfiler data ingestion by @TeddyCr in #17731
• [FIX] Jakarta Activation Dependency by @Siddhanttimeline in #17682
• New Email Templates (OSS) by @Siddhanttimeline in https://github.com/open-metadata/O...

1.5.11-release

What's Changed

• Improvement: Search Index App, Parallelize the search indexing process.
• Improvement: Mask SQL Queries where conditions data in Usage & Lineage Workflow.
• Improvement: Databricks query run optimization to reduce the costs in retrieving metadata.
• Improvement: Added support for lineage default depth settings. Admins can control the default Lineage Upstream and Downstream depth.
• Improvement: Add Column Value to be At Expected Location Test.
• Improvement: Scaling Export API to 10000s of assets, Introduced Async API and push export data using websockets
• Fix: User count doesn't update on adding to the team while using search.
• Fix: Sample data json formatting while displaying in the UI.
• Fix: Return s3 endpoint as str() instead of Url.
• Fix: Materialized View Lineage.
• Fix: Custom property save config.

Full Changelog: 1.5.10-release...1.5.11-release

1.5.10-release

What's Changed

• Fix encoding issue for teams search query.
• Fix disable empty rule for query builder widget.
• Fix unable to add more enum values for enum cp.
• Fix navigate to listing for deleting a service.

Full Changelog: 1.5.9-release...1.5.10-release