Adapter Maintainers: Upgrading to dbt-core v1.9.0

[dbeatty10]

Overview

This discussion is for communicating to adapter maintainers the scope of work needed to make use of the changes in dbt-core 1.9.0. If you have questions and concerns, please ask them here for posterity.

If there's something missing, please comment below!

Release timeline

The below table gives the milestones between up to and including the final release.

• v1.9.0b1 - Oct 1, 2024
• v1.9.0b2 - Oct 7, 2024
• v1.9.0b3 - Oct 30, 2024
• v1.9.0b4 - Nov 6th, 2024
• v1.9.0rc1 - Nov 25th, 2024
• v1.9.0rc2 - Dec 2nd, 2024
• v1.9.0 (final) - Dec 9th, 2024

Prior maintainer upgrade guides

• Adapter Maintainers: Upgrading to dbt-core `v1.8.0` #9798
• Adapter Maintainers: Upgrading to dbt-core `v1.7.0` #8307
• Adapter Maintainers: Upgrading to dbt-core `1.6.0` #7958
• Adapter Maintainers: Upgrading to dbt-core 1.5.0 #7213
• Adapter Maintainers: Upgrading to dbt-core 1.4.0 #6624
• Adapter Maintainers: Upgrading to dbt-core 1.3.0 #6011
• Adapter Maintainers: Upgrading to dbt-core 1.2.0 #5468

Example Diffs

• dbt-labs/dbt-postgres@v1.8.1...v1.9.0
• dbt-labs/dbt-redshift@v1.8.0...v1.9.0
• dbt-labs/dbt-bigquery@v1.8.0...v1.9.0
• dbt-labs/dbt-snowflake@v1.8.0...v1.9.0
• dbt-labs/dbt-spark@v1.8.0...v1.9.0

TL;DR

There should be no breaking changes -- we hope and believe that things should work by bumping package and dependency versions -- please let us know if otherwise! We recommend adding tests where possible / appropriate and updating implementations within your adapter as needed to pass those tests -- see below for details.

Microbatch incremental strategy

See here for a detailed maintainer guide that is specific to microbatches: dbt-labs/dbt-adapters#371

Tests

• dbt.tests.adapter.incremental.test_incremental_microbatch.BaseMicrobatch

Examples how to implement

• Adapters that have already implemented valid_incremental_strategies (first available in 1.3):
  • dbt-snowflake#1179
  • dbt-redshift#924
• Adapters that have not already implemented valid_incremental_strategies:
  • dbt-spark#1108
  • dbt-bigquery#1334

Snapshot improvements

There are several improvements listed below that give users more control over their snapshots.

If you have any customizations for macros related to dbt snapshots, you will need to update them so they are compatible with the following features.

Additional context is also available in the epic.

Snapshots: Enable setting a datetime value for dbt_valid_to for current records instead of NULL

For adapters using the default snapshot macros without overrides, this feature will be automatically inherited due to the changes in dbt-adapters#321.

The documentation for this feature was added in docs.getdbt.com#6308.

Test examples

• BaseSnapshotDbtValidToCurrent

Examples how to implement

• dbt-postgres#152
• dbt-spark#1113

Snapshots: Enable hard_deletes config to add a metadata column if a record has been deleted

For adapters using the default snapshot macros without overrides, this feature will be automatically inherited due to the changes in dbt-adapters#317.

The documentation for this feature was added in docs.getdbt.com#6558.

Tests

• dbt.tests.adapter.simple_snapshot.new_record_mode
  • https://github.com/dbt-labs/dbt-adapters/blob/fb11e9fa169c21abc1de3888c4c52cb42a308448/dbt-tests-adapter/dbt/tests/adapter/simple_snapshot/new_record_mode.py#L177

Snapshots: Allow customizing names of metadata fields

For adapters using the default snapshot macros without overrides, this feature will be automatically inherited due to the changes in dbt-adapters#290.

The documentation for this feature was added in docs.getdbt.com#6211.

Test examples

• BaseSnapshotColumnNamesFromDbtProject
• BaseSnapshotColumnNames
• BaseSnapshotInvalidColumnNames

Examples how to implement

• dbt-postgres#145
• dbt-spark#1097

Snapshots: Enable unique_key as a list

For adapters using the default snapshot macros without overrides, this feature will be automatically inherited due to the changes in dbt-adapters#326.

The documentation for this feature was added in docs.getdbt.com#6438.

Test examples

• BaseSnapshotMultiUniqueKey