get_column_values compiler error never gets raised - failing silently

[jpmmcneill]

Describe the bug

The get_column_values compiler error is not working.

dbt-utils/macros/sql/get_column_values.sql

Line 25 in d279542

{{ exceptions.raise_compiler_error("In get_column_values(): relation " ~ target_relation ~ " does not exist and no default value was provided.") }}

is blocked by

dbt-utils/macros/sql/get_column_values.sql

Line 7 in d279542

{% set default = [] %}

Steps to reproduce

Run the get_column_values against a table where the relation doesn't exist, without a default passed. Instead of raising a compiler error, [] is returned as a value.

Expected results

A compiler error should be raised in the above case.

Actual results

The macro fails silently.

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• snowflake
• other (specify: ____________)

The output of dbt --version:

1.0.1

y other context about the problem here. For example, if you think you know which line of code is causing the issue.
--->

Are you interested in contributing the fix?

I'll open a PR to fix.

[jpmmcneill]

This is quite a serious bug in my opinion. It was causing production code to break in some edge cases, and the macro should catch this case

[joellabes]

Hi @jpmmcneill, thanks for calling this out! The change was made in #386 to resolve a different error.

A potential PR to resolve this, while retaining the same behaviour, would be to remove this block

dbt-utils/macros/sql/get_column_values.sql

Lines 6 to 8 in d279542

	{% if default is none %}
	{% set default = [] %}
	{% endif %}

and replace the return block in

dbt-utils/macros/sql/get_column_values.sql

Lines 10 to 12 in d279542

	{%- if not execute -%}
	{{ return(default) }}
	{% endif %}

with {{ return(default or []) }} which I think would have the same result.

We're looking to cut the next patch of dbt utils next week - if you can get a PR turned around in the next couple of days I'd be happy to include this!

Please use next/patch as your base branch to avoid merge conflicts when it comes time to open your PR.

[jpmmcneill]

Thanks @joellabes, PR open over here: #533

Do you think we should swap out load_relation to get_or_create_relation?

I ran into this where load_relation was silently failing, but in our system we replaced it with a subquery and everything worked out fine, so it was more of a state issue [we're using ECS so this probably isn't typical behaviour]

[joellabes]

Fixed in #533