Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Surfacing of DBT Cloud Logs to the logs of the DBTCloudRunJobOperator #46923

Open
1 of 2 tasks
harveymarshall opened this issue Feb 20, 2025 · 1 comment
Open
1 of 2 tasks
Labels
area:logging kind:feature Feature Requests needs-triage label for new issues that we didn't triage yet provider:dbt-cloud

Comments

@harveymarshall
Copy link

Description

Currently if a DBT Cloud Job triggered by the DBT Cloud Job Operator fails. The airflow task which uses the operator will fail and error but it will not get the logs from DBT Cloud so we are aware of why it failed.

Use case/motivation

Easier developer experience when debugging tasks. Helps remove extra steps when seeing why a DAG has failed.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@harveymarshall harveymarshall added kind:feature Feature Requests needs-triage label for new issues that we didn't triage yet labels Feb 20, 2025
Copy link

boring-cyborg bot commented Feb 20, 2025

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

gavin-burns-US added a commit to gavin-burns-US/airflow that referenced this issue Feb 21, 2025
…in DbtCloudRunJobOperator

- Implemented `get_job_run_logs` method in `DbtCloudHook` to retrieve logs for specific dbt Cloud job runs.
- Updated `execute_complete` method in `DbtCloudRunJobOperator` to log final job run logs.
- Enhanced job run monitoring by streaming logs during job execution, improving visibility into job status.

*Relates to apache#46923*
Surfacing of DBT Cloud Logs to the logs of the DBTCloudRunJobOperator

Signed-off-by: Gavin Burns <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:logging kind:feature Feature Requests needs-triage label for new issues that we didn't triage yet provider:dbt-cloud
Projects
None yet
Development

No branches or pull requests

1 participant