Sling ELT Error when using Sling's env.yaml to manage connections #25925
Labels
integration: embedded-elt
Related to dagster-embedded-elt which uses Sling and data Load Tool (dlt)
type: bug
Something isn't working
What's the issue?
I'm using sling's env.yaml file to manage my database connections, that way I can easily run the same sling commands outside of dagster when needed.
I just add a sling resource with no connections like below and everything worked well prior to version 1.8.12
sling_resource = {"sling": SlingResource(connections=[])}
Now I receive the following error when materializing my sling assets.
I've narrowed the error down to, "dagster_embedded_elt/sling/resources.py", line 407.
object_key = stream.get("config", {}).get("object")
The "config" key is set to None not null, so
stream.get("config", {})
returns None instead of{}
and the.get{"object")
fails.If you update line to 407 to:
object_key = (stream.get("config") or {}).get("object")
It works as expected.
What did you expect to happen?
Assets to be materialized without errors.
How to reproduce?
Reproduce by using sling's env.yaml to define your database connections and use a blank SlingResource() object with no connections in your dagster project.
sling_resource = {"sling": SlingResource(connections=[])}
and attempt to materialize an asset.
Dagster version
dagster, version 1.9.0
Deployment type
Local
Deployment details
No response
Additional information
If you update line to 407 of "dagster_embedded_elt/sling/resources.py" to:
object_key = (stream.get("config") or {}).get("object")
Assets materialize as expected.
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
By submitting this issue, you agree to follow Dagster's Code of Conduct.
The text was updated successfully, but these errors were encountered: