Transformations: Condition deployment steps
AnsweredWithin our deployment.yml file we are chaining multiple dbt commands one after another. If an error occurs, all following jobs are not executed. It would be great, if I could add a condition per step, whether it should ignore previous errors or not. Something like that:
steps:
- name: run some models
command: dbt run -s model_one
- name: step producing an error
command: dbt test -s model_one
condition: succeded
- name: run some other models
command: dbt test -s model_two
condition: always
In this example, the second step only runs if all previous steps didn't produce any errors, the third step, however, ignores all previous errors and always runs.
All of this can be solved by rearranging or splitting up cron jobs within the deployment.yml, but this would be a much more elegant solution.
Curious to hear what you think about it.
Cheers
Leopold
-
Leopold Gabelmann thank you for this request. There certainly could be some more elegant chaining of transformations in the deployment.yml file, as well as our in-product transformations setup. I will add this to our backlog and plan to evaluate it more holistically in FY24. Upvotes on this feature request will also help us prioritize it!
Please sign in to leave a comment.
Comments
1 comment