Allow configuring cascade deletion for pipelines - #5860
Conversation
9f670ea to
a2b7bd7
Compare
a2b7bd7 to
a3653d5
Compare
ronaldz-db
changed the title
cascade deletion for pipelines
|
I have another PR: #5600 that doesn't expose |
|
@alexott oh great! There was a bug in the Go SDK that was recently fixed: databricks/databricks-sdk-go@a00222d We can now merge either PR, but imo we should keep the |
|
I'm not sure about |
|
Synced offline with Alex - unfortunately we do need to provide a default value here and cannot solely rely on the server default. That said, we do need to be careful here and in DABs for changes to defaults |
ronaldz-db
changed the title
cascade deletion for pipelines Align the pipeline destroy-time attribute name with the DABs config field (databricks/cli#5846), which settled on cascade_on_destroy. Co-authored-by: Isaac
GitHub Actions
Inactive
Unit testsIf this PR is from a fork, the If this PR changes Actions -> Warm Go Cache -> Run workflow -> pr_number = 5860 Re-run the failed check once the cache warming completes. Integration testsIntegration tests don't run automatically for external contributors; an authorized user can run them manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
## Changes Add a `cascade_on_destory` field to the pipeline bundle resource that controls whether destroying a pipeline also cascades to its datasets. When unset, we maintain the existing default of true. This change applies to the direct engine. We have a separate change on the Terraform side: databricks/terraform-provider-databricks#5860 ## Why Previously, pipeline deletion would automatically cascade to its tables. After user feedback, we decided to make this configurable in the DeletePipeline API. This extends support to DABs as well, which was another common user ask. ## Tests Unit tests <!-- If your PR needs to be included in the release notes for next release, add a separate entry in NEXT_CHANGELOG.md as part of your PR. -->
2 participants
Changes
For pipeline deletion, we offer a cascade parameter that controls whether the pipeline deletion cascades to datasets as well. We are extending support to Terraform, which is a key customer ask.
Based on DABs core feedback, we will name this new Terraform field for pipelines as
cascade_on_destroyWe are making a corresponding change in DABs as well: databricks/cli#5846
Tests
make testrun locallydocs/folderinternal/acceptanceNEXT_CHANGELOG.mdfile