[ISSUE] databricks_permissions with instance_pool_id shows permanent drift when deployer is a workspace admin

[jkelbick]

Configuration

resource "databricks_instance_pool" "example" {
  instance_pool_name                    = "example-pool"
  min_idle_instances                    = 0
  node_type_id                          = "Standard_D4ds_v5"
  idle_instance_autotermination_minutes = 10
}

resource "databricks_permissions" "pool_usage" {
  instance_pool_id = databricks_instance_pool.example.id

  access_control {
    group_name       = "DataEngineering"
    permission_level = "CAN_ATTACH_TO"
  }
}

The deploying identity (user or service principal) is a workspace admin.

Expected Behavior

After a successful terraform apply, a subsequent terraform plan should show no changes.

Actual Behavior

Every terraform plan shows the access_control blocks being destroyed and recreated. The Permissions API (GET /api/2.0/permissions/instance-pools/<id>) returns the deploying admin's inherited CAN_MANAGE in the access_control_list response. Since this entry is not in the Terraform config, the provider sees a diff on every plan and produces a destroy/recreate cycle that never converges.

The plan output looks like:

# databricks_permissions.pool_usage will be updated in-place
~ resource "databricks_permissions" "pool_usage" {
    id = "/instance-pools/xxx"

  - access_control {
      - permission_level = "CAN_MANAGE" -> null
      - user_name        = "deployer@domain.com" -> null
    }
  - access_control {
      - permission_level = "CAN_ATTACH_TO" -> null
      - group_name       = "DataEngineering" -> null
    }
  + access_control {
      + permission_level = "CAN_ATTACH_TO"
      + group_name       = "DataEngineering"
    }
}

Steps to Reproduce

1. Authenticate Terraform as a workspace admin (user or service principal)
2. terraform apply — creates the pool and permissions successfully
3. terraform plan — shows drift; access_control blocks will be destroyed and recreated
4. terraform apply — re-applies the same permissions
5. terraform plan — drift appears again. Repeat forever.

Terraform and provider versions

Terraform v1.x
databricks/databricks (latest)
Cloud: Azure

Is it a regression?

Unknown. This may have existed since databricks_permissions was introduced for instance pools.

Debug Output

The relevant diff is visible in a standard terraform plan. The root cause is in the Read function: the API response includes access_control_list entries where inherited == true and inherited_from_object contains /admins. The provider does not filter these out before comparing to state.

Important Factoids

All known workarounds fail for this specific scenario:

Workaround	Why it fails
Explicitly declare deployer's CAN_MANAGE in config	Provider auto-strips admin access_control blocks: "It is not possible to lower permissions for admins, so Databricks Terraform Provider removes those access_control blocks automatically" (docs)
Use a non-admin SP as deployer	allow_instance_pool_create entitlement can only be granted to workspace-local groups (legacy), not account groups. Databricks recommends migrating away from workspace-local groups, making this a non-starter for orgs using account-level identity federation.
Transfer pool ownership via API	No change-owner API exists for instance pools. Only clusters have /api/2.1/clusters/change-owner.
Use databricks_access_control_rule_set instead	Docs state: "cannot be used to manage access rules for resources supported by databricks_permissions"
lifecycle { ignore_changes = [access_control] }	Suppresses all permission drift including real changes — not acceptable for governed environments

Related issues:

• [ISSUE]databricks_permissions with cluster_id objects shows permanent drift if the owner is not the same as the TF identifier #4734 — Same drift pattern for cluster_id (open, no assignee)
• [ISSUE] databricks_permissions shows permanent drift if the owner is not the same as the TF identifier #3730 — Same drift pattern for sql_endpoint_id (closed/fixed). The fix added automatic IS_OWNER handling, but instance pools do not have an IS_OWNER permission level, so the same fix cannot be directly applied.
• [FEATURE] Change "owner" of clusters #2543 — Feature request to change cluster owner via provider (open). Does not cover instance pools.

Proposed fix: During the Read/refresh of databricks_permissions for instance_pool_id, filter out access_control_list entries where inherited == true and inherited_from_object contains /admins before writing to state. This prevents inherited admin permissions from appearing in the diff. This is analogous to the #3730 fix, adapted for the fact that pools lack IS_OWNER.

Would you like to implement a fix?

Not at this time, but the fix path is well-defined: modify the Read function in the databricks_permissions resource to filter inherited admin entries from the API response before comparing to state. The pattern already exists in the #3730 fix for SQL warehouses.