Skip to content

bug(aurora-global-member): DB subnet group cannot follow a VPC replacement and wedges the module #2974

Description

@leiicamundi

Summary

aws_db_subnet_group cannot be moved between VPCs, and Terraform cannot see that constraint. When the VPC behind an Aurora member is replaced, the module plans an in-place update that AWS refuses, and every subsequent apply replays the same impossible change.

Observed on aws/modules/aurora-global-member, consumed by aws/kubernetes/eks-multi-region-rdbms (#2940). It wedges the two-region path, so it is independent of the multi-region work and of the region opt-in that blocked #2964.

Evidence

From the integration test on #2940, run 30911132617:

Error: updating RDS DB Subnet Group (eks-mr-2940-london-db): operation error RDS:
ModifyDBSubnetGroup, https response error StatusCode: 400,
api error InvalidParameterValue: The new Subnets are not in the same Vpc as the
existing subnet group

  with module.database_region_0[0].aws_db_subnet_group.this,
  on ../../../../modules/aurora-global-member/main.tf line 22

The plan that produced it:

# module.database_region_0[0].aws_db_subnet_group.this will be updated in-place
~ resource "aws_db_subnet_group" "this" {
      id         = "eks-mr-2940-london-db"
      name       = "eks-mr-2940-london-db"
    ~ subnet_ids = [
        - "subnet-041d32ff3b6ab8016",
        ...
        + "subnet-090572f8f4fc77ea8",
        ...
      ]
  }

The outgoing subnets belong to a VPC that no longer exists; the incoming ones are the live VPC's private subnets. Terraform planned an update because the two look interchangeable to it.

Root cause

aws_db_subnet_group has no vpc_id attribute. The VPC is derived from the subnets when the group is created and is immutable from then on. The Terraform schema therefore offers nothing that ties the resource to a VPC, so:

  • a changed subnet list always looks like an in-place update, whatever VPC the new subnets are in;
  • AWS rejects the call whenever the VPC differs;
  • the group is then permanently stuck, because the same plan is produced on every run.

CI makes this reliably reachable rather than theoretical: the nightly cleanup destroys VPCs out of band while the Terraform state persists between runs, so state and reality diverge by design.

Fix

Encode the VPC id in the subnet group name, so the dependency becomes visible to Terraform and a replaced VPC produces a replaced subnet group instead of an update AWS cannot honour.

name = "${var.cluster_identifier}-${trimprefix(var.vpc_id, "vpc-")}"

Considered and rejected: terraform_data + replace_triggered_by on var.vpc_id. It keeps a cleaner resource name, but it does not recover a group that is already orphaned in a destroyed VPC while the state's vpc_id still points at a live one — which is exactly the state CI is in. It would also require raising the module's required_version from >= 1.0 to >= 1.4.

Cost of the chosen fix: the subnet group name is only known after apply, so the golden plan loses three literal names.

Note for the sibling modules

The same latent defect exists in aws/modules/aurora (main.tf:70) and aws/modules/aurora-global (main.tf:31), both of which use a fixed name with no VPC binding. They are deliberately not changed here: they are on main and back shipped architectures, so renaming the subnet group would force a replacement of the group and, since db_subnet_group_name forces a new resource, of the RDS cluster itself on the next apply for every existing deployment. Worth a separate decision rather than a drive-by change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions