Skip to content

Fix: Add configurable timeout for VM deletion cleanup (#300)#346

Merged
ppc64le-cloud-bot merged 1 commit intoppc64le-cloud:mainfrom
mkumatag:fix/issue-300-cleanup-timeout
Mar 14, 2026
Merged

Fix: Add configurable timeout for VM deletion cleanup (#300)#346
ppc64le-cloud-bot merged 1 commit intoppc64le-cloud:mainfrom
mkumatag:fix/issue-300-cleanup-timeout

Conversation

@mkumatag
Copy link
Contributor

Problem

Fixes #300

When IBM Cloud PowerVS instance deletion fails during cleanup (e.g., VM enters ERROR state), Packer's cleanup process enters an infinite loop, blocking execution indefinitely. This causes CI/CD pipelines to hang and requires manual intervention.

Solution

Added a configurable cleanup_timeout option with intelligent ERROR state detection and graceful failure handling:

  • New Configuration: cleanup_timeout field (default: "10m")
  • Timeout Mechanism: Uses existing pollUntil() helper for consistency
  • ERROR State Detection: Identifies and warns about ERROR states
  • Retry Logic: Gives ERROR states 5 retry attempts before escalating
  • Elapsed Time Tracking: Shows progress in logs
  • Graceful Failure: Provides clear error messages with instance details for manual cleanup

Changes

  • Added CleanupTimeout configuration field to RunConfig
  • Implemented timeout logic in StepCreateInstance.Cleanup() method
  • Added constants: DefaultCleanupTimeout (10m), CleanupPollInterval (10s)
  • Updated builder to parse and pass timeout configuration
  • Auto-generated HCL2 spec and documentation

Usage Example

source "powervs" "example" {
  # ... existing configuration ...
  
  # Optional: Set custom cleanup timeout (default: 10m)
  cleanup_timeout = "15m"
}

Benefits

  • ✅ No more infinite loops during cleanup
  • ✅ Configurable timeout for different environments
  • ✅ Backward compatible (existing configs work with default)
  • ✅ Clear error messages guide manual cleanup when needed
  • ✅ CI/CD friendly (pipelines won't hang indefinitely)

Testing

  • Code builds successfully
  • All imports and syntax verified
  • Documentation auto-generated
  • Backward compatible

Files Modified

  • builder/powervs/common/run_config.go (+20 lines)
  • builder/powervs/step_create_instance.go (+87 lines, -15 lines)
  • builder/powervs/builder.go (+14 lines, -1 line)
  • builder/powervs/builder.hcl2spec.go (+2 lines, auto-generated)
  • docs-partials/builder/powervs/common/RunConfig-not-required.mdx (+7 lines, auto-generated)

Total: +115 insertions, -15 deletions across 5 files

@ppc64le-cloud-bot ppc64le-cloud-bot added the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Mar 14, 2026
@ppc64le-cloud-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mkumatag

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ppc64le-cloud-bot ppc64le-cloud-bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 14, 2026
- Add cleanup_timeout config option (default: 10m)
- Implement timeout mechanism in Cleanup() method
- Add ERROR state detection and retry logic
- Provide graceful failure with clear error messages
- Update documentation and auto-generated code
- Backward compatible with existing configurations

Fixes issue 300
@mkumatag mkumatag force-pushed the fix/issue-300-cleanup-timeout branch from 3c61176 to a87f4b0 Compare March 14, 2026 01:26
@ppc64le-cloud-bot ppc64le-cloud-bot removed the do-not-merge/invalid-commit-message Indicates that a PR should not merge because it has an invalid commit message. label Mar 14, 2026
@mkumatag mkumatag added the lgtm Indicates that a PR is ready to be merged. label Mar 14, 2026
@ppc64le-cloud-bot ppc64le-cloud-bot merged commit ccbefce into ppc64le-cloud:main Mar 14, 2026
19 checks passed
@mkumatag mkumatag deleted the fix/issue-300-cleanup-timeout branch March 14, 2026 01:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

If packer fails to delete a machine the execution is blocked indefinitely

2 participants