Cleaning up multiple test resources - @AutoClose with collections?

[manhole]

First off, congrats on the 6.0 release! I've been using JUnit for over 20 years and it keeps getting better. Thanks for all the great work.

I've been looking for a good way to clean up multiple test resources, and thought @AutoClose might be a good fit. Here's an idea that might be useful for others too.

Motivation

@AutoClose currently works well for test fixtures declared as fields, but doesn't seem designed for multiple resources created dynamically within test methods. Tests often need to create multiple resources and ensure they're all cleaned up. Since these resources typically have dependencies, cleanup needs to happen in reverse order of creation (LIFO).

Current workaround:

@Test
void test() {
    Group group = createGroup();
    User user1 = createUser();
    User user2 = createUser();
    
    group.addMembers(user1, user2);
    
    // Test with group and users...
    assertThat(group.getMembers()).hasSize(2);
    
    // Need manual cleanup (e.g., @AfterEach or try-finally) in order: user2, user1, group...
}

Proposal

Allow @AutoClose to work with List:

@AutoClose
private List<AutoCloseable> resources = new ArrayList<>();

@Test
void test() {
    Group group = createGroup();
    resources.add(() -> deleteGroup(group));
    
    User user1 = createUser();
    resources.add(() -> deleteUser(user1));
    
    User user2 = createUser();
    resources.add(() -> deleteUser(user2));
    
    group.addMembers(user1, user2);
    
    // Test with group and users...
    assertThat(group.getMembers()).hasSize(2);
    
    // Automatically closed in LIFO order: user2, user1, group
}

Resources are closed in LIFO order (reverse of addition). Even if closing fails, all resources are still closed (exceptions are added as suppressed).

Why This Helps

1. Simpler tests: No need for boilerplate code to manually clean up individual resources per test, making tests more readable and maintainable
2. Familiar API: Uses standard Java collections, consistent with @TempDir using Path

---

Would this be useful for your use cases?

I've shown List in the proposal, but would other collection interfaces like Deque be better suited for this? (With Java 21, SequencedCollection would be ideal, but JUnit currently targets Java 17.)

This proposal relates to #3367 where ordering was deferred "for now".

This idea is inspired by the Automated Teardown pattern from Gerard Meszaros's xUnit Test Patterns.

Happy to contribute a PR if there's interest!

[mpkorstanje]

At first review, this proposal seems useful but also non-trivial to with respect to the order in which dependencies are closed and the collections that are supported. As such it would be prudent to incubate this first. Have you considered writing a JUnit Jupiter extension? Similar to the current AutoCloseExtension it could operate on a @AutoCloseCollection annotation instead.

If this seems feasible, you could then propose it to the JUnit Pioneer project and incubate it there.

I imagine that for the end user it might look something like this:

@AutoCloseCollection(order = REVERSE)
private List<AutoCloseable> resources = new ArrayList<>();

@Test
void test() {
    Group group = createGroup();
    resources.add(() -> deleteGroup(group));
    
...
    
    // Automatically closed in LIFO order: user2, user1, group
}

[marcphilipp]

Alternatively, this could be addressed using a custom AutoCloseable:

@AutoClose
private ResourceManager resources = new ResourceManager();

@Test
void test() {
    Group group = createGroup();
    resources.add(() -> deleteGroup(group));
    
    User user1 = createUser();
    resources.add(() -> deleteUser(user1));
    
    User user2 = createUser();
    resources.add(() -> deleteUser(user2));
    
    group.addMembers(user1, user2);
    
    // Test with group and users...
    assertThat(group.getMembers()).hasSize(2);
    
    // Automatically closed in LIFO order: user2, user1, group
}

static class ResourceManager implements AutoCloseable {
	private final List<AutoCloseable> resources = new ArrayList<>();
	void add(AutoCloseable resource) {
		resources.add(resource);
	}
    @Override void close() throws Exception {
		List<Exception> failures = new ArrayList<>();
		for (var resource : resources) { // maybe in reverse order?
			try {
				resource.close();
			} catch (Exception e) {
				failures.add(e);
			}
		}
		if (!failures.isEmpty()) {
			var e = failures.getFirst();
			failures.stream().skip(1).forEach(e::addSuppressed);
			throw e;
		}
	}
}

[manhole]

Thank you for the feedback. After my initial post, I learned that @AutoClose runs at different times depending on @TestInstance. Since my main use case is cleanup per test method, I revisited the proposal with that in mind.

Proposals I am Withdrawing

• Adding List support to @AutoClose
  • With @TestInstance(PER_CLASS), it cannot achieve per-test-method cleanup, so it does not meet my goal.
• Introducing a new annotation like @AutoCloseCollection (including providing it as an extension)
  • A name that closely resembles @AutoClose but behaves at different times would be prone to misuse.
  • It would also require implementation that accounts for differences in the @TestInstance lifecycle, which may reduce ease of understanding for users.

Current Direction: A Custom AutoCloseable

Following marcphilipp's suggestion, I will use a custom AutoCloseable. My implementation is a class named AutoCloser, available here (as a personal utility):

https://github.com/manhole/java-autocloser

The closing order is reverse (LIFO), the same as try-with-resources. This is the correct order because resources created later often depend on those created earlier, so they must be closed first.

Usage (Differences by @TestInstance)

With @TestInstance(Lifecycle.PER_METHOD), it can be used together with @AutoClose:

@AutoClose
private final AutoCloser closer = new AutoCloser();

With @TestInstance(Lifecycle.PER_CLASS), it should be closed explicitly in @AfterEach:

private final AutoCloser closer = new AutoCloser();

@AfterEach
void tearDown() throws Exception {
    closer.close();
}

What I Would Like to Ask

This approach solves my own problem, but in practice development happens across multiple people and teams. Introducing a custom class adds additional costs such as agreeing on shared rules, training, and maintaining reusable infrastructure.

Therefore, if possible, it would be very helpful for JUnit to provide a clear, easy-to-recognize path for this use case (either as a standard feature or as an officially recommended approach). If such a path were standardized, it would be easier to build shared understanding within teams, leading to better test quality and maintainability.

In this regard, is the extension approach the recommended route for JUnit, or are user-side solutions the expected default for now?

[marcphilipp]

Therefore, if possible, it would be very helpful for JUnit to provide a clear, easy-to-recognize path for this use case (either as a standard feature or as an officially recommended approach). If such a path were standardized, it would be easier to build shared understanding within teams, leading to better test quality and maintainability.

Personally, I'd be open to including a class like AutoCloser (we'll need to debate the name) into junit-jupiter-api. Please create an issue for it!

[manhole]

Thank you for taking the time to understand the problem — I’m glad the concern came across clearly.
I’ve gone ahead and created an issue #5277 based on this discussion. I look forward to continuing the conversation.

[mpkorstanje]

Thank you for the feedback. After my initial post, I learned that @autoclose runs at different times depending on @testinstance. Since my main use case is cleanup per test method, I revisited the proposal with that in mind.

@manhole could you clarify why your resources must be tied to the test method and not to the test instance? I.e. why is does test instance have a PER_CLASS life cycle when you need PER_METHOD?

[manhole]

@mpkorstanje

Sharing the test instance (@TestInstance(PER_CLASS)) and sharing resources created per test method are different concerns.

PER_CLASS optimizes the test instance lifecycle, but it does not mean resources should be shared across test methods. To preserve test isolation, resources created within a test method must be cleaned up after that method completes.

Therefore, even with a PER_CLASS test instance, resource cleanup still needs per-test-method semantics.

[sormuras]

From

• Add a JUnit Jupiter utility for per-test-method cleanup of multiple AutoCloseable resources #5277

I think that "simple, reliable, and easy-to-understand" contradicts "dynamically created dependent resources, or have lifecycle semantics that are not strictly per-test-method." – Especially the non-locality of the latter, not having everything that matters inside the body of a test method, is far from "easy-to-understand" by other folks, including future-you/me.

Leaving out the create* calls and the actual test logic, this boils down to calling custom runnables to cleanup multiple test resources in LIFO order, right?

jshell> try (
   ...> AutoCloseable _ = () -> IO.println("Delete Group");
   ...> AutoCloseable _ = () -> IO.println("Delete User 1");
   ...> AutoCloseable _ = () -> IO.println("Delete User 2")) {
   ...> }
Delete User 2
Delete User 1
Delete Group

Seems like try-with-resources is fit for the "all in a method body" case.

[sormuras]

Applied to the use-case presented at https://github.com/manhole/java-autocloser?tab=readme-ov-file#use-cases

@Test
void test() {
    var group = createGroup();
    var user1 = createUser();
    var user2 = createUser();
    try (
      AutoCloseable _ = () -> deleteGroup(group);
      AutoCloseable _ = () -> deleteUser(user1);
      AutoCloseable _ = () -> deleteUser(user2);
      ) {
        group.addMembers(user1, user2);
        assertThat(group.getMembers()).hasSize(2);        
    }
}

And if groups and users were auto-closeable resources, it would even collapse to:

@Test
void test() {
    try (
      var group = createGroup();
      var user1 = createUser();
      var user2 = createUser();
      ) {
        group.addMembers(user1, user2);
        assertThat(group.getMembers()).hasSize(2);        
    }
}

In both cases, an extra "resource manager" class and an extension seems to be ... boilerplate-y.

[manhole]

Thank you for the concrete example.
I also appreciate your interest in this topic.

For simple cases, I agree that try-with-resources is a good fit.

However, in real test code it is common to have exception-prone steps between resource creations, and the number of resources can vary per test.
In those cases, try-with-resources becomes deeply nested, and the parts I want to highlight in a test (preconditions, exercise, assertions) tend to get buried.

Another practical issue is that many resources used in tests do not implement AutoCloseable at all (for example, domain objects such as User or Group).
In such cases, try-with-resources cannot be applied directly, and wrapping every resource just to make it AutoCloseable often feels artificial.
Using a small utility with an explicit register() step tends to integrate more naturally with these kinds of APIs.

Example:

try (var group = createGroup()) {
    // Exception-prone work...
    try (var user1 = createUser()) {
        // Exception-prone work...
        try (var user2 = createUser()) {
            // Exception-prone work...
            group.addMembers(user1, user2);
            assertThat(group.getMembers()).hasSize(2);
        }
    }
}

In such cases, I find it more readable to keep the same LIFO / suppressed-exception semantics, but lay the steps out vertically so the subject of the test stays in focus:

var closeables = new AutoCloser();
var group = closeables.register(createGroup());
// Exception-prone work...
var user1 = closeables.register(createUser());
// Exception-prone work...
var user2 = closeables.register(createUser());
// Exception-prone work...

group.addMembers(user1, user2);
assertThat(group.getMembers()).hasSize(2);

So I am not arguing that try-with-resources is wrong.
Rather, my goal is to preserve the same safety guarantees (LIFO ordering and suppressed exceptions) while keeping test methods readable and focused on their intent, rather than on the mechanics of resource management.