Skip to content

GROOVY-9381: Add native async/await support#2387

Open
daniellansun wants to merge 1 commit intomasterfrom
GROOVY-9381_3
Open

GROOVY-9381: Add native async/await support#2387
daniellansun wants to merge 1 commit intomasterfrom
GROOVY-9381_3

Conversation

@daniellansun
Copy link
Contributor

@daniellansun daniellansun commented Mar 1, 2026

@codecov-commenter
Copy link

codecov-commenter commented Mar 1, 2026

Codecov Report

❌ Patch coverage is 90.02268% with 88 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.9330%. Comparing base (7b18440) to head (6106905).

Files with missing lines Patch % Lines
.../org/apache/groovy/runtime/async/AsyncSupport.java 91.4498% 13 Missing and 10 partials ⚠️
...che/groovy/runtime/async/FlowPublisherAdapter.java 83.0508% 7 Missing and 13 partials ⚠️
...ehaus/groovy/transform/AsyncASTTransformation.java 82.6531% 5 Missing and 12 partials ⚠️
...che/groovy/runtime/async/AsyncStreamGenerator.java 84.6154% 4 Missing and 10 partials ⚠️
...va/org/apache/groovy/parser/antlr4/AstBuilder.java 93.9024% 5 Missing ⚠️
...odehaus/groovy/transform/AsyncTransformHelper.java 93.9394% 0 Missing and 4 partials ⚠️
...g/apache/groovy/parser/antlr4/ModifierManager.java 62.5000% 2 Missing and 1 partial ⚠️
...va/groovy/concurrent/AwaitableAdapterRegistry.java 98.6111% 0 Missing and 1 partial ⚠️
...org/apache/groovy/runtime/async/GroovyPromise.java 96.2963% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@                Coverage Diff                 @@
##               master      #2387        +/-   ##
==================================================
+ Coverage     66.7565%   66.9330%   +0.1765%     
- Complexity      29868      30163       +295     
==================================================
  Files            1382       1393        +11     
  Lines          116134     117014       +880     
  Branches        20476      20630       +154     
==================================================
+ Hits            77527      78321       +794     
- Misses          32272      32308        +36     
- Partials         6335       6385        +50     
Files with missing lines Coverage Δ
src/main/java/groovy/concurrent/AsyncStream.java 100.0000% <100.0000%> (ø)
src/main/java/groovy/concurrent/AwaitResult.java 100.0000% <100.0000%> (ø)
src/main/java/groovy/concurrent/Awaitable.java 100.0000% <100.0000%> (ø)
.../main/java/groovy/concurrent/AwaitableAdapter.java 100.0000% <100.0000%> (ø)
...ain/java/org/codehaus/groovy/ast/ModifierNode.java 77.7778% <100.0000%> (+0.4193%) ⬆️
...va/groovy/concurrent/AwaitableAdapterRegistry.java 98.6111% <98.6111%> (ø)
...org/apache/groovy/runtime/async/GroovyPromise.java 96.2963% <96.2963%> (ø)
...g/apache/groovy/parser/antlr4/ModifierManager.java 90.3614% <62.5000%> (-3.0596%) ⬇️
...odehaus/groovy/transform/AsyncTransformHelper.java 93.9394% <93.9394%> (ø)
...va/org/apache/groovy/parser/antlr4/AstBuilder.java 86.7654% <93.9024%> (+0.2581%) ⬆️
... and 4 more

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements GROOVY-9381 by adding native async/await syntax to the Groovy language (including for await, yield return async generators, and defer), plus runtime support and extensive documentation/tests.

Changes:

  • Extend the Groovy grammar + AST builder to parse/compile async, await, for await, yield return, and defer.
  • Add/extend runtime primitives (Awaitable, AsyncStream, adapters/registry, promise + generator implementation) and @Async AST transformation support.
  • Add comprehensive tests and new spec documentation; add Reactor/RxJava test dependencies for adapter interop coverage.

Reviewed changes

Copilot reviewed 29 out of 30 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
versions.properties Adds version pins for Reactor and RxJava3 used by new integration tests.
build.gradle Adds Reactor/RxJava3 as testImplementation dependencies.
gradle/verification-metadata.xml Adds verification metadata entries for new test dependencies.
src/antlr/GroovyLexer.g4 Introduces lexer tokens for async, await, defer.
src/antlr/GroovyParser.g4 Adds grammar support for async closure/lambda, await expressions, for await, yield return, and defer.
src/main/java/org/apache/groovy/parser/antlr4/AstBuilder.java Lowers new syntax into runtime calls (e.g., AsyncSupport.await, stream conversion/cleanup, defer, yieldReturn).
src/main/java/org/codehaus/groovy/ast/ModifierNode.java Treats ASYNC as a non-bytecode modifier (opcode 0).
src/main/java/org/codehaus/groovy/transform/AsyncTransformHelper.java Centralizes AST construction + scanning/rewriting for async constructs.
src/main/java/org/codehaus/groovy/transform/AsyncASTTransformation.java Implements @Async transformation using shared helper logic (await rewrite, generator/defer handling).
src/main/java/groovy/transform/Async.java Adds @Async annotation and documentation.
src/main/java/groovy/concurrent/* Adds new async public API (Awaitable, AsyncStream, adapters/registry, AwaitResult).
src/main/java/org/apache/groovy/runtime/async/* Adds runtime implementations (GroovyPromise, AsyncStreamGenerator) used by compiler output.
src/spec/doc/core-async-await.adoc Adds user-facing language documentation/spec for async/await.
src/test/groovy/org/codehaus/groovy/transform/* Adds extensive test coverage for async/await, generators, defer, adapters, and virtual threads.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

| VOLATILE
| DEF
| VAR
| ASYNC
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding ASYNC to the general modifier rule means async can now appear anywhere modifiers are allowed (e.g., on fields, variables, classes) and will likely be silently ignored (since it maps to opcode 0). If async is intended to be a method-only modifier, consider restricting it in the grammar or adding a parse/semantic error when async is used in unsupported contexts, to avoid misleading code compiling without effect.

Suggested change
| ASYNC

Copilot uses AI. Check for mistakes.
Comment on lines +153 to +158
def customPool = Executors.newFixedThreadPool(2, { r ->
def t = new Thread(r)
t.setName("custom-async-" + t.getId())
t
})
Awaitable.setExecutor(customPool)
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The custom fixed thread pool created here is never shut down. Since the thread factory creates non-daemon threads, this can leak threads and potentially hang the test JVM. Please ensure the executor is shut down (preferably in the existing finally block) after restoring the previous executor.

Copilot uses AI. Check for mistakes.
Comment on lines +180 to +190
Awaitable.setExecutor(Executors.newSingleThreadExecutor())
assert Awaitable.getExecutor() != originalExecutor
// Reset to null — should restore default
Awaitable.setExecutor(null)
def restored = Awaitable.getExecutor()
assert restored != null
// Verify it works
def task = async { 42 }; def awaitable = task()
assert await(awaitable) == 42
// Restore original
Awaitable.setExecutor(originalExecutor)
Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test sets the global executor to a newSingleThreadExecutor but never shuts that executor down. Even though the executor is reset to null later, the underlying thread can remain alive and leak across the suite. Please shut down the created executor (e.g., keep a reference and shutdown in a finally block).

Suggested change
Awaitable.setExecutor(Executors.newSingleThreadExecutor())
assert Awaitable.getExecutor() != originalExecutor
// Reset to null — should restore default
Awaitable.setExecutor(null)
def restored = Awaitable.getExecutor()
assert restored != null
// Verify it works
def task = async { 42 }; def awaitable = task()
assert await(awaitable) == 42
// Restore original
Awaitable.setExecutor(originalExecutor)
def customExecutor = Executors.newSingleThreadExecutor()
try {
Awaitable.setExecutor(customExecutor)
assert Awaitable.getExecutor() != originalExecutor
// Reset to null — should restore default
Awaitable.setExecutor(null)
def restored = Awaitable.getExecutor()
assert restored != null
// Verify it works
def task = async { 42 }; def awaitable = task()
assert await(awaitable) == 42
} finally {
// Restore original and shut down custom executor to avoid thread leaks
Awaitable.setExecutor(originalExecutor)
customExecutor.shutdown()
}

Copilot uses AI. Check for mistakes.
Comment on lines +202 to +207
static Executor myPool = Executors.newFixedThreadPool(1, { r ->
def t = new Thread(r)
t.setName("my-pool-thread")
t
})

Copy link

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The static Executor created via Executors.newFixedThreadPool uses non-daemon threads and is never shut down. This can leak threads for the remainder of the test run and may prevent the JVM from exiting. Consider using daemon threads (as other tests do) and/or explicitly shutting down the pool at the end of the script.

Copilot uses AI. Check for mistakes.
@asf-gitbox-commits asf-gitbox-commits force-pushed the GROOVY-9381_3 branch 2 times, most recently from c3808d6 to 1afcb33 Compare March 8, 2026 05:25
@asf-gitbox-commits asf-gitbox-commits force-pushed the GROOVY-9381_3 branch 5 times, most recently from 7dde6e5 to 9124f63 Compare March 10, 2026 16:53
@daniellansun daniellansun changed the title GROOVY-9381: Support async/await like ES7 GROOVY-9381: Add native async/await support Mar 10, 2026
@daniellansun daniellansun requested a review from Copilot March 12, 2026 22:09
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 29 out of 32 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

throw createParsingFailedException("for await requires enhanced for syntax: for await (item in source)", ctx);
}

ClassNode varType = this.visitType(enhCtrl.type());
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enhCtrl.type() is likely optional in enhanced-for syntax (e.g. for (item in src)), so calling visitType(enhCtrl.type()) can throw if the type node is absent. Align this with the existing enhanced-for handling by defaulting to an implicit/dynamic type when enhCtrl.type() is null (and still applying modifiers via variableModifiersOpt).

Suggested change
ClassNode varType = this.visitType(enhCtrl.type());
ClassNode varType = (enhCtrl.type() != null ? this.visitType(enhCtrl.type()) : ClassHelper.DYNAMIC_TYPE);

Copilot uses AI. Check for mistakes.
Comment on lines +422 to +428
private static <T> void completeFrom(CompletableFuture<T> cf, Future<T> future) {
try {
cf.complete(future.get());
} catch (Exception e) {
cf.completeExceptionally(e.getCause() != null ? e.getCause() : e);
}
}
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hides interrupt semantics: if future.get() throws InterruptedException, the thread interrupt flag is not restored, and the adapted awaitable completes exceptionally with InterruptedException rather than making interruption/cancellation behavior consistent with the rest of the async runtime (which restores interrupts and commonly uses CancellationException with a cause). Handle InterruptedException explicitly (restore interrupt flag and complete exceptionally with a consistent cancellation/interrupt exception) and handle ExecutionException separately to unwrap its cause without conflating it with interruption.

Copilot uses AI. Check for mistakes.
Comment on lines +63 to +68
* Awaitable&lt;Map&gt; fetchProfile(long id) {
* return [name: "User$id"]
* }
*
* {@code @}Async
* Awaitable&lt;List&gt; fetchOrders(long id) {
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Javadoc example contradicts the transformation’s validation in AsyncASTTransformation, which rejects methods that already return Awaitable. Update the example to use a non-Awaitable return type for @Async methods (e.g., Map / List / def) so the documented usage matches the actual compiler behavior.

Suggested change
* Awaitable&lt;Map&gt; fetchProfile(long id) {
* return [name: "User$id"]
* }
*
* {@code @}Async
* Awaitable&lt;List&gt; fetchOrders(long id) {
* Map fetchProfile(long id) {
* return [name: "User$id"]
* }
*
* {@code @}Async
* List fetchOrders(long id) {

Copilot uses AI. Check for mistakes.
ClassNode originalReturnType = mNode.getReturnType();
if (AWAITABLE_TYPE.getName().equals(originalReturnType.getName())
|| ASYNC_STREAM_TYPE.getName().equals(originalReturnType.getName())
|| "java.util.concurrent.CompletableFuture".equals(originalReturnType.getName())) {
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error message says “already returns an async type”, but the check only blocks Awaitable, AsyncStream, and CompletableFuture. Since CompletionStage/Future are also async types in this PR’s model (and are handled by await/adapters), allowing @Async on those return types can yield confusing nested async types (e.g., Awaitable<CompletionStage<T>>). Consider expanding this validation to include CompletionStage and Future (or adjust the error text/documentation to reflect the narrower restriction).

Suggested change
|| "java.util.concurrent.CompletableFuture".equals(originalReturnType.getName())) {
|| "java.util.concurrent.CompletableFuture".equals(originalReturnType.getName())
|| "java.util.concurrent.CompletionStage".equals(originalReturnType.getName())
|| "java.util.concurrent.Future".equals(originalReturnType.getName())) {

Copilot uses AI. Check for mistakes.
@Test
void testAwaitReactorMonoDeferred() {
assertScript REACTOR_PREAMBLE + '''
def mono = Mono.fromSupplier { Thread.sleep(50); "deferred" }
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests relying on small Thread.sleep(...) durations can be flaky on loaded CI agents (timing variance) while not materially improving coverage of the feature. Prefer using deterministic synchronization (e.g., latches) or removing the sleep if the test intent is only to validate await semantics on a deferred publisher.

Suggested change
def mono = Mono.fromSupplier { Thread.sleep(50); "deferred" }
def mono = Mono.fromSupplier { "deferred" }

Copilot uses AI. Check for mistakes.
@daniellansun daniellansun requested a review from Copilot March 13, 2026 14:39
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 29 out of 32 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +225 to +238
public Awaitable<Boolean> moveNext() {
if (closed.get()) {
return Awaitable.of(false);
}
Thread currentThread = Thread.currentThread();
consumerThread.set(currentThread);
// Double-check after registration: if close() raced between the first
// closed check and consumerThread.set(), the consumer reference was not
// yet visible to close(), so no interrupt was delivered. Without this
// re-check the consumer would block in queue.take() indefinitely.
if (closed.get()) {
consumerThread.compareAndSet(currentThread, null);
return Awaitable.of(false);
}
Comment on lines +286 to +322
private static <T> AsyncStream<T> publisherToAsyncStream(Flow.Publisher<T> publisher) {
BlockingQueue<Object> queue = new LinkedBlockingQueue<>(256);
AtomicReference<Flow.Subscription> subRef = new AtomicReference<>();
AtomicBoolean closedRef = new AtomicBoolean(false);

publisher.subscribe(new Flow.Subscriber<T>() {
@Override
public void onSubscribe(Flow.Subscription s) {
if (!closedRef.get()) {
subRef.set(s);
s.request(1);
} else {
s.cancel();
}
}

@Override
public void onNext(T item) {
if (!closedRef.get()) {
queue.offer(new ValueSignal<>(item));
}
}

@Override
public void onError(Throwable t) {
if (!closedRef.get()) {
queue.offer(new ErrorSignal(t));
}
}

@Override
public void onComplete() {
if (!closedRef.get()) {
queue.offer(COMPLETE_SENTINEL);
}
}
});
Comment on lines +412 to +417
cf.completeExceptionally(t);
}

@Override
public void onComplete() {
if (!cf.isDone()) cf.complete(null);
if (AWAITABLE_TYPE.getName().equals(originalReturnType.getName())
|| ASYNC_STREAM_TYPE.getName().equals(originalReturnType.getName())
|| "java.util.concurrent.CompletableFuture".equals(originalReturnType.getName())) {
addError(MY_TYPE_NAME + " cannot be applied to a method that already returns an async type", mNode);
@asf-gitbox-commits asf-gitbox-commits force-pushed the GROOVY-9381_3 branch 3 times, most recently from a0115c5 to 88e960c Compare March 13, 2026 17:51
@daniellansun daniellansun requested a review from Copilot March 13, 2026 17:53
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 34 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +55 to +65
Thread producer = Thread.start {
gen.attachProducer(Thread.currentThread())
try {
gen.yield(1)
gen.yield(2)
gen.yield(3)
gen.complete()
} finally {
gen.detachProducer(Thread.currentThread())
}
}
Comment on lines +369 to +370
// Using put() after clear() guarantees delivery because
// the queue was just emptied and has capacity.
Comment on lines +244 to +248
Thread.start {
Thread.sleep(50)
publisher.submit('hello')
publisher.close()
}
@asf-gitbox-commits asf-gitbox-commits force-pushed the GROOVY-9381_3 branch 2 times, most recently from 6529bbe to e38ef50 Compare March 14, 2026 02:46
@daniellansun daniellansun requested a review from Copilot March 14, 2026 02:55
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 34 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +1791 to +1794
// Inject @Async annotation for methods declared with the 'async' keyword modifier
if (modifierManager.containsAny(ASYNC)) {
methodNode.addAnnotation(new AnnotationNode(ClassHelper.make("groovy.transform.Async")));
}
Comment on lines +934 to +950
public ExpressionStatement visitDeferStmtAlt(final DeferStmtAltContext ctx) {
Expression action;
ExpressionStatement stmtExprStmt = (ExpressionStatement) this.visit(ctx.statementExpression());
Expression expr = stmtExprStmt.getExpression();
if (expr instanceof ClosureExpression) {
// Already a closure/lambda — use directly as the defer action
action = expr;
} else {
// Wrap the statement expression in a closure: { -> expr }
ClosureExpression wrapper = closureX(Parameter.EMPTY_ARRAY, block(stmtExprStmt));
wrapper.setSourcePosition(stmtExprStmt);
action = wrapper;
}
// Emit: AsyncSupport.defer($__deferScope__, action)
Expression deferCall = AsyncTransformHelper.buildDeferCall(action);
return configureAST(new ExpressionStatement(deferCall), ctx);
}
Comment on lines +268 to +269
while (publisher.getNumberOfSubscribers() == 0) { Thread.sleep(1) }
subscribed.countDown()
@asf-gitbox-commits asf-gitbox-commits force-pushed the GROOVY-9381_3 branch 4 times, most recently from 98c29c0 to aa4e567 Compare March 14, 2026 07:11
@daniellansun daniellansun requested a review from Copilot March 14, 2026 07:12
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 30 out of 34 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +114 to +130
void attachProducer(Thread thread) {
producerThread.set(thread);
if (closed.get()) {
thread.interrupt();
}
}

/**
* Unregisters the given thread as the producer for this stream.
* Called from a {@code finally} block after the generator body
* completes (normally or exceptionally).
*
* @param thread the producer thread to unregister
*/
void detachProducer(Thread thread) {
producerThread.compareAndSet(thread, null);
}
Comment on lines +926 to +931
@Override
public ExpressionStatement visitYieldReturnStmtAlt(final YieldReturnStmtAltContext ctx) {
Expression expr = (Expression) this.visit(ctx.expression());
Expression yieldCall = AsyncTransformHelper.buildYieldReturnCall(expr);
return configureAST(new ExpressionStatement(yieldCall), ctx);
}
public void onComplete() {
// Publisher completed before emitting — resolve to null
if (done.compareAndSet(false, true)) {
cf.complete(null);
Comment on lines +325 to +360
@Override
public void onError(Throwable t) {
// Cancel subscription eagerly to release upstream resources
Flow.Subscription sub = subRef.getAndSet(null);
if (sub != null) sub.cancel();
try {
// Terminal signals use blocking put() to guarantee delivery —
// the consumer MUST see the error to propagate it correctly.
queue.put(new ErrorSignal(t));
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
// Blocking put() was interrupted. Fall back to non-blocking
// offer(). If that also fails (queue full), set streamClosed
// so the consumer's next moveNext() returns false instead of
// blocking indefinitely.
if (!queue.offer(new ErrorSignal(t))) {
streamClosed.set(true);
}
}
}

@Override
public void onComplete() {
try {
// Blocking put() guarantees the consumer will see the sentinel,
// even if the queue was temporarily full from buffered values.
queue.put(COMPLETE_SENTINEL);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
// Same fallback as onError: try non-blocking offer,
// set streamClosed if that also fails.
if (!queue.offer(COMPLETE_SENTINEL)) {
streamClosed.set(true);
}
}
}
}
def future = task()
// Wait until the subscriber is registered with the publisher
while (publisher.getNumberOfSubscribers() == 0) { Thread.sleep(1) }
def publisher = new SubmissionPublisher<Integer>()
def future = new StreamConsumer().consumeAll(publisher)
Thread.start {
Thread.sleep(50)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants