Skip to content

Commit 3a8786b

Browse files
LuciferYangdongjoon-hyun
authored andcommitted
[SPARK-57713][CORE] Deduplicate AUTO IOMode resolution in NettyUtils
### What changes were proposed in this pull request? `NettyUtils.createEventLoop`, `getClientChannelClass`, and `getServerChannelClass` each contained an identical copy of the `IOMode.AUTO` resolution (EPOLL on Linux, KQUEUE on macOS, NIO otherwise). This PR extracts that logic into a single private `resolveMode` helper, and each method now switches on the resolved concrete mode. The `AUTO` switch arm becomes unreachable, so it is kept as an explicit arm that throws, which preserves the exhaustiveness check over `IOMode`. ### Why are the changes needed? The resolution logic was duplicated three times, so the methods could easily drift apart when one is updated and the others are not. Keeping it in one place removes that risk. Because every switch still lists all `IOMode` constants explicitly, adding a future transport (for example io_uring) will fail to compile at all three call sites and force each to be handled deliberately. ### Does this PR introduce _any_ user-facing change? No. This is a behavior-preserving refactor: non-AUTO modes pass through unchanged, and AUTO resolves to the same transport as before. ### How was this patch tested? No new tests. The change has no functional difference, and the affected code paths are exercised by the existing `common/network-common` transport suites that run in CI. Built locally with `build/sbt network-common/compile`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 4.8) Closes #56806 from LuciferYang/nettyutils-auto-iomode. Authored-by: YangJie <yangjie01@baidu.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 688064e) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
1 parent 87878a3 commit 3a8786b

1 file changed

Lines changed: 28 additions & 30 deletions

File tree

  • common/network-common/src/main/java/org/apache/spark/network/util

common/network-common/src/main/java/org/apache/spark/network/util/NettyUtils.java

Lines changed: 28 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -65,60 +65,58 @@ public static ThreadFactory createThreadFactory(String threadPoolPrefix) {
6565
return new DefaultThreadFactory(threadPoolPrefix, true);
6666
}
6767

68+
/** Message for the unreachable AUTO arms below; resolveMode never returns AUTO. */
69+
private static final String UNRESOLVED_AUTO_MODE = "AUTO should be resolved by resolveMode";
70+
71+
/**
72+
* Resolves {@link IOMode#AUTO} to a concrete transport for the current platform: EPOLL on
73+
* Linux, KQUEUE on macOS, and NIO otherwise (including when the native transport is not
74+
* available). Any other mode is returned unchanged. Keeping this in one place stops the
75+
* event-loop and channel factories below from drifting apart.
76+
*/
77+
private static IOMode resolveMode(IOMode mode) {
78+
if (mode != IOMode.AUTO) {
79+
return mode;
80+
}
81+
if (JavaUtils.isLinux && Epoll.isAvailable()) {
82+
return IOMode.EPOLL;
83+
} else if (JavaUtils.isMac && KQueue.isAvailable()) {
84+
return IOMode.KQUEUE;
85+
} else {
86+
return IOMode.NIO;
87+
}
88+
}
89+
6890
/** Creates a Netty EventLoopGroup based on the IOMode. */
6991
public static EventLoopGroup createEventLoop(IOMode mode, int numThreads, String threadPrefix) {
7092
ThreadFactory threadFactory = createThreadFactory(threadPrefix);
7193

72-
IoHandlerFactory handlerFactory = switch (mode) {
94+
IoHandlerFactory handlerFactory = switch (resolveMode(mode)) {
7395
case NIO -> NioIoHandler.newFactory();
7496
case EPOLL -> EpollIoHandler.newFactory();
7597
case KQUEUE -> KQueueIoHandler.newFactory();
76-
case AUTO -> {
77-
if (JavaUtils.isLinux && Epoll.isAvailable()) {
78-
yield EpollIoHandler.newFactory();
79-
} else if (JavaUtils.isMac && KQueue.isAvailable()) {
80-
yield KQueueIoHandler.newFactory();
81-
} else {
82-
yield NioIoHandler.newFactory();
83-
}
84-
}
98+
case AUTO -> throw new IllegalStateException(UNRESOLVED_AUTO_MODE);
8599
};
86100
return new MultiThreadIoEventLoopGroup(numThreads, threadFactory, handlerFactory);
87101
}
88102

89103
/** Returns the correct (client) SocketChannel class based on IOMode. */
90104
public static Class<? extends Channel> getClientChannelClass(IOMode mode) {
91-
return switch (mode) {
105+
return switch (resolveMode(mode)) {
92106
case NIO -> NioSocketChannel.class;
93107
case EPOLL -> EpollSocketChannel.class;
94108
case KQUEUE -> KQueueSocketChannel.class;
95-
case AUTO -> {
96-
if (JavaUtils.isLinux && Epoll.isAvailable()) {
97-
yield EpollSocketChannel.class;
98-
} else if (JavaUtils.isMac && KQueue.isAvailable()) {
99-
yield KQueueSocketChannel.class;
100-
} else {
101-
yield NioSocketChannel.class;
102-
}
103-
}
109+
case AUTO -> throw new IllegalStateException(UNRESOLVED_AUTO_MODE);
104110
};
105111
}
106112

107113
/** Returns the correct ServerSocketChannel class based on IOMode. */
108114
public static Class<? extends ServerChannel> getServerChannelClass(IOMode mode) {
109-
return switch (mode) {
115+
return switch (resolveMode(mode)) {
110116
case NIO -> NioServerSocketChannel.class;
111117
case EPOLL -> EpollServerSocketChannel.class;
112118
case KQUEUE -> KQueueServerSocketChannel.class;
113-
case AUTO -> {
114-
if (JavaUtils.isLinux && Epoll.isAvailable()) {
115-
yield EpollServerSocketChannel.class;
116-
} else if (JavaUtils.isMac && KQueue.isAvailable()) {
117-
yield KQueueServerSocketChannel.class;
118-
} else {
119-
yield NioServerSocketChannel.class;
120-
}
121-
}
119+
case AUTO -> throw new IllegalStateException(UNRESOLVED_AUTO_MODE);
122120
};
123121
}
124122

0 commit comments

Comments
 (0)