Code of Conduct
Search before asking
Describe the bug
This is a follow-up to #7163.
When an uncaught error such as an OutOfMemoryError stops the SparkContext, an engine shutdown can start on another thread but stall before Serverable.stopServer() is reached. For example, a service may block while releasing resources from an already memory-constrained process.
SparkSQLEngine.main waits only on countDownLatch. The latch is released by stopServer(), so if shutdown stalls before that point, the engine main thread waits forever even though its SparkContext has already stopped. In YARN cluster mode this can leave the application master process alive after the driver RPC endpoint has gone away.
#7167 makes the session terminating checker call stop() after detecting a stopped SparkContext, but it cannot unblock the main thread when that stop() call itself stalls.
The engine main thread should also observe the SparkContext state and stop waiting when the context has already stopped.
Affects Version(s)
master
Kyuubi Server Log Output
Not applicable.
Kyuubi Engine Log Output
The failure starts with an uncaught error stopping the SparkContext. Engine shutdown then begins but does not reach the final SparkSQLEngine stopped message, while the process remains alive.
Kyuubi Server Configurations
No special configuration is required.
Kyuubi Engine Configurations
No special configuration is required.
Additional context
Related public reports:
Are you willing to submit PR?
Code of Conduct
Search before asking
Describe the bug
This is a follow-up to #7163.
When an uncaught error such as an
OutOfMemoryErrorstops theSparkContext, an engine shutdown can start on another thread but stall beforeServerable.stopServer()is reached. For example, a service may block while releasing resources from an already memory-constrained process.SparkSQLEngine.mainwaits only oncountDownLatch. The latch is released bystopServer(), so if shutdown stalls before that point, the engine main thread waits forever even though itsSparkContexthas already stopped. In YARN cluster mode this can leave the application master process alive after the driver RPC endpoint has gone away.#7167 makes the session terminating checker call
stop()after detecting a stoppedSparkContext, but it cannot unblock the main thread when thatstop()call itself stalls.The engine main thread should also observe the
SparkContextstate and stop waiting when the context has already stopped.Affects Version(s)
master
Kyuubi Server Log Output
Not applicable.
Kyuubi Engine Log Output
The failure starts with an uncaught error stopping the SparkContext. Engine shutdown then begins but does not reach the final
SparkSQLEnginestopped message, while the process remains alive.Kyuubi Server Configurations
No special configuration is required.
Kyuubi Engine Configurations
No special configuration is required.
Additional context
Related public reports:
Are you willing to submit PR?