[serve] Enable gRPC streaming tests under HAProxy - #65089
Open
eicherseiji wants to merge 2 commits into
Open
Conversation
HAProxy proxies gRPC end-to-end over HTTP/2 (proto h2 on the frontend bind and the backend servers), so client-streaming and bidirectional streaming pass through transparently. Drop the skipif markers that gated the 13 gRPC streaming tests in test_grpc.py under RAY_SERVE_ENABLE_HA_PROXY so the HAProxy CI suite exercises them. Verified locally: all 20 streaming cases pass with RAY_SERVE_ENABLE_HA_PROXY=1. Signed-off-by: Seiji Eicher <seiji@anyscale.com>
test_serving_grpc_requests and test_serving_request_through_grpc_proxy each gated their Streaming-method assertion behind an inline conditional that skipped it under HAProxy. RAY_SERVE_ENABLE_HA_PROXY forces RAY_SERVE_ENABLE_DIRECT_INGRESS on (constants.py), so both `if not RAY_SERVE_ENABLE_HA_PROXY` and `if not RAY_SERVE_ENABLE_DIRECT_INGRESS` were false under HAProxy. HAProxy proxies gRPC streaming over end-to-end HTTP/2, so run the assertion unconditionally and drop the now-unused RAY_SERVE_ENABLE_HA_PROXY import. Verified locally with RAY_SERVE_ENABLE_HA_PROXY=1: both tests pass. Signed-off-by: Seiji Eicher <seiji@anyscale.com>
eicherseiji
marked this pull request as ready for review
July 28, 2026 20:24
Contributor
There was a problem hiding this comment.
Code Review
This pull request removes the RAY_SERVE_ENABLE_HA_PROXY constant and the associated conditional skips for gRPC streaming tests in test_grpc.py and test_cli_4.py, allowing these tests to run unconditionally. There are no review comments to evaluate, so I have no additional feedback to provide.
abrarsheikh
approved these changes
Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
HAProxy proxies gRPC end to end over HTTP/2. The gRPC frontend
bindand the backendserverlines both carryproto h2, so HAProxy decodes and forwards HTTP/2 streams, including client-streaming and bidirectional streaming, without any extra handling. gRPC-over-HAProxy support landed in #63735, and the streaming cases were skipped at that time as a scope cut rather than because of a real limitation.What changed
skipif(RAY_SERVE_ENABLE_HA_PROXY, reason="HAProxy does not support streaming.")markers from the 13 gRPC streaming tests intest_grpc.py.Streamingmethod assertion unconditionally intest_serving_grpc_requests(test_grpc.py) andtest_serving_request_through_grpc_proxy(test_cli_4.py). Each previously gated that assertion behind an inline conditional (if not RAY_SERVE_ENABLE_HA_PROXYandif not RAY_SERVE_ENABLE_DIRECT_INGRESS). HAProxy forcesRAY_SERVE_ENABLE_DIRECT_INGRESSon (constants.py), so both conditions were false under HAProxy and the assertion never ran there. Also drops the now-unusedRAY_SERVE_ENABLE_HA_PROXYimport intest_grpc.py.The
serve: HAProxy testsCI step runs the full serve suite underRAY_SERVE_ENABLE_HA_PROXY=1, so these all execute there now instead of being skipped.Testing
Ran under the bundled HAProxy 2.8.25 with
RAY_SERVE_ENABLE_HA_PROXY=1 RAY_SERVE_ENABLE_DIRECT_INGRESS=0:Streaming coverage now exercised under HAProxy: client and bidirectional streaming, the grpc-context variants, timeout, cancellation, backpressure, mid-stream client errors, and the server-streaming
Streamingmethod in the gRPC and CLI end-to-end tests.Not a duplicate
No open PR or issue targets gRPC streaming under HAProxy (checked
gh pr listandgh issue liston ray-project/ray).