Skip to content

[serve] Enable gRPC streaming tests under HAProxy - #65089

Open
eicherseiji wants to merge 2 commits into
ray-project:masterfrom
eicherseiji:seiji/haproxy-grpc-streaming-tests
Open

[serve] Enable gRPC streaming tests under HAProxy#65089
eicherseiji wants to merge 2 commits into
ray-project:masterfrom
eicherseiji:seiji/haproxy-grpc-streaming-tests

Conversation

@eicherseiji

@eicherseiji eicherseiji commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Why

HAProxy proxies gRPC end to end over HTTP/2. The gRPC frontend bind and the backend server lines both carry proto h2, so HAProxy decodes and forwards HTTP/2 streams, including client-streaming and bidirectional streaming, without any extra handling. gRPC-over-HAProxy support landed in #63735, and the streaming cases were skipped at that time as a scope cut rather than because of a real limitation.

What changed

  1. Remove the skipif(RAY_SERVE_ENABLE_HA_PROXY, reason="HAProxy does not support streaming.") markers from the 13 gRPC streaming tests in test_grpc.py.
  2. Run the Streaming method assertion unconditionally in test_serving_grpc_requests (test_grpc.py) and test_serving_request_through_grpc_proxy (test_cli_4.py). Each previously gated that assertion behind an inline conditional (if not RAY_SERVE_ENABLE_HA_PROXY and if not RAY_SERVE_ENABLE_DIRECT_INGRESS). HAProxy forces RAY_SERVE_ENABLE_DIRECT_INGRESS on (constants.py), so both conditions were false under HAProxy and the assertion never ran there. Also drops the now-unused RAY_SERVE_ENABLE_HA_PROXY import in test_grpc.py.

The serve: HAProxy tests CI step runs the full serve suite under RAY_SERVE_ENABLE_HA_PROXY=1, so these all execute there now instead of being skipped.

Testing

Ran under the bundled HAProxy 2.8.25 with RAY_SERVE_ENABLE_HA_PROXY=1 RAY_SERVE_ENABLE_DIRECT_INGRESS=0:

python -m pytest python/ray/serve/tests/test_grpc.py -k streaming
# 20 passed, 23 deselected

python -m pytest \
  python/ray/serve/tests/test_grpc.py::test_serving_grpc_requests \
  python/ray/serve/tests/test_cli_4.py::test_serving_request_through_grpc_proxy
# 2 passed

Streaming coverage now exercised under HAProxy: client and bidirectional streaming, the grpc-context variants, timeout, cancellation, backpressure, mid-stream client errors, and the server-streaming Streaming method in the gRPC and CLI end-to-end tests.

Not a duplicate

No open PR or issue targets gRPC streaming under HAProxy (checked gh pr list and gh issue list on ray-project/ray).

HAProxy proxies gRPC end-to-end over HTTP/2 (proto h2 on the frontend
bind and the backend servers), so client-streaming and bidirectional
streaming pass through transparently. Drop the skipif markers that
gated the 13 gRPC streaming tests in test_grpc.py under
RAY_SERVE_ENABLE_HA_PROXY so the HAProxy CI suite exercises them.

Verified locally: all 20 streaming cases pass with
RAY_SERVE_ENABLE_HA_PROXY=1.

Signed-off-by: Seiji Eicher <seiji@anyscale.com>
@eicherseiji eicherseiji self-assigned this Jul 28, 2026
@eicherseiji eicherseiji added the go add ONLY when ready to merge, run all tests label Jul 28, 2026
test_serving_grpc_requests and test_serving_request_through_grpc_proxy
each gated their Streaming-method assertion behind an inline conditional
that skipped it under HAProxy. RAY_SERVE_ENABLE_HA_PROXY forces
RAY_SERVE_ENABLE_DIRECT_INGRESS on (constants.py), so both
`if not RAY_SERVE_ENABLE_HA_PROXY` and `if not
RAY_SERVE_ENABLE_DIRECT_INGRESS` were false under HAProxy. HAProxy
proxies gRPC streaming over end-to-end HTTP/2, so run the assertion
unconditionally and drop the now-unused RAY_SERVE_ENABLE_HA_PROXY import.

Verified locally with RAY_SERVE_ENABLE_HA_PROXY=1: both tests pass.

Signed-off-by: Seiji Eicher <seiji@anyscale.com>
@eicherseiji
eicherseiji marked this pull request as ready for review July 28, 2026 20:24
@eicherseiji
eicherseiji requested a review from a team as a code owner July 28, 2026 20:24

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes the RAY_SERVE_ENABLE_HA_PROXY constant and the associated conditional skips for gRPC streaming tests in test_grpc.py and test_cli_4.py, allowing these tests to run unconditionally. There are no review comments to evaluate, so I have no additional feedback to provide.

@ray-gardener ray-gardener Bot added the serve Ray Serve Related Issue label Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

go add ONLY when ready to merge, run all tests serve Ray Serve Related Issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants