[Serve] Add experimental ray-haproxy support behind RAY_SERVE_EXPERIMENTAL_PIP_HAPROXY#62589
Draft
eicherseiji wants to merge 10 commits intoray-project:masterfrom
Draft
[Serve] Add experimental ray-haproxy support behind RAY_SERVE_EXPERIMENTAL_PIP_HAPROXY#62589eicherseiji wants to merge 10 commits intoray-project:masterfrom
eicherseiji wants to merge 10 commits intoray-project:masterfrom
Conversation
…ENTAL_PIP_HAPROXY Signed-off-by: seiji <[email protected]>
2 tasks
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces experimental support for using a vendored HAProxy binary from the ray-haproxy PyPI package, controlled by the RAY_SERVE_EXPERIMENTAL_PIP_HAPROXY feature flag. The changes include updates to CI/CD configurations, Dockerfiles, and the internal HAProxy resolution logic to prioritize the bundled binary when the flag is enabled. Feedback was provided to improve the robustness of the binary resolution by catching broader OSError exceptions instead of just FileNotFoundError, ensuring a more reliable fallback to the system binary in case of permission issues.
Signed-off-by: seiji <[email protected]>
# Conflicts: # python/ray/serve/_private/constants.py # python/ray/serve/_private/haproxy.py
aslonnie
pushed a commit
that referenced
this pull request
Apr 14, 2026
## Summary Install the `ray-haproxy` PyPI package in the Serve CI Docker image. This is a prerequisite for the main PR #62589 which adds `RAY_SERVE_EXPERIMENTAL_PIP_HAPROXY` support. `ray-haproxy` is a standalone binary wheel containing a vendored HAProxy binary at `site-packages/ray_haproxy/bin/haproxy`. It has no Python dependencies (`--no-deps`) and does not add anything to PATH — the binary is only accessible via `from ray_haproxy import get_haproxy_binary`. It will not interfere with the existing system-installed HAProxy in the Serve image. The package is installed but not used by default. It will only be activated when the experimental flag is set in a follow-up PR. ## Test plan - [x] `serve.build.Dockerfile` builds successfully - [x] No impact on existing Serve CI tests (ray-haproxy is installed but not referenced) --------- Signed-off-by: seiji <[email protected]>
The upstream merge already contains ray depsets. Revert CI-only files (serve.rayci.yml, serve.build.Dockerfile, BUILD.bazel, release_tests.yaml) to upstream/master state, keeping only the core code changes (constants.py, haproxy.py, test_haproxy_binary.py). Signed-off-by: seiji <[email protected]>
…/ray into seiji/ray-haproxy-m2 Signed-off-by: seiji <[email protected]>
- Re-add py_test_module_list_with_env_variants for HAProxy tests with system and pip_haproxy variants (was incorrectly reverted with CI files). - Rewrite test_haproxy_binary.py: extract shared setup into fixtures, group tests by resolution stage (flag-off, explicit path, pip package, system fallback), use descriptive names, flatten nesting. Signed-off-by: seiji <[email protected]>
eicherseiji
force-pushed
the
seiji/ray-haproxy-m2
branch
from
5701b10 to
fb41a18
Compare
and trim unit tests to what matters - haproxy.py: widen except FileNotFoundError to except OSError so that PermissionError from a broken bundled binary also falls through to the system haproxy instead of crashing the proxy actor. - test_haproxy_binary.py: 8 tests → 4. Keep: flag-off noop guarantee, explicit path filesystem validation, OSError fallthrough, and actionable error message. Drop tests that just verify return values from mocked branches. Signed-off-by: seiji <[email protected]>
Signed-off-by: seiji <[email protected]>
…list The Ray CI pytest_checker requires all test files to have the if __name__ == "__main__" snippet. Also reverts BUILD.bazel back to py_test_module_list per manual edit. Signed-off-by: seiji <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for using the
ray-haproxyPyPI package (vendored HAProxy binary) in Ray Serve, gated behindRAY_SERVE_EXPERIMENTAL_PIP_HAPROXY=1(defaults to off).When disabled (default): zero behavior change.
get_haproxy_binary()returns"haproxy"and Serve uses the system-installed binary as it does today.When enabled, resolution order:
RAY_SERVE_HAPROXY_BINARY_PATHenv var overrideray-haproxypackagehaproxyon PATHFileNotFoundErrorwith actionable messageOSError (including PermissionError) from the pip package is caught so that a broken bundled binary falls through to the system haproxy rather than crashing the proxy actor.
Changes
constants.py: AddRAY_SERVE_EXPERIMENTAL_PIP_HAPROXYflaghaproxy.py: Addget_haproxy_binary(), wire into_start_and_wait_for_haproxy()BUILD.bazel: Add_pip_haproxyenv variant for all 3 HAProxy test filestest_haproxy_binary.py: Unit tests covering flag-off noop, explicit path validation, OSError fallthrough, and error message qualityCI infrastructure changes (Dockerfile, serve.rayci.yml, release_tests.yaml) are deferred to a follow-up since upstream now uses ray depsets.
Context
This is Milestone 2 of the ray-haproxy deployment plan. Milestone 1 (publishing
ray-haproxyto PyPI) is complete: https://pypi.org/project/ray-haproxy/Test plan
test_haproxy_metricsintegration test passes locally withRAY_SERVE_EXPERIMENTAL_PIP_HAPROXY=1_pip_haproxyBuildkite variant passes