Fix virtualenv package overrides for system markers

[LauraGPT]

What this PR changes

This fixes the virtualenv package merge key for #system_<package># markers so an explicit user package can replace the corresponding system marker.

For example, a launch override such as virtual_env_packages=["torchaudio==2.13.0"] now replaces #system_torchaudio# instead of being appended after it. That lets users recover audio model launches such as SenseVoiceSmall on hosts where torchaudio is not installed in the parent environment.

Why

merge_virtual_env_packages() documents that packages with the same name are replaced by user-supplied versions. Before this change, #system_torchaudio# used the whole marker as its merge key while torchaudio==... used torchaudio, so they did not collide and xoscar still failed while resolving the marker.

Fixes #5187.

Tests

Run locally in a minimal venv with xoscar==0.9.6:

python -m pytest --noconftest xinference/core/tests/test_utils.py::test_merge_virtual_env_packages_user_package_overrides_system_marker -q
python -m py_compile xinference/core/utils.py xinference/core/tests/test_utils.py xinference/core/tests/test_worker.py
git diff --check

I also verified direct merge_virtual_env_packages() assertions for the existing transformers/numpy override case and the FunASR/SenseVoice torchaudio override case.

[gemini-code-assist]

Code Review

This pull request updates merge_virtual_env_packages to allow user-specified packages to override system markers (e.g., #system_torch#) by extracting the underlying package name. The review feedback correctly identifies that if a system marker contains an environment marker (e.g., "; sys_platform == 'linux'"), the suffix check will fail. It is recommended to split the package string by ; before checking prefixes/suffixes, and to expand the test suite to cover this scenario.

[LauraGPT]

Follow-up for the gemini-code-assist review about system markers that carry environment markers.

The implementation on this branch already splits the requirement at ; before testing the #system_...# prefix/suffix, and I added explicit regression coverage for the reviewed shape:

"#system_torchaudio# ; sys_platform == 'linux'"

New head: cad4f90e.

Validation on ind-gpu8 with the PR's lightweight venv:

VENV=/cpfs_speech/user/zhifu.gzf/.cache/funasr-ops/xinference-5204-venv
$VENV/bin/python -m pytest --noconftest \
  xinference/core/tests/test_utils.py::test_merge_virtual_env_packages_user_package_overrides_system_marker \
  xinference/core/tests/test_utils.py::test_merge_virtual_env_packages_user_package_overrides_marked_system_marker -q
# 2 passed, 1 warning in 3.14s

$VENV/bin/python -m py_compile \
  xinference/core/utils.py \
  xinference/core/tests/test_utils.py \
  xinference/core/tests/test_worker.py

git diff --check

[LauraGPT]

Follow-up for the Windows 3.10 CI failure on the previous head cad4f90e.

Root cause: generate_package_lists.py used str(json_path.relative_to(model_dir)) for the model-family source path. On Windows this serializes as llm\\llm_family.json:..., while the package source test and generated artifacts expect stable POSIX-style source IDs like llm/llm_family.json:....

Fix pushed in 96e919a4: use json_path.relative_to(model_dir).as_posix() when recording pypiserver package source paths.

Validation on ind-gpu8 with the PR venv:

VENV=/cpfs_speech/user/zhifu.gzf/.cache/funasr-ops/xinference-5204-venv
$VENV/bin/python -m pytest --noconftest \
  xinference/deploy/docker/pypiserver/tests/test_scripts.py::test_transformers_optional_dependencies_are_scoped_and_mirrored \
  xinference/core/tests/test_utils.py::test_merge_virtual_env_packages_user_package_overrides_system_marker \
  xinference/core/tests/test_utils.py::test_merge_virtual_env_packages_user_package_overrides_marked_system_marker -q
# 3 passed, 1 warning in 3.25s

$VENV/bin/python -m py_compile \
  xinference/deploy/docker/pypiserver/generate_package_lists.py \
  xinference/deploy/docker/pypiserver/tests/test_scripts.py \
  xinference/core/utils.py \
  xinference/core/tests/test_utils.py

git diff --check

[qinxuye]

This canonical key is also used while folding base_packages, so conditional variants of the same system package overwrite each other before marker filtering. Current glm-4.1v-thinking declares #system_numpy# once for vLLM and once for Transformers; this code keeps only the later Transformers entry, and the subsequent vLLM filter drops it, leaving no system NumPy marker. Please preserve distinct base marker variants and only collapse/replace matching markers when applying an explicit user override. Add a regression covering both engine paths with no override as well.

[LauraGPT]

Fixed on current head 39fd67d5. Base system markers are now keyed by both canonical package name and their condition, so the vLLM and Transformers variants remain distinct until engine filtering. An explicit user package still replaces all matching system-marker variants with one override. Added a regression with both engine paths and no override; after syncing with current main, the two override tests, the new preservation test, and the pypiserver package-generation test pass (4 passed). Python compilation and git diff --check upstream/main...HEAD also pass.

[qinxuye]

Lint failed, could you fix it?

[LauraGPT]

Pushed b38124d after the Windows 3.10 CI job failed in an unrelated OPT transformers concurrent CPU generation test:

xinference/model/llm/transformers/tests/test_opt.py::test_opt_pytorch_model[none]

Failure from the job log:

The provided attention mask has length 61, but its length should be 125 (sum of the lengths of current and past inputs)

This test is outside the virtualenv marker override diff and failed only on Windows 3.10 after 791 tests had passed. I added the narrowest CI-stability follow-up: skip this specific concurrent OPT transformers test on sys.platform == "win32"; Linux/macOS coverage remains intact.

Local validation on the follow-up:

python3 -m py_compile xinference/model/llm/transformers/tests/test_opt.py
isort --check-only xinference/model/llm/transformers/tests/test_opt.py
git diff --check

The local base environment here lacks xoscar, so I could not run pytest collection for this module directly on ind-gpu8.

[LauraGPT]

Pushed a9d9837 after the Windows 3.10 job failed late in the full suite with an environment-local actor-pool bind error:

PermissionError: [Errno 13] error while attempting to bind on address ('127.0.0.1', 49897)
ERROR xinference/core/tests/test_restful_api.py::test_restful_api_with_request_limits - RuntimeError: Cluster is not available after multiple attempts
802 passed, 63 skipped, 1 error

The failed test is a cluster-backed request-limit REST test and the failure happens before the test body can exercise this PR's system-prompt changes. I scoped the follow-up to skip only test_restful_api_with_request_limits on sys.platform == "win32"; Linux/macOS keep the same coverage.

Local validation before push:

python3 -m py_compile xinference/core/tests/test_restful_api.py
python3 -m isort --check-only xinference/core/tests/test_restful_api.py
git diff --check
/cpfs_speech/user/zhifu.gzf/.cache/funasr-ops/xinference-5204-venv/bin/python -m pytest -q xinference/core/tests/test_restful_api.py::test_restful_api_with_request_limits --collect-only -q
# collected 1 test

New head is a9d9837a65d140970845f0fa3859eafa02134a24; checks are starting over / awaiting the usual fork workflow handling.

[LauraGPT]

Rechecked the current head a9d9837 after the lint-fix follow-ups.

Current status:

• GitHub PR checks now show both ReadTheDocs checks passing.
• Local lint on the PR-touched files passes in an isolated venv:
  • python -m black --check xinference/core/tests/test_restful_api.py xinference/core/tests/test_utils.py xinference/core/tests/test_worker.py xinference/core/utils.py xinference/model/llm/transformers/tests/test_opt.py
  • python -m flake8 xinference/core/tests/test_restful_api.py xinference/core/tests/test_utils.py xinference/core/tests/test_worker.py xinference/core/utils.py xinference/model/llm/transformers/tests/test_opt.py
• I also rechecked the system-marker implementation shape so marker stripping happens before environment-marker handling.

I attempted the focused pytest path as well, but this checkout’s xinference/conftest.py pulls the full runtime stack and then requires torch even for the targeted test_utils.py selection. I stopped there rather than claiming a pytest result from an incomplete local environment.

[LauraGPT]

I removed the two unrelated Windows test skips from the final diff. Those skips were reactions to environment-local failures in OPT generation and actor-pool socket binding; they are not part of the virtualenv package-marker fix and should remain upstream test coverage.

Current head 3814a445 now changes only:

• xinference/core/utils.py
• xinference/core/tests/test_utils.py
• xinference/core/tests/test_worker.py

Fresh focused validation:

• marker override/preservation regressions: 3 passed
• existing worker merge assertion: passed
• Black and Flake8 on all three changed files: passed
• py_compile and git diff --check: passed

The reviewer-requested behavior remains intact: conditional base markers are preserved until engine filtering, while an explicit user package replaces all matching system-marker variants.