fix(simple_shuffle): honor weights when first deployment has none

[devin-ai-integration]

Relevant issues

Fixes #33329

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Config with two deployments in one model group where the first declares no weight and the second has weight: 99

model_list:
  - model_name: shuffle-test
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
    model_info: {id: no-weight-first}
  - model_name: shuffle-test
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
      weight: 99
    model_info: {id: weighted-second}
router_settings:
  routing_strategy: simple-shuffle

Sending 40 live /v1/chat/completions requests and counting the picked deployment via the x-litellm-model-id response header

Before the fix (commit 69a491e168); weights ignored, roughly uniform split

$ for i in $(seq 1 40); do curl -s -X POST http://localhost:4000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"shuffle-test","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' \
    -D - -o /dev/null | grep -i "x-litellm-model-id" | awk '{print $2}'; done | sort | uniq -c
     21 no-weight-first
     19 weighted-second

After the fix (commit f77948b762); the weight: 99 deployment is honored (the unset weight is treated as 0)

$ for i in $(seq 1 40); do curl -s -X POST http://localhost:4000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"shuffle-test","messages":[{"role":"user","content":"hi"}],"max_tokens":1}' \
    -D - -o /dev/null | grep -i "x-litellm-model-id" | awk '{print $2}'; done | sort | uniq -c
     40 weighted-second

Type

🐛 Bug Fix

Changes

simple_shuffle decided whether weighted routing was enabled by inspecting only the first healthy deployment

weight = healthy_deployments[0].get("litellm_params").get(weight_by, None)
if weight is not None:
    ...

so when the first deployment in a group had no weight/rpm/tpm for the metric, the metric was skipped entirely and every other deployment's weight was silently ignored, falling through to a uniform random pick. Deployment ordering is not something the user controls, so this made weighted routing unreliable

The check now enables the weighted pick when any deployment declares the metric, and a missing value is treated as 0 for that deployment

raw_weights = [m["litellm_params"].get(weight_by, None) for m in healthy_deployments]
if any(weight is not None for weight in raw_weights):
    weights = [weight if weight is not None else 0 for weight in raw_weights]
    ...

Added tests/test_litellm/router_strategy/test_simple_shuffle.py with a regression test that puts a deployment without a weight first, followed by weighted deployments, and asserts the zero-weight one is never picked while the higher-weight one dominates

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/23a58e873a094267bfbd2b3e2496eaf7

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes a bug in simple_shuffle where weighted routing was silently disabled when the first deployment in a group had no weight/rpm/tpm field — causing weights on all other deployments to be ignored and falling back to a uniform random pick. The fix scans all deployments to detect whether any weight is declared, then treats missing values as 0 rather than as "no weighting at all."

• simple_shuffle.py: Replaces the single first-deployment check with an any() scan across all deployments; the existing total_weight <= 0 guard already handles the case where every non-None weight is still zero.
• test_simple_shuffle.py: Adds a mock-only regression test that puts a no-weight deployment first and asserts it is never selected (probability-0 weight) while the 99-weight deployment dominates over 2,000 draws.

Confidence Score: 5/5

Safe to merge — the change is minimal, targeted, and well-tested.

The three-line logic change correctly expands the weighted-routing check from a single deployment to all deployments, and the existing total_weight <= 0 guard already covers the edge case where every non-None weight evaluates to zero. The new test is mock-only (no real network calls), the counts["A"] == 0 assertion is deterministic (weight=0 means random.choices never picks it), and the counts["C"] > counts["B"] * 2 assertion is effectively deterministic given the 99:1 ratio over 2,000 draws.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/router_strategy/simple_shuffle.py	Fixes weighted-routing detection to inspect all deployments instead of only the first; missing weights are now treated as 0 rather than disabling weighting entirely.
tests/test_litellm/router_strategy/test_simple_shuffle.py	New regression test using only mocks (no real network calls); deterministically validates that a zero-weight deployment is never picked and that the higher-weight deployment dominates.

_{Reviews (1): Last reviewed commit: "fix(simple_shuffle): honor weights when ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_simple_shuffle_weights_33329 (f77948b) with litellm_internal_staging (69a491e)}