feat(training): add ModelEvaluate with reasoning and performance metrics

[aby42]

Closes #1153

Summary

• Scope: cli, training
• Primary skill: Python, LLM Evaluation
• Impacted surfaces:
  • src/training/model_eval/onboard/
  • src/training/model_eval/onboard_eval.py
• Conditional surfaces intentionally skipped: Full E2E inference pipeline (due to local hardware/network constraints)
• Behavior-visible change: yes (Partial implementation of model onboarding CLI)
• Debt entry: none

Validation

• Environment: cpu-local (WSL2 / Debian)
• Fast gate: Partly run (CLI entry and basic parameter parsing verified)
• Feature gate: n/a
• Local smoke / E2E: N/A (Full model inference not verified due to environment constraints)
• CI expectations / blockers: none

Description & Handover Note

I have implemented the core logic for the onboarding CLI and partial system signal evaluation.

What's verified: - The CLI entry points and configuration parsing are functional locally.

• Basic signal calculation logic has been implemented and "vibe-checked".

Constraints:
Due to persistent local environment issues (WSL network hijacking and hardware limitations), I cannot fully validate the heavy model-inference parts of the pipeline. To avoid blocking the project's progress, I am unassigning myself so someone with a more robust setup can take this over.

My progress is available in the feat/model_onboard branch as a reference.

Checklist

• PR title uses the repo prefix format: [Feat]
• If the PR spans multiple categories, the title includes all relevant prefixes
• Commits in this PR are signed off with git commit -s
• Source-of-truth docs or indexed debt entries were updated when applicable
• The validation results above reflect the actual commands or blockers for this change

[netlify]

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	a25cfc6
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/69ca76072f3179000846e712
😎 Deploy Preview	https://deploy-preview-1650--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 Root Directory

Owners: @rootfs, @Xunzhuo
Files changed:

• .gitignore

📁 src

Owners: @rootfs, @Xunzhuo, @wangchen615
Files changed:

• src/training/model_eval/onboard/README.md
• src/training/model_eval/onboard/__init__.py
• src/training/model_eval/onboard/__main__.py
• src/training/model_eval/onboard/arc_eval.py
• src/training/model_eval/onboard/cli.py
• src/training/model_eval/onboard/constants.py
• src/training/model_eval/onboard/evaluator.py
• src/training/model_eval/onboard/mmlu_eval.py
• src/training/model_eval/onboard/report.py
• src/training/model_eval/onboard/system_eval.py
• src/training/model_eval/onboard/thresholds.py
• src/training/model_eval/onboard/types.py
• src/training/model_eval/onboard_eval.py

---

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

[github-actions]

✅ Supply Chain Security Report — All Clear

Scanner	Status	Findings
AST Codebase Scan (Py, Go, JS/TS, Rust)	✅	29 finding(s) — MEDIUM: 22 · LOW: 7
AST PR Diff Scan	✅	No issues detected
Regex Fallback Scan	✅	No issues detected

---

Scanned at 2026-03-30T13:27:11.708Z · View full workflow logs

[rootfs]

@aby42 thanks, can you fix the precommit error? you can run it locally make precommit-local

[Xunzhuo]

what is the relationship with #1624?