Skip to content

feat: add OrcaRouter support for language models - #3680

Open
xilema2 wants to merge 1 commit into
QuivrHQ:mainfrom
xilema2:feat/orcarouter-support
Open

feat: add OrcaRouter support for language models#3680
xilema2 wants to merge 1 commit into
QuivrHQ:mainfrom
xilema2:feat/orcarouter-support

Conversation

@xilema2

@xilema2 xilema2 commented May 26, 2026

Copy link
Copy Markdown

PR Description

# Description

This PR adds support for **OrcaRouter** (https://www.orcarouter.ai) as a new
LLM supplier in `quivr-core`. OrcaRouter is an OpenAI-compatible multi-vendor
routing gateway that exposes 150+ models from OpenAI / Anthropic / Google /
DeepSeek / xAI / MiniMax / Qwen / Kimi / etc. under a single API key, plus a
synthetic `orcarouter/auto` router that adaptively picks an upstream per
request.

Following the same pattern as the recently-merged Groq (#3633) and Gemini
(#3632) integrations, this PR:

1. Adds `ORCAROUTER = "orcarouter"` to `DefaultModelSuppliers`.
2. Registers eight flagship model entries in `LLMModelConfig._model_defaults`
   under the new supplier (auto router + one flagship per vendor). Any other
   OrcaRouter model ID is still accepted as a free-form string — the
   `LLMEndpointConfig` defaults just fall back to the generic 20k context /
   4k output budget for unrecognized models.
3. Wires up an `ORCAROUTER` branch in `LLMEndpoint.from_config` that uses
   `ChatOpenAI` against `https://api.orcarouter.ai/v1` (the SDK base URL —
   override via `LLMEndpointConfig.llm_base_url` if you point at a
   self-hosted instance). Reasoning models (`anthropic/claude-opus-4.7`,
   `openai/gpt-5*`, `deepseek-reasoner`) reject `temperature` upstream, so
   it's set to `None` for those.

### Usage

```python
from quivr_core.rag.entities.config import (
    LLMEndpointConfig,
    DefaultModelSuppliers,
)
from quivr_core.llm import LLMEndpoint

# API key from env: ORCAROUTER_API_KEY
config = LLMEndpointConfig(
    supplier=DefaultModelSuppliers.ORCAROUTER,
    model="orcarouter/auto",  # or "openai/gpt-5.5", "anthropic/claude-opus-4.7", ...
)
llm = LLMEndpoint.from_config(config)

Disclosure

I'm an engineer on the OrcaRouter team. Happy to iterate on naming, model
list, or default budgets — just let me know what fits Quivr's conventions
best.

Checklist before requesting a review

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented hard-to-understand areas
  • I have ideally added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged

Summary by cubic

Add OrcaRouter as a new LLM supplier, enabling OpenAI-compatible routing to 150+ models through a single endpoint and a synthetic orcarouter/auto router. This adds default model configs, a default base URL, and small behavior tweaks for reasoning models.

  • New Features
    • Added ORCAROUTER to DefaultModelSuppliers.
    • Registered eight default model configs (including orcarouter/auto and vendor flagships); other OrcaRouter IDs still work with generic defaults.
    • Integrated via ChatOpenAI targeting https://api.orcarouter.ai/v1 by default (override with llm_base_url); sets temperature=None for reasoning models like anthropic/claude-opus-4.7, openai/gpt-5*, and deepseek-reasoner.
    • info() now reports the resolved base URL using _effective_base_url.
    • Supports ORCAROUTER_API_KEY and avoids mutating the caller’s config in from_config.

Written for commit dc0a6f5. Summary will update on new commits. Review in cubic

Add OrcaRouter (https://www.orcarouter.ai) as a new LLM supplier alongside
the existing OpenAI / Anthropic / Mistral / Gemini / Groq providers.

OrcaRouter is an OpenAI-compatible multi-vendor routing gateway: clients
talk to a single endpoint with one API key and OrcaRouter routes each
request to the appropriate upstream (or to a configurable router policy
via the synthetic 'orcarouter/auto' model).

- Add ORCAROUTER value to DefaultModelSuppliers.
- Register eight flagship model entries in LLMModelConfig._model_defaults
  (auto router + one flagship per vendor). Any other OrcaRouter model ID
  is still accepted as a free-form string.
- Wire up an ORCAROUTER branch in LLMEndpoint.from_config that uses
  ChatOpenAI against https://api.orcarouter.ai/v1 (overridable via
  llm_base_url). Reasoning models (claude-opus-4.7, gpt-5*,
  deepseek-reasoner) reject 'temperature' upstream, so it is passed as
  None for those.
- Have LLMEndpoint.info() report the resolved base URL via a small
  _effective_base_url helper, so diagnostics show the real endpoint
  instead of the generic 'openai' placeholder.
- Add unit tests covering: default base URL, reasoning-model temperature
  drop, custom base URL override, info() base URL reporting, env var
  resolution (ORCAROUTER_API_KEY), and non-mutation of the caller's
  config inside from_config.
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. area: backend Related to backend functionality or under the /backend directory labels May 26, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Re-trigger cubic

@arturict

Copy link
Copy Markdown

Could you include expected token limits or performance delta from adding Astraflow provider compared to existing providers?

@arturict

Copy link
Copy Markdown

(follow-up) For provider support changes, please include a minimal config example for one new provider to improve review clarity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: backend Related to backend functionality or under the /backend directory size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants