feat: add Mistral, Codex, Chrome on-device providers; fix Gemini/DeepSeek streaming and reasoning - #1393
Closed
oakimov wants to merge 37 commits into
Closed
feat: add Mistral, Codex, Chrome on-device providers; fix Gemini/DeepSeek streaming and reasoning#1393oakimov wants to merge 37 commits into
oakimov wants to merge 37 commits into
Conversation
Add MistralTransformer to enable direct integration with Mistral AI API. This transformer handles the conversion from Anthropic-style requests to Mistral's OpenAI-compatible format by: - Removing cache_control fields (not supported by Mistral) - Flattening array content to strings where appropriate - Converting tool_choice 'required' to 'any' (Mistral's equivalent) - Preserving image content in array format when present Fixes musistudio#509
- Remove JSON.parse/stringify deep clone, mutate request directly for consistency with other transformers (groq, cleancache, deepseek) - Add removal from tool function parameters (like groq.transformer) - Remove unnecessary 'as any' type assertion on tool_choice - Simplify transformMessage to mutate in place instead of creating new object
…Docker Compose configuration
…formers and API routes
feat: improve OpenAI API handling borrowig functionality from y-router and claude-code-proxy
…ities for better maintainability across transformers
- gemini.util.ts: flush pendingContent on stream end, fixing lost text for Gemma models with thinking but no thoughtSignature (e.g. gemma-4-31b-it); map Gemini 400 context-too-long errors to finish_reason "model_context_window_exceeded" to trigger Claude Code autocompact; remove temporary debug logs - anthropic.transformer.ts: pass through model_context_window_exceeded as stop_reason in both streaming and non-streaming paths - schema.ts (new): sanitizeJsonSchema strips unknown fields via whitelist and removes invalid format/enum/uri combinations; fixes bug where property schemas bypassed the whitelist; normalizeToolParameters wraps it for tools - groq.transformer.ts, mistral.util.ts: use normalizeToolParameters for full schema sanitization instead of only deleting $schema - gemini.transformer.ts: resolve model name from context before URL build - vertex-gemini.transformer.ts: fall back to provider.model when request.model is empty
…nversations
- gemini.util.ts: Rebuild thinking history as proper {thought:true, thoughtSignature} parts
instead of attaching thoughtSignature to text/functionCall parts; Gemini requires the
thought part structure when replaying conversation history
- anthropic.transformer.ts: Extract plain text from tool_result content arrays instead of
JSON.stringify-ing them, so models receive actual file content rather than JSON wrappers
- routes.ts: Only use SSE streaming path when response.ok; error responses now return as
JSON regardless of stream mode, preventing Web ReadableStream being sent to Fastify
- index.ts: Remove buggy done(payload.error, null)/done(payload, null) calls in onSend hook
that were treating valid payloads as Fastify errors, causing unhandledRejection crashes
…emini tool call handling
…ze thought_signature naming in Gemini integration
…g to use shared stream utilities
… include thinking and tool call parameters.
… reader resilience, and introduce parity testing suite.
… instead of global string replacement
…add Docker Compose configuration to the documentation
… blocks from message payloads
…ze example documentation
…gs.json configuration
…odel fetching utility
… Prompt API Gemini Nano is a ~4GB on-device LLM built into Chrome, accessed through the Prompt API (window.LanguageModel). It runs locally with no API costs or network round-trips, using Chrome's built-in model download. The chrome-on-device transformer routes requests through a host-side bridge (ccr chrome-bridge) that connects to Chrome via CDP/Puppeteer. The bridge maintains a persistent LanguageModel session with responseConstraint (JSON Schema) to force structured tool-call output, replacing Claude Code's verbose system prompt with a compact 7-tool version. It strips system-reminder blocks, truncates tool results, detects whitespace stalling, and auto-compacts at 85% context usage to fit within the ~9K token window and ~1200 char/turn output limit.
… usage for Codex/Responses
- Implement dynamic temperature scaling and fallback retry for whitespace stalling - Add 'OPERATIONAL OVERRIDE' and strict path guidelines to prevent hallucinations - Implement tool result labeling and context-aware result checking - Fix session recreation loop and add parameter clamping for Prompt API - Improve JSON extraction robustness and update documentation
…rome bridge support in README
…tool_result> XML tags and updating system prompts for improved tool interaction consistency.
…hen reflexive tool-loop prevention - Add per-session stats tracking with lastActivityAt timestamps - Add session list table to dashboard with turn count, idle time, context %, and Evict buttons - Add idle session eviction timer (5 min threshold, 60s check interval, 'cli' protected) - Fix updateDashboard() to update for ALL sessions, not just 'cli' gate - Fix updateStats() to accept sessionId instead of hardcoding 'cli' - Fix resetSession() to clear lastActivityAt for proper eviction hygiene - Strengthen [NOTICE] prefix to name exact tools already called and instruct ExitTool usage - Update README.md: multi-session architecture, session eviction, system-reminder transformation docs - Update lessons.md: reflexive loop prevention, first-turn assistant inclusion, multi-session pitfalls - Update TODO.md: mark assistant history management as complete
Revert README.md to upstream version, remove tasks/TODO.md, and remove TODO.md reference from CLAUDE.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR consolidates 30 commits from a downstream fork, bringing new provider support, architectural refactoring, deployment tooling, and reliability improvements.
96 files changed, +10,774 / -2,929 lines (excluding fork-specific README and TODO).
New Providers
Mistral Transformer (
packages/core/src/transformer/mistral.transformer.ts)packages/core/src/utils/mistral.util.ts) for transformation logic decouplingCodex Transformer (
packages/core/src/transformer/codex.transformer.ts)packages/server/src/routes/codex-auth.ts,packages/core/src/utils/codex-auth.ts)packages/cli/src/utils/codex-cli-auth.ts)Chrome On-Device Bridge — Gemini Nano (
packages/cli/src/utils/chrome-device-bridge.ts)packages/core/src/transformer/chrome-on-device.transformer.ts<tool_result>XML tag migration for consistent tool interactionArchitectural Refactoring
Modularized Shared Utilities
packages/core/src/utils/stream.ts): Reusable stream utilities extracted from individual transformerspackages/core/src/utils/thinking.ts): Ordered thinking sequence handlingpackages/core/src/utils/google.util.ts): Shared Gemini/Vertex utilitiespackages/core/src/utils/deepseek.util.ts): Reasoning replay logicpackages/core/src/utils/openai.util.ts): Common OpenAI-compatible API handlingpackages/core/src/utils/schema.ts): Tool schema sanitizationpackages/core/src/utils/cacheControl.ts): Cache header managementStreaming Improvements
ThinkingSequencerfor ordered Gemini streaming outputCodebase Localization
New Features
Custom Provider Model Discovery
packages/cli/src/utils/modelGet.ts: Utility for fetching available models from custom providersDocker Compose Deployment
packages/server/docker-compose.yml: Full containerized deployment with config mountingpackages/server/Dockerfile: Rebuilt to include UI package assets and multi-stage build.dockerignoreadded for optimized Docker contextConfiguration Improvements
Bug Fixes
server_tool_usein usage statsTesting & Documentation
Testing
packages/core/src/tests/gemini.parity.ts: Gemini parity testing suitepackages/core/src/tests/deepseek.reasoning.ts: DeepSeek reasoning validationpackages/cli/src/tests/ccr-anthropic-flow.test.ts: Anthropic flow integration testDocumentation
tasks/lessons.md: Hard-won integration knowledge (DeepSeek reasoning, Mistral thinking, Gemini streaming, Nano constraints)examples/README.md: Internationalized with dynamic preset examplesCLAUDE.md: Updated with Docker Compose instructions, knowledge base references, and core objectivesTest Plan
pnpm buildpasses locallycd packages/server && docker compose buildsucceeds