Conversation
- Refactor PreflightPredictor to track transitions between capabilities per application/session scope. - Introduce global fallback for scopes with insufficient data. - Add metrics for preflight effectiveness and expose them via API. - Implement SubscriptionRegistry for managing capability subscriptions with TTL. - Enhance Router to support memory and lifecycle reporting, and subscription management. - Add new EngineEvent variants for model eviction and preflight warming events.
- Introduced a new `stream` method in the `Provider` trait to allow providers to emit incremental text deltas during inference. - Added `StreamChunk` enum to represent different types of events in a streaming inference response, including `Started`, `Text`, `Completed`, and `Error`. - Implemented `invoke_stream` in the `DaemonClient` to handle streaming responses over SSE. - Updated the HTTP server to include a new `/v1/invoke/stream` endpoint for streaming inference output. - Enhanced the `Router` to support bearer token authentication for API routes. - Added metrics endpoint for Prometheus scraping. - Improved error handling and request correlation with `x-request-id` headers. - Updated dependencies in `Cargo.toml` for `reqwest` and `tokio`.
feat: implement per-scope Markov chain preflight prediction
- Refactor router to implement new locality preferences (`Prefer::Local`, `Prefer::Cloud`, `Prefer::Auto`) for improved routing decisions. - Introduce reasoning tier matching bonus to prioritize models based on reasoning effort. - Update tests to cover new routing scenarios, including confidential data handling and reasoning tier selection. - Add new examples demonstrating the engine's capabilities in code review, document AI, and meeting brief scenarios. - Implement structured error handling for failed routing attempts, providing detailed failure chains for better debugging. - Extend message structure to support multimodal inputs with image references.
…tion - Introduced `LocalImageGen` provider to support local image generation. - Enhanced `ProviderConfig` to include `diarize_args` for ASR speaker diarization. - Updated `Engine` to handle new provider and incorporate cost estimation for requests. - Implemented a quality gate for video outputs, ensuring integrity and motion checks. - Added example for orchestrating an explainer video with a hard quality gate. - Updated existing examples to include cost control and speaker diarization features.
- Removed the `route` method from the Router struct, simplifying the routing logic. - Updated tests to use `route_ranked` directly for model selection. - Refactored the explainer video example to utilize the `ExplainerVideo` struct for orchestration stages. - Introduced the `MeetingBrief` struct for handling meeting-related orchestration, encapsulating the run and preview logic. - Moved error serialization logic into the `EmbeddedEngine` implementation for better encapsulation. - Changed the model ID handling to use a dedicated `ModelId` struct for canonical ID operations, improving clarity and maintainability. - Updated the server implementation to encapsulate HTTP server logic within a `Server` struct, enhancing organization and readability. - Refactored middleware functions to be methods of the `AppState` struct, promoting better structure and encapsulation. - Improved error handling and response formatting across various endpoints.
- Introduced `ResponsesRequest` and `ResponsesResponse` types for managing multi-turn interactions. - Implemented `respond` method in the `Engine` to handle stateful conversation logic, including message history storage and retrieval. - Added conversation management methods: `store_conversation`, `conversation_messages`, and `delete_conversation`. - Updated `Engine` initialization to include a `conversations` storage mechanism. - Enhanced the `EmbeddedEngine` and `DaemonClient` to support the new Responses API. - Added corresponding HTTP routes and handlers in the server for managing responses. - Updated tests to validate the new functionality and ensure proper error handling for invalid input. - Introduced a new `LocalVideoGen` provider kind to support video generation capabilities.
BH3GEI
left a comment
BH3GEI
left a comment
There was a problem hiding this comment.
There's clearly a lot of work here, but this can't be reviewed or landed as one unit — and two of the bundled changes have security implications that need their own scrutiny.
1. This PR must be split (blocker). 14,235 lines across 41 files with 10+ orthogonal features and an empty description is unreviewable — no reviewer can hold the blast radius in their head, and a revert of one bad piece would revert nine good ones. Please split by feature (roughly: preflight/subscriptions, artifact sweeper, image-input plumbing, liter-llm provider, constant-time compare, etc.), each with its own description and tests. I'll review the splits promptly.
2. messages[].images bypasses the security.input_roots whitelist (security, blocker). check_input_path only validates req.input_file, but the image path resolution in ollama.rs (resolve_image_b64) and liter_llm.rs (image_to_url) reads arbitrary local filesystem paths straight from request JSON. Any caller can point images[] at files outside the configured roots — and on the liter-llm path those file contents are then uploaded to a cloud provider, i.e. a local-file-read primitive becomes data exfiltration. All image paths must go through the same input_roots validation before any read.
3. PreflightConfig::subscription_ttl_secs is never read. The config field is parsed and documented but nothing consumes it, so subscriptions are permanent and unbounded — exactly what the TTL was meant to prevent. Either wire it up or drop the field until the follow-up lands.
4. ArtifactSweeper deletes from the shared system temp by name prefix. Sweeping mofa_* files out of the world-writable shared temp dir will happily delete another tenant's (or another app's) files that happen to match. Sweep a mofa-owned subdirectory instead.
Smaller items: the liter-llm integration pulls a new external dependency chain — please pin exact versions and add a note on how it's vetted (supply-chain); and the constant_time_eq helper leaks length via early return — compare over the max length or hash both sides first.
Happy to look again once split — #2 should be its own PR and land first.
…en compare - Validate local messages[].images paths against security.input_roots before routing: backends read them from disk (ollama base64-encodes, liter-llm uploads the contents to a cloud provider), so an unchecked path was a local-file-read primitive. http(s)/data: URLs skip the check; input_file goes through the same shared helper. - Apply PreflightConfig::subscription_ttl_secs as the default TTL when a subscription omits one, so subscriptions no longer pin models warm forever; an explicit per-call TTL still wins. - Write and sweep artifacts in a mofa-owned subdirectory of the system temp dir (mofa_artifacts) instead of the shared temp dir itself: all six artifact-writing backends and the sweeper resolve through one helper, created best-effort, so sweeping a world-writable shared directory can never delete another tenant's mofa_* files. - Make bearer-token comparison independent of length too: hash both sides and compare fixed-width digests instead of early-returning on length mismatch. - Pin liter-llm exactly (=1.11.0) with a vetting note; upgrades are a deliberate reviewed step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Pushed
Verification: The split question from the review is intentionally not addressed by this commit — that decision stays with the maintainers. My recommendation stands: these security/runtime fixes should land first (ideally as their own PR), then the remainder can be split and reviewed piece by piece. |
…token verification
|
Verified Re-ran the full gate on the branch tip: That closes out every code-level finding from my review. The one open item remains the split (the PR is ~14k lines across 41 files); per my earlier comment, that decision stays with the maintainers. |
- Introduced a new application, mofa-explainer, that generates narrated, subtitled explainer videos from a given topic using the MoFA engine. - Implemented a command-line interface (CLI) with options for output file, duration, number of scenes, and voice selection. - Integrated FFmpeg for video composition, ensuring quality checks and graceful degradation when resources are unavailable. - Enhanced the mofa-kernel to clarify the inference request structure regarding cost ceilings. - Updated mofa-podcast documentation to reflect improvements in TTS setup and removed the need for a wrapper script on macOS. - Refactored mofa-podcast to utilize the engine's built-in system voice for TTS, simplifying the configuration process.
- Extend the OpenAI-compatible provider to support image generation via the `/images/generations` endpoint. - Introduce new request and response structures for image generation. - Implement the `invoke_image_gen` method to handle image generation requests and manage artifact storage. - Update the provider configuration to include a dialect for cloud video generation. - Add a new `mofa-studio` package with a browser-based UI for generating images and videos. - Implement a simple API for capabilities and generation, serving artifacts from a specified directory. - Enhance error handling and user feedback in the UI for generation requests.
No description provided.