Skip to content

RFC: Publish dynamo-protocols and dynamo-parsers as standalone crates #7563

@ishandhanani

Description

@ishandhanani

Summary

Publish dynamo-protocols and dynamo-parsers as standalone, reusable crates on crates.io so that the broader Rust inference ecosystem can leverage battle-tested protocol types and streaming parsers without pulling in all of Dynamo.

Architecture

dynamo-protocols depends on upstream async-openai v0.34 for base OpenAI types and layers inference-serving extensions on top.

dynamo-protocols
├── async-openai v0.34 (upstream, re-exported)
│   ├── chat types: CompletionUsage, FunctionCall, Role, Prompt, ...
│   ├── completion types: CreateCompletionResponse
│   ├── embedding types: full re-export
│   ├── responses types: full re-export
│   └── image types: full re-export
├── inference-serving extensions (locally defined)
│   ├── ChatCompletionResponseMessage (multimodal content + reasoning_content)
│   ├── ChatChoiceStream (stop_reason from backend)
│   ├── CreateChatCompletionRequest (mm_processor_kwargs, extended stream_options)
│   ├── ReasoningContent, StopReason, ChatCompletionMessageContent
│   ├── VideoUrl, AudioUrl (multimodal input)
│   └── ImageUrl (url::Url + uuid tracking)
└── anthropic/ (full Anthropic Messages API)
    ├── AnthropicCreateMessageRequest, AnthropicMessageResponse
    ├── AnthropicStreamEvent, CacheControl, SystemContent
    └── Token counting, error types

PR Stack

PR 1: Remove nvext from base types + extract Anthropic types

#7564 | idhanani/remove-nvext-from-base-types | Open

PR 2: Rename to dynamo-protocols

#7565 | idhanani/rename-to-dynamo-protocols | Open

PR 3: Switch to upstream async-openai + remove fork

#7625 | idhanani/protocols-cleanup | Open

  • Add async-openai = "0.34" (types-only features)
  • Re-export upstream types for chat, completion, embedding, responses, images
  • Keep ~15 locally-defined types for inference-serving extensions
  • Remove utoipa/ToSchema
  • Delete entire forked HTTP client, API handlers, and unused type modules
  • -16,811 lines total across the PR stack

Remaining work

  • cargo package validation and publish to crates.io

Types retained locally

Type Extension Rationale
ChatCompletionResponseMessage content: ChatCompletionMessageContent, reasoning_content Multimodal response content + reasoning output
ChatCompletionStreamResponseDelta Same as above Streaming variant
ChatChoice / ChatChoiceStream stop_reason: Option<StopReason> Backend-reported stop condition
CreateChatCompletionRequest mm_processor_kwargs vLLM multimodal processor config
ChatCompletionStreamOptions continuous_usage_stats Per-chunk usage reporting
ChatCompletionRequestAssistantMessage reasoning_content: Option<ReasoningContent> KV cache-correct reasoning
ChatCompletionRequestUserMessageContentPart VideoUrl, AudioUrl variants Multimodal input
ImageUrl url::Url, uuid field Strongly-typed URLs, asset tracking
CreateCompletionRequest prompt_embeds, echo validation Pre-computed embeddings
All anthropic/ types Fully custom Anthropic Messages API

Usage

[dependencies]
dynamo-protocols = "1.0"
dynamo-parsers = "1.0"
use dynamo_protocols::types::{
    CreateChatCompletionRequest,
    CreateChatCompletionStreamResponse,
    CompletionUsage,
    ReasoningContent,
};
use dynamo_protocols::types::responses::{CreateResponse, Response};
use dynamo_protocols::types::anthropic::{
    AnthropicCreateMessageRequest,
    AnthropicMessageResponse,
    CacheControl,
};

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions