MLX_Whisper_API

MLX Whisper API - Production Deployment Guide
Overview
This is a robust, self-discovering API interface for MLX Whisper that automatically adapts to library changes and provides comprehensive error handling.

Key Features
1. Automatic Service Discovery
Automatically discovers all available functions in the MLX Whisper library
Analyzes function signatures and documentation
Exposes discovered services through REST endpoints
No manual updates needed when library changes
2. Resilience Patterns
Circuit Breaker: Prevents cascading failures
Retry with Backoff: Automatic retry for transient failures
Connection Pooling: Efficient resource usage
Health Checks: Continuous monitoring
Graceful Degradation: Partial functionality during issues
3. Multiple Interfaces
REST API (FastAPI)
WebSocket for streaming
File upload support
Batch processing
4. Production Ready
Comprehensive error handling
Detailed logging
Metrics collection
Auto-scaling support
Docker deployment
Installation
bash
# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install fastapi uvicorn aiohttp numpy pydantic mlx-whisper

# For production
pip install gunicorn redis celery prometheus-client
Configuration Files
1. docker-compose.yml
yaml
version: '3.8'

services:
  mlx-whisper-api:
    build: .
    ports:
      - "8000:8000"
    environment:
      - LOG_LEVEL=INFO
      - MAX_WORKERS=4
      - MODEL_CACHE=/models
    volumes:
      - ./models:/models
      - ./logs:/logs
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./ssl:/etc/nginx/ssl
    depends_on:
      - mlx-whisper-api
    restart: unless-stopped

  redis:
    image: redis:alpine
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data
    restart: unless-stopped

volumes:
  redis-data:
2. Dockerfile
dockerfile
FROM python:3.11-slim

WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Copy requirements
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application
COPY mlx_whisper_api.py .
COPY mlx_whisper_client.py .

# Create directories
RUN mkdir -p /models /logs

# Run as non-root
RUN useradd -m -u 1000 api && chown -R api:api /app /models /logs
USER api

# Start server
CMD ["uvicorn", "mlx_whisper_api:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]
3. nginx.conf
nginx
events {
    worker_connections 1024;
}

http {
    upstream mlx_api {
        least_conn;
        server mlx-whisper-api:8000 max_fails=3 fail_timeout=30s;
    }

    server {
        listen 80;
        server_name api.example.com;
        
        # Redirect to HTTPS
        return 301 https://$server_name$request_uri;
    }

    server {
        listen 443 ssl http2;
        server_name api.example.com;

        ssl_certificate /etc/nginx/ssl/cert.pem;
        ssl_certificate_key /etc/nginx/ssl/key.pem;

        # Security headers
        add_header X-Content-Type-Options nosniff;
        add_header X-Frame-Options DENY;
        add_header X-XSS-Protection "1; mode=block";

        # Rate limiting
        limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
        limit_req zone=api burst=20 nodelay;

        location / {
            proxy_pass http://mlx_api;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            
            # Timeouts
            proxy_connect_timeout 30s;
            proxy_send_timeout 30s;
            proxy_read_timeout 30s;
        }

        location /ws {
            proxy_pass http://mlx_api;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
        }
    }
}
4. systemd service (mlx-whisper-api.service)
ini
[Unit]
Description=MLX Whisper API
After=network.target

[Service]
Type=notify
User=api
WorkingDirectory=/opt/mlx-whisper-api
Environment="PATH=/opt/mlx-whisper-api/venv/bin"
ExecStart=/opt/mlx-whisper-api/venv/bin/gunicorn \
    -w 4 \
    -k uvicorn.workers.UvicornWorker \
    --bind 0.0.0.0:8000 \
    --timeout 60 \
    --access-logfile /var/log/mlx-whisper/access.log \
    --error-logfile /var/log/mlx-whisper/error.log \
    mlx_whisper_api:app

Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
5. requirements.txt
txt
fastapi==0.104.1
uvicorn[standard]==0.24.0
aiohttp==3.9.0
numpy==1.26.0
pydantic==2.5.0
mlx-whisper
gunicorn==21.2.0
prometheus-client==0.19.0
redis==5.0.1
python-multipart==0.0.6
Usage Examples
Basic Usage
python
import asyncio
from mlx_whisper_client import MLXWhisperSync

# Synchronous client
client = MLXWhisperSync("http://localhost:8000")

# Transcribe audio
text = client.transcribe(audio_array, model="base", language="en")
print(text)
Advanced Usage with Failover
python
from mlx_whisper_client import MLXWhisper

async def main():
    # Multiple endpoints for failover
    whisper = MLXWhisper([
        "http://api1.example.com",
        "http://api2.example.com",
        "http://api3.example.com"
    ])
    
    # Automatic failover if one endpoint fails
    text = await whisper.transcribe(audio_array)
    print(text)

asyncio.run(main())
Streaming Transcription
python
async def audio_generator():
    # Generate audio chunks
    while True:
        chunk = get_audio_chunk()  # Your audio source
        yield chunk

async def stream_transcribe():
    async with WhisperClient() as client:
        async for result in client.stream_transcribe(audio_generator()):
            print(f"Transcription: {result['text']}")
Monitoring
Health Check Endpoint
bash
curl http://localhost:8000/health
Response:

json
{
  "status": "healthy",
  "timestamp": "2024-01-20T10:30:00Z",
  "services": {
    "transcribe": {
      "available": true,
      "call_count": 1523,
      "error_count": 2,
      "avg_response_time": 0.342
    }
  }
}
Prometheus Metrics
Add to mlx_whisper_api.py:

python
from prometheus_client import Counter, Histogram, generate_latest

# Metrics
request_count = Counter('mlx_api_requests_total', 'Total requests')
request_duration = Histogram('mlx_api_request_duration_seconds', 'Request duration')

@app.get("/metrics")
async def metrics():
    return Response(generate_latest(), media_type="text/plain")
Security Best Practices
API Key Authentication
python
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials

security = HTTPBearer()

@app.post("/transcribe")
async def transcribe(
    request: TranscriptionRequest,
    credentials: HTTPAuthorizationCredentials = Depends(security)
):
    # Validate API key
    if not validate_api_key(credentials.credentials):
        raise HTTPException(status_code=401, detail="Invalid API key")
Rate Limiting
python
from slowapi import Limiter
from slowapi.util import get_remote_address

limiter = Limiter(key_func=get_remote_address)
app.state.limiter = limiter

@app.post("/transcribe")
@limiter.limit("10/minute")
async def transcribe(request: TranscriptionRequest):
    # Process request
Input Validation
Max file size limits
Audio format validation
Request size limits
Scaling Strategies
Horizontal Scaling
Deploy multiple API instances
Use load balancer (nginx/HAProxy)
Shared model cache (Redis/NFS)
Vertical Scaling
Increase worker threads
Optimize model loading
Use GPU acceleration
Caching
Cache transcription results
Cache model files
Use CDN for static assets
Troubleshooting
Common Issues
Model Loading Fails
Check internet connection
Verify model cache permissions
Check available disk space
High Memory Usage
Limit concurrent requests
Use smaller models
Enable model quantization
Slow Response Times
Check CPU/GPU utilization
Enable response caching
Use connection pooling
Debug Mode
bash
# Enable debug logging
LOG_LEVEL=DEBUG uvicorn mlx_whisper_api:app --reload
Conclusion
This architecture provides:

✅ Automatic adaptation to library changes
✅ Comprehensive error handling
✅ Production-ready deployment
✅ High availability with failover
✅ Easy scaling and monitoring
The API will continue working even as MLX Whisper evolves, automatically discovering and exposing new functionality without code changes.