Skip to content

README.md

Latest commit

 

History

History
166 lines (103 loc) · 6.14 KB

README.md

File metadata and controls

166 lines (103 loc) · 6.14 KB
Raw

ML-Scaler

A comprehensive declarative ML deployment management system for Kubernetes-based machine learning model for real time scaling and orchestration. ML-Scaler provides automated deployment, scaling, real time inference and lifecycle management for ML models on Kubernetes Service such as AKS, GKE, EKS etc levearging KEDA.

Features

  • Automated ML Model Deployment - Deploy ML models to Kubernetes with automated configuration
  • Dynamic Scaling - Auto-scale model deployments based on demand
  • Deployment Management - List, update, and delete model deployments via CLI
  • Database Persistence - Track deployment specs, compute instances, storage, and event sources
  • Azure DevOps Integration - Trigger CI/CD pipelines for model deployments
  • CLI Interface - Typer-based command-line tool for operations

Prerequisites

  • Python 3.10 or higher
  • Docker and Docker Compose
  • PostgreSQL database
  • Azure Kubernetes Service (AKS) cluster
  • Azure DevOps account (for pipeline integration)
  • Service Principal credentials such as Azure Service Principal

Quick Start Guide

Installation

Using Docker Compose (Recommended)

Using a Docker Compose file along with a local environment file and configuration YAML values will bring the CLI wrapper up and running.

# Clone the repository
git clone <repository-url>
cd <directory>

# Start services
docker-compose up -d

Local Installation

# Install dependencies
pip install -e .

# Set up environment variables (see Configuration)
cp env.example .env
# Edit .env with your configuration

Configuration

Security Setup

IMPORTANT: Never commit credentials to version control. Use environment variables for all sensitive data.

  1. Create a .env file from the example:

    cp .env.example .env

  2. Configure required environment variables in .env:

    • Database: DATABASE_URL
    • Azure DevOps: AZURE_DEVOPS_PAT
    • Azure Service Principal: TENANT_ID, CLIENT_ID, CLIENT_SECRET
    • Azure Databricks Configurations: DATABRICKS_HOST, DATABRICKS_TOKEN

  3. Update app/config.yaml with non-sensitive configuration:

    • Azure DevOps organization, project, pipeline, and branch name
    • AKS api_server & namespace
    • You can also handle the logging_level, db_config non-sensitive informations accordingly.

NOTE:----> You can set the configurations either in ".env" file or "app/config.yaml" accordingly based on sensitivity.

  1. For Production: Use Azure Key Vault or similar secret management service instead of .env files

CLI Interface

CLI commands offered by mlscaler:

mlscaler --help

[ABOVE COMMAND WILL GUIDE YOU HOW TO USE DEFINED CLI COMMANDS]

image

initialize

Initialize environment, check DB health, and ensure required tables are present.

image

add-resource

Add configuration for required instance resources (storage, compute, event source, deployment spec) to DB.

image

create-deployment --deployment-name <name>

Trigger deployment for a specific deployment name. Checks AKS for existing deployment first. Use --overwrite-deployment flag if deployment already exists. It has provision to check the pipeline status as well.

image image image

update-deployment --deployment-name <name>

Update configuration key-values for a deployment in DB. On user confirmation, triggers pipeline with updated configurations. On trigger, it has provision to check the pipeline status as well.

image

view-deployment --deployment-name <name>

View DB configurations and AKS deployment details for a specific deployment. image image

list-deployments

List all deployments available in AKS under the configured namespace.

image

check-deployment-status --deployment-name <name>

Check deployment status in AKS for a specific deployment. image

delete-deployment --deployment-name <name>

Delete deployment from AKS and DB configuration on user confirmation. image

Architecture

  • SQLAlchemy - ORM for database operations
  • PostgreSQL - Relational database for persistence
  • Typer - Modern CLI framework
  • Kubernetes Client - AKS cluster interaction
  • Azure DevOps API - CI/CD pipeline orchestration

Code Quality

This project follows Python best practices and PEP 8 guidelines. Future versions may include automated testing.

Contributing

Contributions are welcome! Please ensure:

  • Code follows project style guidelines
  • Documentation is updated as needed

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For issues, questions, or contributions, please open an issue in the repository.