Computer Vision YOLOv8 Tracking API Scaffold

A production-oriented scaffold that takes you from raw footage to a live HTTP API for object detection and multi-object tracking.

What This Project Does - In Plain English

Imagine you hand someone thousands of photos of cars, people, or whatever objects matter to your business, and you say "learn what these things look like." After enough examples, that person can look at a brand new photo and immediately point out every car, every person, and draw a box around each one. They can also watch a video and say "that car in frame 1 is the same car in frame 50 - its ID is 7." This project teaches a computer to do exactly that, and then wraps it in a web service so any other program can ask "what's in this image?" and get an instant, structured answer back.

The project has three jobs that happen in sequence:

Job 1 - Teaching the AI (Training). You collect photos or video clips of the objects you care about and draw labeled boxes around them. The AI studies those examples over many passes until it learns the patterns - the shapes, edges, colors, and textures - that distinguish a car from a background wall. This is called training, and it produces a small file called a model weight file that encodes everything the AI learned.

Job 2 - Recognizing objects in new images or video (Inference). Once trained, the model can look at a photo it has never seen before and predict where every object is and what class it belongs to. For video, it goes one step further: it connects detections across frames so that the same physical object gets the same ID number throughout the clip. That "follow this specific object" ability is called multi-object tracking.

Job 3 - Serving results over the internet (API). To make all this useful to other software, the project wraps the detection and tracking logic in a small web server. Any program can send an HTTP request saying "here is an image path, tell me what you see," and it gets back a clean JSON list of objects with their coordinates, class names, and confidence scores.

Note

You do not need to understand neural networks to use this project. Once trained, the API behaves like any other web service: you send a request, you get structured data back.

System Snapshot

_Area	_{What it contains}	_{Why it matters}
_Training	_{YOLOv8 config and launch script}	_{Keeps model training reproducible and separate from API logic}
_Tracking	_{ByteTrack config and tracking service}	_{Adds identity continuity across frames instead of per-frame detections}
_Serving	_{FastAPI routers, schemas, and services}	_{Provides a stable HTTP interface for downstream consumers}
_Deployment	_{Docker assets and environment-driven settings}	_{Makes it easier to move from local development to containerized runs}
_Validation	_{Scaffold validator and unit tests}	_{Catches missing files or broken structure early}

flowchart LR
    A[Raw images and video\ndata/raw] --> B[Annotations and split prep\ndata/annotations]
    B --> C[YOLOv8 training\nscripts/train_model.py]
    C --> D[Trained weights\nmodels/best.pt]
    D --> E[Model export\nscripts/export_model.py]
    D --> F[Detection API\nPOST /api/v1/detect]
    D --> G[Tracking API\nPOST /api/v1/track]
    H[ByteTrack config\nconfigs/bytetrack.yaml] --> G
    E --> I[Optimized artifact\nmodels/best.onnx]
    I --> J[Benchmarking and deployment decisions]

Figure 1 - End-to-end pipeline from raw data to a served API. Training and export flow left to right. The detection and tracking APIs both consume the trained weights, while tracking additionally relies on the ByteTrack configuration to associate detections across frames.

Technology Stack

The stack is small by design. Every dependency earns its place because computer vision projects can accumulate bloat quickly, and bloat makes deployments fragile.

_Technology	_{Version range}	_{Used for}	_{Why this choice}
_Python	_3.11+	_{Primary language}	_{Keeps training scripts, inference code, and service code in one ecosystem}
_FastAPI	_0.136.x	_{HTTP API framework}	_{Automatic request validation, OpenAPI docs, and clean routing}
_Uvicorn	_0.35.x	_{ASGI server}	_{Runs the FastAPI app locally and in containers}
_{Pydantic v2}	_2.11.x	_{Request and response validation}	_{Rejects malformed inference payloads before expensive model work starts}
_{Ultralytics YOLOv8}	_8.3.x	_{Detection, tracking, export}	_{Supplies model loading, prediction, ByteTrack integration, and ONNX export APIs}
_PyTorch	_2.3.x	_{Training and default inference backend}	_{Powers model training and weight artifacts}
_OpenCV	_4.10.x	_{Image and video operations}	_{Supports pre-processing and frame handling outside pure model inference}
_NumPy	_1.26+	_{Numerical array work}	_{Underlies tensor operations and image preparation}
_PyYAML	_6.x	_{Config file loading}	_{Allows training and tracker config to live outside code as editable files}

Why the stack is split this way

Ultralytics handles model-specific behavior. FastAPI handles request transport and validation. That split matters because model code changes often during a project, while your public API contract should change slowly. It also lets you benchmark or swap export targets without touching the HTTP layer, and it lets you update the API response format without retraining the model.

Architecture

The application code is organized around a standard service-oriented FastAPI structure. Routes translate HTTP requests into typed objects and turn domain errors into appropriate HTTP responses. Schemas define what requests and responses look like. Services hold the inference logic so model code is never mixed directly into router functions. Configuration is centralized so environment-specific choices can change without touching application logic.

flowchart TD
    Client[Client application] --> Router[FastAPI routers\napp/api/routes/]
    Router --> Schema[Pydantic schemas\napp/schemas/inference.py]
    Router --> Service[Inference services\napp/services/inference.py]
    Service --> Config[Settings dataclass\napp/core/config.py]
    Service --> YOLO[Ultralytics YOLO\nmodel.predict / model.track]
    Service --> Tracker[ByteTrack config\nconfigs/bytetrack.yaml]
    YOLO --> Serializer[Box serializer\n_serialize_detections / _serialize_tracks]
    Tracker --> Serializer
    Serializer --> Response[Response schemas\nDetectionResponse / TrackResponse]
    Response --> Client

Figure 2 - Internal request flow. Each box is a real module or function in this repository. The serializer converts raw Ultralytics box objects into stable Pydantic response models so the API contract is independent of internal Ultralytics data structures.

Runtime Request Flow

_Step	_Component	_{What happens}	_{Why it exists}
₁	_Client	_{Sends image or video path in JSON}	_{Establishes a machine-readable integration boundary}
₂	_Router	_{Accepts the request on /api/v1/detect or /api/v1/track}	_{Keeps HTTP concerns out of the service layer}
₃	_Schema	_{Pydantic validates the payload}	_{Rejects malformed inputs before expensive model work starts}
₄	_Service	_{Loads model path and thresholds from settings}	_{Centralizes inference behavior}
₅	_Ultralytics	_{Runs model.predict or model.track}	_{Performs the actual model execution}
₆	_Serializer	_{Converts boxes into API response items}	_{Produces stable JSON instead of framework-specific objects}
₇	_Response	_{Returns detections or tracked objects}	_{Gives downstream systems a clean, typed result}

Repository Layout

_Path	_Role	_{Typical contents}
_app/	_{Service implementation}	_{FastAPI app, routers, schemas, services, utilities}
_configs/	_{Externalized runtime and training settings}	_{Dataset YAML, training YAML, ByteTrack YAML}
_data/	_{Local working data directories}	_{Raw inputs, annotations, processed splits}
_docker/	_{Container deployment assets}	_{Dockerfile and compose file}
_docs/	_{Supporting design and workflow docs}	_{Architecture, dataset workflow, deployment notes}
_models/	_{Trained and exported artifacts}	_{best.pt, best.onnx}
_outputs/	_{Generated experiment results}	_{Training runs, exports, benchmarks}
_scripts/	_{Operational scripts}	_{Validation, training, export, benchmark}
_tests/	_{Automated checks}	_{Scaffold and structural tests}

.
|-- app/
|   |-- api/routes/        <- HTTP endpoint handlers
|   |-- core/              <- Settings dataclass, env loading
|   |-- schemas/           <- Pydantic request and response models
|   |-- services/          <- Inference and tracking business logic
|   `-- utils/             <- Shared video/image helpers
|-- configs/               <- dataset.yaml, train.yaml, bytetrack.yaml
|-- data/                  <- raw/, annotations/, processed/
|-- docker/                <- Dockerfile, docker-compose.yml
|-- docs/                  <- architecture.md, dataset-workflow.md, deployment.md
|-- models/                <- model weight artifacts (not committed)
|-- outputs/               <- training runs, benchmark results
|-- scripts/               <- validate_scaffold.py, train_model.py, export_model.py
`-- tests/                 <- test_scaffold.py

Quick Start

1. Create a virtual environment and install dependencies

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

2. Copy environment defaults

cp .env.example .env

3. Validate the scaffold

make validate

4. Start the API

uvicorn app.main:app --reload

5. Run tests

make test

Tip

Run make validate before debugging any import or runtime problems. The validator checks that every expected file is present and that the FastAPI entrypoint in pyproject.toml matches app.main:app.

Configuration

Configuration lives in two places: environment variables and YAML files. Environment variables control runtime behavior such as model paths and numeric thresholds. YAML files describe dataset layout, training job parameters, and tracking behavior. This separation matters because environment-specific values (which machine this runs on, which GPU) should not be mixed with experiment-level values (which model variant, how many epochs).

Environment variables

_Variable	_Default	_{What it controls}	_{When you would change it}
_{MODEL_PATH}	_{models/best.pt}	_{Main PyTorch weight file}	_{Point the API to a newly trained model}
_{EXPORT_MODEL_PATH}	_{models/best.onnx}	_{ONNX export destination}	_{Store optimized artifact in a specific location}
_{TRACKER_CONFIG}	_{configs/bytetrack.yaml}	_{ByteTrack YAML file path}	_{Tune thresholds or swap tracker configurations}
_DEVICE	_cpu	_{Runtime device hint}	_{Set to cuda:0 for GPU inference}
_{CONFIDENCE_THRESHOLD}	_0.25	_{Minimum box confidence}	_{Tune precision vs recall tradeoff}
_{IOU_THRESHOLD}	_0.45	_{NMS overlap threshold}	_{Reduce duplicate boxes on dense scenes}
_{API_HOST}	_0.0.0.0	_{Server bind host}	_{Useful in container and cloud environments}
_{API_PORT}	₈₀₀₀	_{Server bind port}	_{Change the listening port without editing code}

YAML configuration files

_File	_Purpose	_{Key values}
_{configs/dataset.yaml}	_{Defines YOLO dataset root, split folders, and class map}	_{Root: data/processed, one placeholder class}
_{configs/train.yaml}	_{Defines training job: model variant, epochs, image size, batch}	_{yolo8n.pt, 100 epochs, imgsz 640, batch 16}
_{configs/bytetrack.yaml}	_{Defines ByteTrack thresholds and frame buffer}	_{High threshold 0.5, low 0.1, buffer 30 frames}

Important

CONFIDENCE_THRESHOLD and IOU_THRESHOLD apply as defaults when a request does not specify its own value. Per-request values passed in the JSON body override the environment defaults inside DetectionService and TrackingService.

API Surface

The FastAPI app is created in app/main.py and mounts two routers: a health router at root and an inference router under /api/v1. The inference routes instantiate service objects per request and convert ValueError exceptions to HTTP 400 responses.

_Method	_Path	_{Request model}	_{Response model}	_{What it does}
_GET	_/health	_None	_{{"status": "ok"}}	_{Confirms the service is up}
_POST	_{/api/v1/detect}	_{DetectionRequest}	_{DetectionResponse}	_{Object detection on a single image}
_POST	_{/api/v1/track}	_TrackRequest	_{TrackResponse}	_{Multi-object tracking on a video source}

Detection request example

curl -X POST http://127.0.0.1:8000/api/v1/detect \
  -H "Content-Type: application/json" \
  -d '{"source": "data/raw/frame.jpg", "confidence": 0.35}'

Tracking request example

curl -X POST http://127.0.0.1:8000/api/v1/track \
  -H "Content-Type: application/json" \
  -d '{
    "source": "data/raw/clip.mp4",
    "confidence": 0.35,
    "tracker": "configs/bytetrack.yaml",
    "persist": true
  }'

Response field reference

_Field	_{Present in}	_Type	_Meaning
_{model_path}	_{Both responses}	_str	_{Model file used for the request}
_source	_{Both responses}	_str	_{Original source string passed in the request}
_detections	_{Detection response}	_list	_{Detected objects with class, confidence, and bounding box}
_tracker	_{Tracking response}	_str	_{Tracker config path used during the tracking run}
_tracks	_{Tracking response}	_list	_{Tracked objects including optional track_id}
_{bbox_xyxy}	_{Detection and track items}	_list[float]	_{Bounding box as [x1, y1, x2, y2] pixel coordinates}
_{class_id}	_{Detection and track items}	_int	_{Numeric class index matching your dataset label map}
_{class_name}	_{Detection and track items}	_str	_{Human-readable class name from model's name map}
_confidence	_{Detection and track items}	_float	_{Model's certainty score for this detection, 0.0 to 1.0}
_{track_id}	_{Track items only}	_{int or null}	_{Persistent identity integer assigned by ByteTrack}

Caution

The current service accepts a raw string source and passes it to Ultralytics. Before exposing this API outside a trusted network, validate the allowed schemes, restrict file system access to expected directories, and decide whether remote URLs are permitted.

Workflow From Data To Deployment

Every stage below exists because a downstream stage depends on its quality. Shortcuts at any step create failure modes that appear much later and are hard to diagnose.

1. Collect data

Store raw images or video clips under data/raw/. Capture representative scenes - different lighting conditions, varying object densities, the angles and distances you expect in the real deployment environment. Coverage breadth here directly limits how well the model generalizes later.

2. Annotate and prepare splits

Store YOLO-format labels under data/annotations/labels/ and keep class definitions aligned with configs/dataset.yaml. Prepare processed splits following a 70 / 20 / 10 train, validation, and test split under data/processed/. Reviewing label quality before each training run matters because mislabeled boxes and class imbalance are not visible as training errors - they appear later as unexpectedly poor precision or recall on specific classes.

3. Train the detector

python scripts/train_model.py

This loads configs/train.yaml, constructs a YOLO model from config["model"], and calls model.train(**config) through the Ultralytics API. Weights, logs, and validation curves are saved to outputs/train/.

4. Export for deployment

python scripts/export_model.py

This loads the trained model from MODEL_PATH and calls model.export(format="onnx", dynamic=True, simplify=True). ONNX artifacts are more portable than PyTorch weights and often faster on CPU-focused serving environments.

5. Benchmark in the target environment

python scripts/benchmark_model.py

Currently writes a placeholder to outputs/benchmark.txt. Replace it with real latency and throughput measurements that reflect your actual hardware, input resolution, and batch size before making deployment decisions.

6. Serve through the API

uvicorn app.main:app --host 0.0.0.0 --port 8000

Once weights are in place and environment variables are configured, the API is ready to serve detection and tracking requests.

Development Commands

_Command	_{What it does}	_{When to use it}
_{make validate}	_{Runs scripts/validate_scaffold.py}	_{After structural changes or before onboarding}
_{make compile}	_{Compiles Python sources under app, scripts, tests}	_{Quick syntax check without running tests}
_{make test}	_{Runs unit tests discovered under tests/}	_{Before merging or after changing behavior}
_{uvicorn app.main:app --reload}	_{Starts the API in hot-reload mode}	_{Local development}
_{python scripts/train_model.py}	_{Launches a YOLO training run from YAML config}	_{Model training iteration}
_{python scripts/export_model.py}	_{Exports the configured model to ONNX}	_{Deployment preparation}
_{python scripts/benchmark_model.py}	_{Writes the current benchmark placeholder}	_{Benchmark scaffolding}

Technical Deep Dive

This section explains the algorithms, data transformations, and mathematical operations that drive every stage of the pipeline. It assumes you are comfortable reading code and want to understand what is actually happening inside each component.

YOLOv8 Model Architecture

YOLOv8 is a single-stage anchor-free object detector. A single forward pass through the network produces every bounding box and class prediction simultaneously, unlike two-stage detectors that first propose regions and then classify them. This design is what makes YOLOv8 fast enough for near-real-time use.

The network is divided into three structural sections:

flowchart LR
    Input[Input image\n640x640x3] --> Backbone
    Backbone[CSP Darknet backbone\nfeature extraction] --> Neck
    Neck[PANet neck\nmulti-scale feature fusion] --> Head
    Head[Decoupled detection head\nbox regression + classification] --> Output[Prediction tensors\nN x 84 per scale]

Figure 3 - YOLOv8 forward pass. The backbone extracts semantic features at multiple resolutions. The neck merges those resolutions to improve detection of both small and large objects. The head converts the fused features into box predictions and class probabilities.

Backbone - C2f and Cross-Stage Partial connections. The backbone uses a modified CSPDarknet structure with C2f (Cross Stage Partial with 2 feature branches) blocks. Each C2f block splits input feature maps into two paths, processes one through a series of bottleneck layers, and then concatenates the paths. This preserves gradient flow across deep stacks while keeping parameter count manageable. The output is a set of feature maps at three spatial scales, typically at strides of 8, 16, and 32 pixels relative to the input.

Neck - PANet multi-scale feature pyramid. The neck uses a Path Aggregation Network structure. It takes feature maps from the backbone at three scales and merges them using both top-down upsampling (to pass high-level semantic context to finer scales) and bottom-up downsampling (to pass precise spatial information to coarser scales). The result is that each prediction scale has access to both semantic richness and spatial precision, which is what allows the model to detect both large and small objects in the same image.

Head - Decoupled box and class prediction. Unlike earlier YOLO versions that predicted box coordinates and class scores through the same convolutional path, YOLOv8 uses a decoupled head: a separate branch for bounding box regression and a separate branch for class scores. This matters because these two tasks have different optimal feature representations.

YOLOv8 is also anchor-free: it predicts the center point and the size of a box directly, rather than predicting adjustments relative to a set of fixed reference boxes. This removes the need to design anchor configurations for each dataset.

Loss Functions Used During Training

Training a YOLO model involves minimizing a combined loss over all predictions in a batch. Three loss components are summed with scaling factors to produce the total loss.

Box regression loss. YOLOv8 uses CIoU (Complete Intersection over Union) loss for bounding box regression.

$$\mathcal{L}_{box} = 1 - \text{CIoU}(B_{pred}, B_{gt})$$

where CIoU extends the basic IoU (overlap ratio between predicted and ground-truth box) by also penalizing center distance and aspect ratio inconsistency:

$$\text{CIoU} = \text{IoU} - \frac{\rho^2(b, b^{gt})}{c^2} - \alpha v$$

$\rho^2(b, b^{gt})$ is the squared Euclidean distance between the centers of the predicted and ground-truth boxes
$c$ is the diagonal length of the smallest enclosing box
$v = \frac{4}{\pi^2}\left(\arctan\frac{w^{gt}}{h^{gt}} - \arctan\frac{w}{h}\right)^2$ measures aspect ratio consistency
$\alpha = \frac{v}{(1 - \text{IoU}) + v}$ is the trade-off coefficient

CIoU converges faster than plain IoU loss and avoids pathological behavior when boxes do not overlap.

Classification loss. Binary Cross-Entropy loss is applied to each class independently (one class per logit), not a softmax over all classes:

$$\mathcal{L}_{cls} = -\sum_{c} \left[ y_c \log(\hat{p}_c) + (1 - y_c) \log(1 - \hat{p}_c) \right]$$

This formulation allows a prediction to belong to multiple classes simultaneously, which matters for datasets where objects overlap semantically.

Distribution Focal Loss (DFL) for box refinement. YOLOv8 uses DFL to model the uncertainty of edge location predictions. Instead of predicting a single scalar for each box edge, the model outputs a probability distribution over a discrete set of locations and the expected value of that distribution is used as the final coordinate:

$$\hat{x} = \sum_{i=0}^{n} p_i \cdot i, \quad \text{with} \quad \sum_{i=0}^{n} p_i = 1$$

DFL is more stable on ambiguous or occluded boxes where a single precise coordinate is genuinely uncertain.

Total training loss:

$$\mathcal{L}_{total} = \lambda_{box} \cdot \mathcal{L}_{box} + \lambda_{cls} \cdot \mathcal{L}_{cls} + \lambda_{dfl} \cdot \mathcal{L}_{dfl}$$

The $\lambda$ weights are set by Ultralytics defaults and can be overridden in the training config.

Task-Aligned Assigner

Before the loss can be computed, the model must decide which ground-truth box each predicted box is responsible for. YOLOv8 uses a Task-Aligned Assigner (TAL) rather than fixed IoU-based matching. TAL scores each anchor-ground-truth pair using:

$$s = p^\alpha \cdot \text{IoU}(B_{pred}, B_{gt})^\beta$$

where $p$ is the classification score and $\alpha$, $\beta$ are hyperparameters that control how much classification quality vs. localization quality influence the assignment. The top-$k$ anchors per ground-truth object by this score are selected as positives. This alignment between the training signal and the inference score improves consistency between the loss and what the model learns to optimize.

Non-Maximum Suppression

After the forward pass, the model produces many overlapping box predictions. NMS filters them to a clean set:

Sort all predictions by confidence score descending.
Take the highest-confidence prediction as a confirmed detection.
Remove any other prediction whose IoU with the confirmed box exceeds the IOU_THRESHOLD.
Repeat with the next highest remaining prediction.

The IOU_THRESHOLD (default 0.45) controls how aggressively overlapping boxes are suppressed. Lower values remove more duplicates but may accidentally suppress nearby objects of the same class.

ByteTrack Multi-Object Tracking

ByteTrack is the tracking algorithm configured in configs/bytetrack.yaml. It solves the problem of assigning consistent IDs to detections across video frames. Its key innovation over simpler trackers is that it processes all detection boxes, not just high-confidence ones, which prevents small, occluded, or partially visible objects from losing their track ID.

ByteTrack operates in two stages per frame:

Stage 1 - High-confidence association. All detections above track_high_thresh (default 0.5) are matched to existing tracks using IoU between each detection box and the Kalman-predicted position of each track. The matching is solved as a linear assignment problem using the Hungarian algorithm to find the minimum cost assignment.

Stage 2 - Low-confidence rescue. Any track that was not matched in Stage 1 is given a second chance: low-confidence detections (between track_low_thresh and track_high_thresh) are tried against the unmatched tracks. This is the core insight of BYTE: a low-confidence detection on a real object is more useful than discarding it entirely.

Kalman Filter for Motion Prediction

Each active track maintains a Kalman filter that predicts where the object will be in the next frame. The state vector is:

$$\mathbf{x} = [x_{center},\ y_{center},\ \text{aspect ratio},\ \text{height},\ \dot{x},\ \dot{y},\ \dot{a},\ \dot{h}]^T$$

The prediction step extrapolates the object's position using its last known velocity:

$$\mathbf{x}_{k|k-1} = \mathbf{F}\mathbf{x}_{k-1}$$ $$\mathbf{P}_{k|k-1} = \mathbf{F}\mathbf{P}_{k-1}\mathbf{F}^T + \mathbf{Q}$$

where $\mathbf{F}$ is the state transition matrix encoding constant-velocity motion, $\mathbf{P}$ is the state covariance, and $\mathbf{Q}$ is the process noise. The update step corrects the prediction using the matched detection:

$$\mathbf{K}_k = \mathbf{P}_{k|k-1}\mathbf{H}^T(\mathbf{H}\mathbf{P}_{k|k-1}\mathbf{H}^T + \mathbf{R})^{-1}$$ $$\mathbf{x}_k = \mathbf{x}_{k|k-1} + \mathbf{K}_k(\mathbf{z}_k - \mathbf{H}\mathbf{x}_{k|k-1})$$

where $\mathbf{H}$ extracts observable state, $\mathbf{R}$ is measurement noise, and $\mathbf{K}_k$ is the Kalman gain that weights the prediction vs. the new measurement. Tracks that are not matched for track_buffer (default 30) consecutive frames are removed.

IoU-Based Matching Cost Matrix

The Hungarian algorithm operates on a cost matrix $C$ where each entry $C_{ij}$ is the matching cost between detection $i$ and track $j$. ByteTrack uses:

$$C_{ij} = 1 - \text{IoU}(B^{detect}_i,\ B^{track}_j)$$

A perfect overlap gives cost 0. No overlap gives cost 1. Pairs whose cost exceeds 1 - match_thresh (default match_thresh = 0.8) are not allowed, preventing nonsensical long-range associations.

ONNX Export and Inference Optimization

The export script calls model.export(format="onnx", dynamic=True, simplify=True), which performs three operations:

Tracing. PyTorch traces the model forward pass with a dummy input to record the computation graph as a sequence of ONNX operators.
Simplification. onnxsim folds constant expressions, removes redundant operations, and merges compatible layers. This reduces runtime overhead without changing model accuracy.
Dynamic axes. Setting dynamic=True allows the exported model to accept variable batch sizes and input spatial dimensions, which is important for deployment systems that process different image sizes.

ONNX benefits CPU-focused deployments because ONNX Runtime applies backend-specific kernel optimizations that PyTorch's default CPU path does not always apply. The model can also be further compiled to TensorRT for NVIDIA GPU targets or OpenVINO for Intel hardware using the same ONNX artifact as the starting point.

Tip

For latency-sensitive deployments, profile the ONNX model with onnxruntime.InferenceSession before reaching for TensorRT or quantization. In many CPU workloads, the simplified ONNX graph already outperforms eager PyTorch without additional optimization passes.

Deployment Notes

_{Deployment concern}	_{Current scaffold behavior}	_{What you will likely add later}
_{Base image}	_{python:3.12-slim}	_{GPU-specific base images for CUDA workloads}
_{Process model}	_{Single Uvicorn process, port 8000}	_{Process management, autoscaling, ASGI worker tuning}
_{Model delivery}	_{Expects model files at configured paths}	_{Artifact download, volume mounts, or model registry integration}
_Optimization	_{ONNX export path is included}	_{TensorRT, OpenVINO, or quantized runtimes}
_{Observability}	_{No logging or metrics layer}	_{Structured logs, tracing, and latency dashboards}
_{Input validation}	_{Source string passed directly to Ultralytics}	_{Scheme allowlist, path sandboxing, size limits}

Warning

A container that starts successfully is not proof that inference is production-ready. Validate model artifact presence, warm-start behavior, input validation rules, peak memory use, and latency under expected concurrent traffic before going live.

Customization Checklist

Replace the placeholder class map in configs/dataset.yaml
Add real training, validation, and test image splits under data/processed/
Supply a trained weight file at models/best.pt or update MODEL_PATH
Validate ByteTrack thresholds in configs/bytetrack.yaml against real video clips
Decide whether API callers may use file paths, URLs, or both as source
Add authentication, authorization, and request-size limits for public exposure
Replace the benchmark placeholder with hardware-specific latency measurements
Add structured logging, health metrics, and deployment environment checks

Tips And Operating Notes

_Topic	_{Practical guidance}
_{Model artifacts}	_{Keep training outputs and deployed weights separate so you always know which file is serving traffic}
_{Confidence threshold}	_{Tune using your real validation set, not an initial guess - this single value has large precision vs recall consequences}
_{Tracking stability}	_{Evaluate tracker thresholds on full video clips; ID switching is a temporal problem invisible in frame-by-frame review}
_{Dataset health}	_{Check class balance, empty labels, and duplicates before every serious training run}
_{API contract}	_{Keep bbox_xyxy coordinate ordering and field names stable across model updates}
_{Cold start}	_{Consider preloading the model at FastAPI startup if you cannot tolerate first-request latency spikes}

Additional implementation tips

Keep models/ out of source control for large artifacts and manage weights through an artifact store or deployment pipeline.
Add request logging with model version and threshold metadata so debugging production predictions is possible later.
Consider preloading the model at startup (lifespan event in FastAPI) if cold-start latency matters more than memory usage.
Add stricter source validation before exposing the API outside a trusted network.
Create benchmark scenarios that reflect both image detection and long-running video tracking because the resource profile differs significantly between the two modes.

Notes

The scaffold intentionally avoids bundling real datasets or model weights.
Runtime dependencies such as Ultralytics, PyTorch, and OpenCV are declared, but the repository validation is lightweight and does not require model downloads.
Replace placeholder class names, dataset paths, and deployment values during implementation.
The mathematical formulas in the Technical Deep Dive section describe what Ultralytics YOLOv8 and ByteTrack implement internally. You do not need to implement them yourself; they are included so you understand what the library is doing and can reason about tuning decisions.

Name		Name	Last commit message	Last commit date
.vscode		.vscode
app		app
configs		configs
docker		docker
docs		docs
scripts		scripts
tests		tests
.editorconfig		.editorconfig
.gitattributes		.gitattributes
.gitignore		.gitignore
Makefile		Makefile
README.md		README.md
pyproject.toml		pyproject.toml
requirements-dev.txt		requirements-dev.txt
requirements.txt		requirements.txt

_Document	_{What it explains}
_{docs/architecture.md}	_{High-level system goals and runtime flow}
_{docs/dataset-workflow.md}	_{Data collection, annotation, and split strategy}
_{docs/deployment.md}	_{Deployment structure and optimization direction}

Latest commit

History

Folders and files

Repository files navigation