Skip to content

README.md

Latest commit

 

History

History
40 lines (27 loc) · 1.16 KB

README.md

File metadata and controls

40 lines (27 loc) · 1.16 KB
Raw

k8s-bench

k8s-bench is a benchmark for assessing the performance of LLM models for kubernetes related tasks.

Usage

# build the k8s-bench binary
go build

# Run evaluation for scale related tasks
./k8s-bench run --agent-bin <path/to/kubectl-ai/binary> --task-pattern scale --kubeconfig <path/to/kubeconfig>

# Analyze previous evaluation results and output in markdown format (default)
./k8s-bench analyze --input-dir .build/k8sbench

# Analyze previous evaluation results and output in JSON format
./k8s-bench analyze --input-dir .build/k8sbench --output-format json

# Save analysis results to a file
./k8s-bench analyze --input-dir .build/k8sbench --results-filepath ./results.md

Running the benchmark with the run subcommand will produce results as below:

Evaluation Results:
==================

Task: scale-deployment
  Provider: gemini
    gemini-2.0-flash-thinking-exp-01-21: true

Task: scale-down-deployment
  Provider: gemini
    gemini-2.0-flash-thinking-exp-01-21: true

The analyze subcommand will gather the results from previous runs and display them in a tabular format with emoji indicators for success (✅) and failure (❌).