Independent benchmarks: InclusionAI Ling-mini-2.0 & Ring-mini-2.0

First independent, reproducible third-party evaluation of InclusionAI (Ant Group)'s bailing_moe mini models — a 16B-total / 1.4B-active MoE (1/32 expert activation) and its reasoning-tuned sibling. Every number here was generated on rented hardware with pinned, documented tooling, and every raw result is in /results.

Author: iAmBoosted — HuggingFace · GitHub

---

Why this exists

Both models ship strong efficiency claims and a pile of self-reported benchmarks, but no one had independently verified them. The custom bailing_moe architecture (sigmoid routing, MTP layers, half-RoPE, QK-Norm, aux-loss-free training) doesn't load under standard tooling without specific library versions and a patch — which is most likely why the community left it un-evaluated. This repo documents exactly how to run it and publishes honest, reproducible numbers.

Models

	Ling-mini-2.0	Ring-mini-2.0
Architecture	bailing_moe (custom MoE)	identical
Total / active params	16.26B / 1.43B	16.26B / 1.43B
Expert activation	1/32	1/32
Post-training	instruct SFT	Long-CoT SFT + RLVR + RLHF
Mode	direct response	chain-of-thought (<think>)
License	MIT	MIT

Headline results

All scores are single-run, BF16, no quantization. Full methodology, commands, and stderr in the per-model writeups.

Correction (June 2026): Two of Ring-mini-2.0's rows below do not reflect capability. AIME (10.0%) is invalid — in the 8192-token run, 14/30 responses never closed </think> (median response ~19k characters), so the model was cut off mid-reasoning; n=30 compounds it. HumanEval (65.24% ±3.73%) is within ~1.4 standard errors of Ling and is not a significant gap. GPQA is loglikelihood, where reasoning gives no benefit by construction. No code- or competition-math regression is claimed. Full detail in the Ring writeup.

Benchmark	Ling-mini-2.0	Ring-mini-2.0
GPQA Diamond (0-shot, loglikelihood)	37.88%	37.88%
GSM8K (5-shot, strict)	80.89%	79.76%
MMLU-Pro (5-shot)	53.34%	54.52%
HumanEval (pass@1)	72.56%	65.24% †
AIME 2024 (0-shot, greedy)	16.7%	10.0% ‡

† Within ~1.4 stderr of Ling — not a significant difference. ‡ Invalid: 14/30 responses truncated mid-reasoning at 8192 tokens (n=30). See correction above.

Detailed writeups: Ling-mini-2.0 · Ring-mini-2.0

Self-reported vs. independent

InclusionAI's published suite and this independent run overlap on only two benchmarks. The full coverage map, the head-to-head numbers, and an important caveat about evaluation modes (a reasoning model scores ~30 points apart on GPQA depending only on whether it's allowed to reason) are in comparison/vendor-vs-independent.md.

Reproduce it

Hardware used: NVIDIA RTX A6000 (48 GB), RunPod. Any GPU with ≥40 GB VRAM works.

# 1. Core deps — the transformers pin is mandatory (see writeups for why)
pip install -r requirements.txt

# 2. lm-evaluation-harness from source
git clone https://github.com/EleutherAI/lm-evaluation-harness.git
cd lm-evaluation-harness && pip install -e ".[math]" && cd ..

# 3. Patch vLLM for the bailing_moe architecture
VLLM_PATH=$(python -c "import vllm; print(vllm.__path__[0])")
( cd "$VLLM_PATH/.." && git apply /path/to/bailing_moe_v2.patch )

# 4. Run a benchmark (Ling example)
lm_eval --model vllm \
  --model_args pretrained=inclusionAI/Ling-mini-2.0,trust_remote_code=True,dtype=bfloat16 \
  --tasks mmlu_pro --batch_size auto

# 5. AIME (custom script)
python scripts/aime_eval.py

reinit.sh re-establishes the full environment on a fresh pod in one shot.

Repository layout

evaluations/   full per-model writeups (methodology, results, caveats)
comparison/    self-reported vs. independent coverage map + head-to-head
results/       raw lm-evaluation-harness JSON + AIME outputs (the proof)
scripts/       AIME evaluation scripts (Ling + both Ring variants)
assets/        coverage map graphic
reinit.sh      one-shot environment setup for a fresh GPU pod
bailing_moe_v2.patch   vLLM patch required to load the architecture

Limitations

Single run per benchmark; small-n benchmarks (GPQA n=198, AIME n=30) carry real variance — see stderr in each writeup. Reference comparisons to other models use approximate public numbers under possibly-different methodology. AIME uses a custom script (no canonical harness task). Ling's GPQA/GSM8K were initially run on the hf backend before vLLM was configured; loglikelihood scoring is deterministic across backends, but it's disclosed in the writeups for full transparency.

Citation

@misc{boosted2026inclusionaimini,
  author = {iAmBoosted},
  title  = {Independent Benchmarks: InclusionAI Ling-mini-2.0 and Ring-mini-2.0},
  year   = {2026},
  url    = {https://github.com/iamboosted/inclusionai-mini-2.0-benchmarks}
}

License

MIT — see LICENSE. The models are MIT-licensed by InclusionAI.