examples: add RLM (Recursive Language Model) demo

[hyprh]

Summary

Implement the RLM paper (arXiv:2512.24601) as a Go example demonstrating how LLMs can process very large documents via recursive code-driven decomposition.

Architecture

Root Agent (depth=0, ~2.2M chars loaded as REPL context)
├── Explore context structure with Starlark
├── Decompose → rlm_query_batched() spawns child agents
│   ├── Child Agent (depth=1, <=60K char context)
│   │   └── Analyze directly via llm_query / llm_query_batched()
│   └── Child Agent (depth=1, larger logical scope)
│       └── Further decompose into smaller child contexts
└── Aggregate child results → final_answer

• ReAct Agent Loop — each node is an autonomous agent with execute_code + final_answer tools
• HTTP Service — centralized LLM proxy and recursive RLM orchestrator with rate limiting
• Starlark REPL — embedded Python-subset interpreter for LLM-generated code, with the full external context kept outside the prompt as context
• Symbolic Recursion — rlm_query() spawns child RLM instances via HTTP
• Concurrent Batching — rlm_query_batched / llm_query_batched for parallel execution
• Rate Limiting — token-bucket limiter (default 20 QPM) with exponential backoff retry
• Runtime Guardrails — 8 KiB tool-output truncation with notice, 30K direct LLM prompt limit, 60K child-context limit, and max 10 child agents per RLM batch
• Model Call Timeout — each model request is bounded to 2 minutes to avoid hanging forever on an unreachable gateway

File Structure

File	Responsibility
simple/main.go	CLI entry point, auto-clones demo repo, starts service
simple/service.go	HTTP server: LLM proxy + recursive RLM orchestration
simple/rlm.go	ReAct agent loop: thought → action → observation
simple/repl.go	Starlark REPL with builtins (context, llm_query, rlm_query, ...)
simple/prompt.go	Dynamic system prompt and runtime guidance
simple/tools.go	Tool definitions: execute_code, final_answer, output limiting
simple/ratelimit.go	Token-bucket rate limiter
README.md	Usage docs, architecture overview, and guardrails

Demo

The example auto-clones EbookFoundation/free-programming-books, currently loading about 2.2M chars across 200+ markdown files, and identifies outdated/deprecated content:

cd examples/rlm
export OPENAI_API_KEY="..." OPENAI_BASE_URL="..." MODEL_NAME="..."
go run ./simple/ 2>rlm-run.log

Key Design Decisions

1. ReAct pattern with tool calling instead of a hardcoded processing loop
2. LLM decides decomposition strategy based on context size and content
3. Full document context stays external in the REPL, not in every model prompt
4. rlm_query is called from Starlark so child contexts are explicit slices produced by code
5. Root query is propagated to all sub-agents for global task awareness
6. Guardrails reject oversized direct prompts/child contexts and truncate excessive tool output with a visible notice
7. HTTP recursion keeps a long timeout, while each model request has a shorter timeout for bad gateway/network cases

Validation

• cd examples/rlm && go test ./...
• Manual run with an OpenAI-compatible Venus endpoint verified the demo progresses past the first LLM call and exercises child-agent fan-out under the new context limits.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 905f1668-c024-44e3-aa95-e8dfe314584c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch hyprh/examples-rlm-recursive-language-model

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Flash-LHR]

maxDepth=1 never runs a child RLM because Depth reaches 1 before this check. Use req.Depth > s.maxDepth here so one configured recursive level executes.

中文

`maxDepth=1` 不会运行子级 RLM，因为检查前 `Depth` 已经变成 1。这里改为 `req.Depth > s.maxDepth`，让配置的一级递归实际执行。

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.77838%. Comparing base (2343bf5) to head (9c3b508).
⚠️ Report is 14 commits behind head on main.

Additional details and impacted files

@@                 Coverage Diff                 @@
##                main       #1778         +/-   ##
===================================================
+ Coverage   89.77574%   89.77838%   +0.00263%     
===================================================
  Files            936         936                 
  Lines         151610      151610                 
===================================================
+ Hits          136109      136113          +4     
+ Misses          9770        9769          -1     
+ Partials        5731        5728          -3     

Flag	Coverage Δ
unittests	89.77838% <ø> (+0.00263%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.