Add CLAUDE.md with AI agent instructions and quick reference

[supriyar]

Stack from ghstack (oldest at bottom):

• Expand CODEOWNERS and add GitHub issue templates #4197
• Add llms.txt to docs site for LLM crawler discoverability #4196
• -> Add CLAUDE.md with AI agent instructions and quick reference #4195

Replace empty placeholder with structured documentation for AI coding
assistants (Claude Code, Cursor, Copilot). Includes config class table,
granularity reference, deprecated API warnings, and pointers to in-repo
docs for architecture details.

Comparison: Old CLAUDE.md vs New CLAUDE.md
Instructions+Scripts for repro available in https://github.com/supriyar/torchao-eval
Setup:

• Subject model: Claude Sonnet
• Judge model: Claude Opus

Sonnet/Opus Results (61 prompts, final)

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4195

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 10 Pending

As of commit 15beca9 with merge base 79159f2 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vkuzo]

I think we should set up hooks for running pre-commit/linters: https://code.claude.com/docs/en/hooks, this gets code style for free without spending tokens

[jerryzh168]

you mean pre-commit run? we do have these I think

[supriyar]

I think Vasiliy is referring to claude code hooks for auto-format on edit? That can be a follow-up PR.

Will remove lint instructions from here

[vkuzo]

I have this in my personal CLAUDE.md, but this seems specific to personal preference? maybe leave out of repo-wide one?

[supriyar]

had originally added this since I saw it in the pytorch/pytorch claude.md file. vLLM also has similar instructions in its repo for agent authored commits

[vkuzo]

if pytorch and vllm have it, makes sense, thanks

[vkuzo]

+32% improvement, wrong answers dropped from 13 to 2. The new CLAUDE.md has the biggest impact on architecture questions, MX/NVFP4 formats, and comparison questions - the areas where the old empty CLAUDE.md gave the model nothing

I think this is a good way to eval. Can we check the eval into source control so it's easy to iterate on and measure future improvements? Doesn't have to be in torchao, can be in separate repo if that's easier.

[vkuzo]

also, for the eval, should we just use Opus 4.6 for everything? IMO better to optimize for the best available model, and even doing this things can go out of date really quickly. I'm not sure it's worth spending a lot of time on evals of older models

[supriyar]

also, for the eval, should we just use Opus 4.6 for everything? IMO better to optimize for the best available model, and even doing this things can go out of date really quickly. I'm not sure it's worth spending a lot of time on evals of older models

Sonnet/Opus Results (61 prompts, final)

I ran both Opus/Opus and Sonnet/Opus on 61 prompts. The issue with Opus as subject is it times out on ~26% of prompts (16/61) with the 56K char system prompt, even with 180s timeout. Opus generates verbose responses that exceed the time limit. timeouts score 0 and corrupt the data.

Sonnet as subject had zero timeouts and scored comparably on the prompts where Opus didn't time out. Results with Sonnet subject + Opus judge: 2.51 -> 2.74 (+9%), 47/61 perfect scores with new CLAUDE.md vs 37/61 without.

Eval repro: https://github.com/supriyar/torchao-eval

[supriyar]

@pytorchbot merge

[pytorchmergebot]

Merge failed

Reason: 1 mandatory check(s) are pending/not yet run. The first few are:

• Facebook CLA Check

Dig deeper by viewing the pending checks on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: superuser