The manifesto for making agent-generated skills actually usable.
Turn notebooks, tutorials, and one-off workflows into reusable, source-grounded, reviewable agent skills.
Get Started · View Example · Read Score Report · Codex Tutorial · Claude Code Tutorial · 中文 README
Quick Start · Tutorials · At A Glance · Benchmark Snapshot · Repository Layout · Example · Contributing
Warning
Default agents can already generate skills. Most of those skills are not reusable. This repository exists to change that.
If you are turning domain knowledge into AI products, internal automations, or reusable agent workflows, this repo gives you a path from:
- notebook
- prompt
- transcript
- one-off demo
to:
- reusable skill
- reviewable artifact
- scored deliverable
- standardizable unit of agent work
Most agent-generated skills fail in predictable ways:
- they summarize notebook prose instead of extracting a stable job
- they document only the branch that happened to run once
- they skip live checks against source code, signatures, help text, or CLI flags
- they have weak validation and no reviewer-side execution evidence
- they are hard to maintain because workflow, references, and evidence are all mixed together
awesome-skill-generate is designed to fix those failure modes.
| Source-Grounded The generator pushes the agent to inspect real source code, inspect.signature(...), help(...), and -h/--help before documenting behavior. |
Branch-Aware It checks branch-heavy selectors such as method, recipe, backend, and mode so one notebook path is not mistaken for the whole interface. |
Evidence-Backed It does not stop at text quality. Skills are reviewed with validation rules, acceptance contracts, and reviewer-side empirical checks when the workflow is data-sensitive. |
| Structured Artifacts The output is split into SKILL.md, references/, assets/, and optionally scripts/ instead of dumping everything into one bloated file. |
Human-Visible Reports The scorer produces a readable report with commands, evidence, weighted scores, residual risks, and a short reason under each dimension score. |
Standard-Driven This repo pushes capability-first naming, environment-agnostic skill content, and explicit split-vs-single-skill decisions instead of notebook-shaped output. |
Tip
If you want the fastest path to value, start from one notebook with a clear stable task, generate one skill, score it, and inspect the report before scaling out.
- Open this repository in Codex or Claude Code.
- Pick a notebook you want to convert.
- Ask the agent to use
skill-authoringand write toexamples/generated-skills/<skill-name>/. - Ask it to review the result with
skill-quality-scorer. - Run validation and acceptance.
Example request:
Use the skill-authoring skill in this repository to convert /absolute/path/to/notebook.ipynb into a reusable skill.
Requirements:
1. Write the output to examples/generated-skills/<skill-name>/
2. Do not just summarize the notebook
3. Check real source code, inspect.signature, help, or -h/--help
4. Inspect branch-heavy parameters such as method, recipe, backend, and mode
5. If the notebook mixes multiple independent jobs, split it into multiple skills or justify why one skill boundary is better
6. Do not put local absolute source paths, python interpreter paths, or local environment names into SKILL.md or references
7. For long-running or GPU-heavy steps, validate a representative smoke path unless a full run is explicitly required
8. Review the generated result with skill-quality-scorer
9. Write a score report in the current directory, with a short reason under each dimension score
10. Run validate and acceptance
English
中文
Notebook / Tutorial / Workflow
|
v
skill-authoring
|
v
Generated Skill Directory
(SKILL.md + references + assets)
|
v
skill-quality-scorer
|
v
Score Report + Validation + Acceptance
| Dimension | Typical default agent output | awesome-skill-generate |
|---|---|---|
| Goal | Summarize the notebook | Build a reusable skill |
| API handling | Uses what the notebook happened to show | Checks live source, signatures, help, and branches |
| Validation | Light or missing | Explicit validation and acceptance |
| Data workflows | Often text-only review | Reviewer-side empirical execution when needed |
| Artifact shape | One big document | SKILL.md + references/ + assets/ + optional scripts/ |
| Human trust | Implicit | Report-backed |
| Maintenance | Drift-prone | Traceable and updateable |
Measured example in this repository:
Note
The benchmark shown here is based on the repository's current dynamo-preprocess example skill and its linked score report.
- generated skill:
examples/generated-skills/dynamo-preprocess/ - score report:
dynamo-preprocess-score-report-2026-03-18.md - weighted score:
95/100 - verdict:
pass
Important
The default-agent baseline below is an illustrative rubric-based profile, not a separately versioned benchmark artifact. It is meant to show the failure modes this repository is trying to eliminate.
Comparison baseline used below:
- a typical default agent output that mostly summarizes a notebook
- does not do full source-grounding or reviewer-side empirical validation
- this baseline is an illustrative rubric-based profile, not a separately versioned benchmark artifact
Typical default agent skill 38/100 [########------------]
awesome-skill-generate example 95/100 [###################-]
Dimension Default Agent This Repo Example
Trigger Precision 3/5 ###-- 5/5 #####
Execution Clarity 2/5 ##--- 5/5 #####
Validation Strength 1/5 #---- 4/5 ####-
Empirical Executability 1/5 #---- 5/5 #####
Context Efficiency 3/5 ###-- 4/5 ####-
Reusability 2/5 ##--- 5/5 #####
Resource Partitioning 2/5 ##--- 5/5 #####
Compatibility Robustness 1/5 #---- 5/5 #####
Maintainability 2/5 ##--- 5/5 #####
The gains are not aesthetic. They come from four concrete upgrades:
- source-grounding against live APIs instead of notebook memory
- branch coverage for
method/recipe/backendstyle selectors - reviewer-side empirical execution checks for data workflows
- a structured artifact layout that remains maintainable as upstream code changes
This repository is not just a prompt collection. It is trying to make skill generation auditable, comparable, and standardizable.
The standard is simple:
- A skill must define a stable capability, not mirror a tutorial title, sample dataset, or organism name.
- If a notebook mixes multiple independently triggerable jobs, the generator should split them into multiple skills unless one shared boundary is clearly better.
- A skill must include a trigger contract, execution spine, and validation contract.
- Concrete API or CLI claims should be grounded in live source, signatures, help text, or CLI help.
- Branch-heavy parameters must be checked for coverage, not inferred from a single notebook path.
- Reusable skill content should stay environment-agnostic; local review configuration does not belong in
SKILL.mdorreferences/. - Generated documentation should use repo-relative paths or import paths instead of machine-specific absolute paths.
- Long-running or GPU-heavy workflows should usually be validated with a representative smoke path inside a bounded review budget, not by defaulting to a full expensive run.
- Data workflows should be reviewable with reviewer-side execution evidence when needed.
- The artifact layout should be explicit:
SKILL.md,references/,assets/, and optionallyscripts/. - A score report should be visible to humans and should include commands, evidence, residual risks, and a short reason under each dimension score.
- Quality should be measured with a rubric, not with vibes.
If enough generated skills follow these rules, skill generation stops being ad hoc prompt craft and starts looking like an engineering discipline.
skills/skill-authoring/Converts a notebook into a reusable skill.skills/skill-quality-scorer/Reviews and scores the generated skill, with reviewer-side empirical validation when appropriate.
awesome-skill-generate/
├── README.md
├── README.zh.md
├── codex-tutorial-en.md
├── codex-tutorial-zh.md
├── claude-code-tutorial-en.md
├── claude-code-tutorial-zh.md
├── skills/
│ ├── skill-authoring/
│ └── skill-quality-scorer/
├── examples/
│ └── generated-skills/
├── scripts/
└── tests/
This repository includes a notebook-derived example skill:
Source notebook:
docs/tutorials/notebooks/100_tutorial_preprocess.ipynb
Example score report:
Validate skills:
python3 scripts/validate_skills.py --root allRun acceptance:
python3 scripts/run_skill_acceptance.py --root allRun tests:
python3 -m unittest discover -s tests -vInspect a Python interface:
python3 scripts/inspect_python_interface.py dynamo.preprocessing:Preprocessor --pretty- A skill is not a notebook summary
- A skill should be capability-first, not dataset-first
- Source code is more authoritative than tutorial memory
- Branch-heavy parameters must be checked for coverage
- Complex notebooks may need to be split into multiple skills
- Reusable skill docs should stay environment-agnostic
- Long-running workflows should use representative smoke validation by default
- Scoring should not rely on text alone when empirical execution is needed
Contributions of all types are more than welcome. Whether it is publishing skills, improving the generator, refining the scorer, tightening the benchmark story, or contributing code, feel free to check out our GitHub Issues and start building.
This project is BSD 2-Clause licensed.
Copyright © 2026 Qiu Lab.