Skip to content

Commit 60bb434

Browse files
pwizlapwizla
authored
Allow agents to do a single round of self-check after PR creation (#3019)
* Update Orchestrator * Update Claude Project instructions * Update Drafter interface * Update Cursor rule * Include warning-type issues in self-check run
1 parent 64404d5 commit 60bb434

File tree

4 files changed

+187
-49
lines changed

4 files changed

+187
-49
lines changed

‎.cursor/rules/strapi-docs-orchestrator.mdc‎

Lines changed: 45 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -41,12 +41,18 @@ You are a documentation review assistant for Strapi. You help review documentati
4141
│ Style │ └──────┬──────┘
4242
│ Checker │ │
4343
└──────┬──────┘ ▼
44-
│ ┌─────────────┐
45-
▼ │ Style │
46-
┌─────────────┐ │ Checker │
47-
│ Integrity │ └──────┬──────┘
44+
│ ┌──────────────┐
45+
▼ │ Self-Review │
46+
┌─────────────┐ │ (OC+UX+SC) │
47+
│ Integrity │ └──────┬───────┘
4848
│ Checker │ │
4949
└─────────────┘ ▼
50+
┌──────────────┐
51+
│ Auto-Correct │
52+
│ (if errors) │
53+
└──────┬───────┘
54+
│
55+
▼
5056
┌─────────────┐
5157
│ Integrity │
5258
│ Checker │
@@ -88,7 +94,7 @@ Otherwise, the Outliner runs a Quick Check.
8894

8995
**Sequence:**
9096
```
91-
Router → Outliner (Generator) if needed → Drafter → Style Checker → Integrity Checker
97+
Router → Outliner (Generator) if needed → Drafter → Self-Review (Outline Checker + UX Analyzer + Style Checker) → Auto-Correct (if errors) → Integrity Checker
9298
```
9399

94100
**Use cases:**
@@ -112,9 +118,9 @@ Router → Outliner (Generator) if needed → Drafter → Style Checker → Inte
112118
| "check links", "verify paths" | **Integrity Checker** | *(coming soon)* |
113119
| "review this PR" | **Router** → **Outliner** (auto-selects Quick or Full) → **Style Checker** | `router.md` → `outliner.md` → `style-checker.md` |
114120
| "full review" | **Router** → **Outliner** (Full Review: Checker + UX) → **Style Checker** | `router.md` → `outliner.md` → `style-checker.md` |
115-
| "create docs for...", "document this feature" | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` |
116-
| "how do I update docs with this?", "update docs with this", "how should I update the documentation?" | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` |
117-
| User provides source material (PR, diff, spec) + asks to document/update it | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` |
121+
| "create docs for...", "document this feature" | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` → `outline-checker.md` + `style-checker.md` |
122+
| "how do I update docs with this?", "update docs with this", "how should I update the documentation?" | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` → `outline-checker.md` + `style-checker.md` |
123+
| User provides source material (PR, diff, spec) + asks to document/update it | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` → `outline-checker.md` + `style-checker.md` |
118124
| User provides spec/story/ticket without instructions | **Router** (Create Mode) | `router.md` |
119125
| User pastes Markdown without instructions | Ask: review or create? | — |
120126

@@ -162,9 +168,32 @@ Step 2: DISPATCH targets (process in priority order: primary → required → op
162168
└─ FOR EACH optional target:
163169
└─ RUN Drafter in Micro-edit mode → produce Micro-edit artifact
164170

165-
Step 3: OUTPUT all deliverables as separate artifacts
171+
Step 3: OUTPUT all Drafter deliverables as separate artifacts
172+
173+
Step 4: SELF-REVIEW (automatic, no pause)
174+
├─ FOR EACH Drafter output:
175+
│ ├─ Read outline-checker.md spec
176+
│ ├─ RUN Outline Checker (template compliance)
177+
│ ├─ IF action was create_page:
178+
│ │ ├─ Read outline-ux-analyzer.md spec
179+
│ │ └─ RUN UX Analyzer (reader experience)
180+
│ ├─ Read style-checker.md spec
181+
│ └─ RUN Style Checker (12 Rules compliance)
182+
│
183+
└─ COLLECT all review reports
184+
185+
Step 5: AUTO-CORRECT (conditional, max 1 retry per target)
186+
├─ IF any review report contains errors:
187+
│ ├─ FOR EACH target with errors:
188+
│ │ ├─ Inject review report(s) as context
189+
│ │ └─ RE-RUN Drafter (same mode) with corrections
190+
│ └─ OUTPUT corrected artifacts (replace originals)
191+
│
192+
└─ IF no errors or warnings (suggestions only):
193+
└─ Append review summary to each artifact as a
194+
<!-- drafter:review-notes --> comment block
166195

167-
Step 4: (Optional) Offer Style Checker as follow-up
196+
Step 6: OUTPUT final deliverables + Self-Review Report artifact
168197
```
169198

170199
### Auto-chain rules
@@ -173,10 +202,11 @@ Step 4: (Optional) Offer Style Checker as follow-up
173202
2. **Do NOT stop after the Router to ask "should I continue?"** — the chain is automatic. Only pause for `ask_user` or critical errors.
174203
3. **Output each deliverable as a separate Markdown artifact** with a descriptive title (e.g., "Routing Report — MCP Server feature", "Draft — cms/features/mcp-server.md").
175204
4. **State the chain upfront.** At the start, tell the user what will run:
176-
> "Running **Create / Update Mode**: Router → Outline Generator → Drafter"
177-
5. **Style Checker is deferred by default.** In auto-chain, the goal is to produce drafts quickly. Offer the Style Checker as a follow-up after delivery, unless the user explicitly asked for a "full review".
205+
> "Running **Create / Update Mode**: Router → Outline Generator → Drafter → Self-Review → Auto-Correct (if needed)"
206+
5. **Self-review runs automatically after the Drafter.** The Outline Checker, UX Analyzer (for `create_page` targets), and Style Checker run on every Drafter output. If errors or warnings are found, the Drafter re-runs once with the review reports as context. Suggestions are appended as review notes but do not trigger a retry. Maximum 1 retry per target — never more.
178207
6. **Handle multiple targets sequentially.** Process primary targets first, then required, then optional.
179208
7. **Respect `conditional` targets.** Do not process them until the condition is resolved.
209+
8. **Self-review severity threshold.** `[error]` and `[warning]` findings trigger a Drafter retry. Only `[suggestion]` findings are reported without causing re-runs. This prevents infinite loops while catching clear rule violations (e.g., "easy/simple" slipping through, missing backticks on file paths, procedures not in numbered lists).
180210

181211
### When Outline Generator is needed vs. straight to Drafter
182212

@@ -240,7 +270,7 @@ Always tell the user which prompt is executing and in which mode:
240270

241271
> "Running **Style Checker** on `account-billing.md`..."
242272

243-
> "Running **Create / Update Mode** (auto-chain): Router → Outline Generator → Drafter"
273+
> "Running **Create / Update Mode** (auto-chain): Router → Outline Generator → Drafter → Self-Review → Auto-Correct (if needed)"
244274

245275
> "Running **Outliner** (Full Review: Checker + UX Analyzer) on `admin-configuration-customization.md` — new file, 1103 lines..."
246276

@@ -831,6 +861,6 @@ tags: [...]
831861
6. **[ux-low]** Nice-to-have UX improvement
832862
```
833863

834-
**Create / Update Mode output:** A series of artifacts produced by the auto-chain (Routing Report, Outline Report if applicable, Draft/Patch/Micro-edit for each target). No consolidated report — each artifact stands alone.
864+
**Create / Update Mode output:** A series of artifacts produced by the auto-chain (Routing Report, Outline Report if applicable, Draft/Patch/Micro-edit for each target, Self-Review Report). Each artifact stands alone. The Self-Review Report consolidates findings from Outline Checker, UX Analyzer, and Style Checker. If errors triggered a Drafter retry, the corrected artifacts replace the originals and the Self-Review Report notes which targets were corrected.
835865

836-
**Behavioral rules:** Determine mode first. State mode explicitly. Execute prompts in sequence. In Review Mode, the Outliner auto-selects Quick Check or Full Review based on the escalation conditions defined in the [Review Mode workflow](#review-mode-existing-content). In auto-chain mode, do not pause between steps unless `ask_user` is set. Deduplicate issues across prompts. Prioritize errors → warnings → ux-high → suggestions → ux-medium → ux-low.
866+
**Behavioral rules:** Determine mode first. State mode explicitly. Execute prompts in sequence. In Review Mode, the Outliner auto-selects Quick Check or Full Review based on the escalation conditions defined in the [Review Mode workflow](#review-mode-existing-content). In auto-chain mode, do not pause between steps unless `ask_user` is set. After the Drafter, run Self-Review automatically (Outline Checker + UX Analyzer for new pages + Style Checker). If errors are found, re-run the Drafter once per target with review reports as context. Deduplicate issues across prompts. Prioritize errors → warnings → ux-high → suggestions → ux-medium → ux-low.

‎agents/prompts/claude-project-instructions.md‎

Lines changed: 45 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -35,12 +35,18 @@ You are a documentation review assistant for Strapi. You help review documentati
3535
│ Style │ └──────┬──────┘
3636
│ Checker │ │
3737
└──────┬──────┘ ▼
38-
│ ┌─────────────┐
39-
▼ │ Style │
40-
┌─────────────┐ │ Checker │
41-
│ Integrity │ └──────┬──────┘
38+
│ ┌──────────────┐
39+
▼ │ Self-Review │
40+
┌─────────────┐ │ (OC+UX+SC) │
41+
│ Integrity │ └──────┬───────┘
4242
│ Checker │ │
4343
└─────────────┘ ▼
44+
┌──────────────┐
45+
│ Auto-Correct │
46+
│ (if errors) │
47+
└──────┬───────┘
48+
│
49+
▼
4450
┌─────────────┐
4551
│ Integrity │
4652
│ Checker │
@@ -82,7 +88,7 @@ Otherwise, the Outliner runs a Quick Check.
8288

8389
**Sequence:**
8490
```
85-
Router → Outliner (Generator) if needed → Drafter → Style Checker → Integrity Checker
91+
Router → Outliner (Generator) if needed → Drafter → Self-Review (Outline Checker + UX Analyzer + Style Checker) → Auto-Correct (if errors) → Integrity Checker
8692
```
8793

8894
**Use cases:**
@@ -106,9 +112,9 @@ Router → Outliner (Generator) if needed → Drafter → Style Checker → Inte
106112
| "check links", "verify paths" | **Integrity Checker** | *(coming soon)* |
107113
| "review this PR" | **Router** → **Outliner** (auto-selects Quick or Full) → **Style Checker** | `router.md` → `outliner.md` → `style-checker.md` |
108114
| "full review" | **Router** → **Outliner** (Full Review: Checker + UX) → **Style Checker** | `router.md` → `outliner.md` → `style-checker.md` |
109-
| "create docs for...", "document this feature" | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` |
110-
| "how do I update docs with this?", "update docs with this", "how should I update the documentation?" | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` |
111-
| User provides source material (PR, diff, spec) + asks to document/update it | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` |
115+
| "create docs for...", "document this feature" | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` → `outline-checker.md` + `style-checker.md` |
116+
| "how do I update docs with this?", "update docs with this", "how should I update the documentation?" | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` → `outline-checker.md` + `style-checker.md` |
117+
| User provides source material (PR, diff, spec) + asks to document/update it | **Auto-chain** (see below) | `router.md` → `outline-generator.md` → `drafter.md` → `outline-checker.md` + `style-checker.md` |
112118
| User provides spec/story/ticket without instructions | **Router** (Create Mode) | `router.md` |
113119
| User pastes Markdown without instructions | Ask: review or create? | — |
114120

@@ -156,9 +162,32 @@ Step 2: DISPATCH targets (process in priority order: primary → required → op
156162
└─ FOR EACH optional target:
157163
└─ RUN Drafter in Micro-edit mode → produce Micro-edit artifact
158164
159-
Step 3: OUTPUT all deliverables as separate artifacts
165+
Step 3: OUTPUT all Drafter deliverables as separate artifacts
166+
167+
Step 4: SELF-REVIEW (automatic, no pause)
168+
├─ FOR EACH Drafter output:
169+
│ ├─ Read outline-checker.md spec
170+
│ ├─ RUN Outline Checker (template compliance)
171+
│ ├─ IF action was create_page:
172+
│ │ ├─ Read outline-ux-analyzer.md spec
173+
│ │ └─ RUN UX Analyzer (reader experience)
174+
│ ├─ Read style-checker.md spec
175+
│ └─ RUN Style Checker (12 Rules compliance)
176+
│
177+
└─ COLLECT all review reports
178+
179+
Step 5: AUTO-CORRECT (conditional, max 1 retry per target)
180+
├─ IF any review report contains errors:
181+
│ ├─ FOR EACH target with errors:
182+
│ │ ├─ Inject review report(s) as context
183+
│ │ └─ RE-RUN Drafter (same mode) with corrections
184+
│ └─ OUTPUT corrected artifacts (replace originals)
185+
│
186+
└─ IF no errors or warnings (suggestions only):
187+
└─ Append review summary to each artifact as a
188+
<!-- drafter:review-notes --> comment block
160189
161-
Step 4: (Optional) Offer Style Checker as follow-up
190+
Step 6: OUTPUT final deliverables + Self-Review Report artifact
162191
```
163192

164193
### Auto-chain rules
@@ -167,10 +196,11 @@ Step 4: (Optional) Offer Style Checker as follow-up
167196
2. **Do NOT stop after the Router to ask "should I continue?"** — the chain is automatic. Only pause for `ask_user` or critical errors.
168197
3. **Output each deliverable as a separate Markdown artifact** with a descriptive title (e.g., "Routing Report — MCP Server feature", "Draft — cms/features/mcp-server.md").
169198
4. **State the chain upfront.** At the start, tell the user what will run:
170-
> "Running **Create / Update Mode**: Router → Outline Generator → Drafter"
171-
5. **Style Checker is deferred by default.** In auto-chain, the goal is to produce drafts quickly. Offer the Style Checker as a follow-up after delivery, unless the user explicitly asked for a "full review".
199+
> "Running **Create / Update Mode**: Router → Outline Generator → Drafter → Self-Review → Auto-Correct (if needed)"
200+
5. **Self-review runs automatically after the Drafter.** The Outline Checker, UX Analyzer (for `create_page` targets), and Style Checker run on every Drafter output. If errors or warnings are found, the Drafter re-runs once with the review reports as context. Suggestions are appended as review notes but do not trigger a retry. Maximum 1 retry per target — never more.
172201
6. **Handle multiple targets sequentially.** Process primary targets first, then required, then optional.
173202
7. **Respect `conditional` targets.** Do not process them until the condition is resolved.
203+
8. **Self-review severity threshold.** `[error]` and `[warning]` findings trigger a Drafter retry. Only `[suggestion]` findings are reported without causing re-runs. This prevents infinite loops while catching clear rule violations (e.g., "easy/simple" slipping through, missing backticks on file paths, procedures not in numbered lists).
174204

175205
### When Outline Generator is needed vs. straight to Drafter
176206

@@ -234,7 +264,7 @@ Always tell the user which prompt is executing and in which mode:
234264
235265
> "Running **Style Checker** on `account-billing.md`..."
236266
237-
> "Running **Create / Update Mode** (auto-chain): Router → Outline Generator → Drafter"
267+
> "Running **Create / Update Mode** (auto-chain): Router → Outline Generator → Drafter → Self-Review → Auto-Correct (if needed)"
238268
239269
> "Running **Outliner** (Full Review: Checker + UX Analyzer) on `admin-configuration-customization.md` — new file, 1103 lines..."
240270
@@ -825,6 +855,6 @@ tags: [...]
825855
6. **[ux-low]** Nice-to-have UX improvement
826856
```
827857

828-
**Create / Update Mode output:** A series of artifacts produced by the auto-chain (Routing Report, Outline Report if applicable, Draft/Patch/Micro-edit for each target). No consolidated report — each artifact stands alone.
858+
**Create / Update Mode output:** A series of artifacts produced by the auto-chain (Routing Report, Outline Report if applicable, Draft/Patch/Micro-edit for each target, Self-Review Report). Each artifact stands alone. The Self-Review Report consolidates findings from Outline Checker, UX Analyzer, and Style Checker. If errors triggered a Drafter retry, the corrected artifacts replace the originals and the Self-Review Report notes which targets were corrected.
829859

830-
**Behavioral rules:** Determine mode first. State mode explicitly. Execute prompts in sequence. In Review Mode, the Outliner auto-selects Quick Check or Full Review based on the escalation conditions defined in the [Review Mode workflow](#review-mode-existing-content). In auto-chain mode, do not pause between steps unless `ask_user` is set. Deduplicate issues across prompts. Prioritize errors → warnings → ux-high → suggestions → ux-medium → ux-low.
860+
**Behavioral rules:** Determine mode first. State mode explicitly. Execute prompts in sequence. In Review Mode, the Outliner auto-selects Quick Check or Full Review based on the escalation conditions defined in the [Review Mode workflow](#review-mode-existing-content). In auto-chain mode, do not pause between steps unless `ask_user` is set. After the Drafter, run Self-Review automatically (Outline Checker + UX Analyzer for new pages + Style Checker). If errors are found, re-run the Drafter once per target with review reports as context. Deduplicate issues across prompts. Prioritize errors → warnings → ux-high → suggestions → ux-medium → ux-low.

0 commit comments

Comments
 (0)