Skip to content

feat: add context-drift skill for detecting documentation drift#12

Open
tomiir wants to merge 1 commit intomainfrom
feat/context-updater
Open

feat: add context-drift skill for detecting documentation drift#12
tomiir wants to merge 1 commit intomainfrom
feat/context-updater

Conversation

@tomiir
Copy link
Contributor

@tomiir tomiir commented Jan 20, 2026

Summary

  • Adds new /context-drift skill that analyzes .claude/ context files for drift from the codebase
  • Detects broken references (missing files, invalid scripts, bad tool names)
  • Identifies semantic mismatches (documented behavior vs actual implementation)
  • Flags stale documentation for human review
  • Finds orphaned code that lacks documentation

Files Added

  • .claude/skills/context-drift/SKILL.md (304 lines) - Main skill definition with workflow, examples, evaluation prompts
  • .claude/skills/context-drift/REFERENCE.md (243 lines) - Detection patterns, heuristics, severity matrix

Test plan

  • Run /context-drift on this repository
  • Verify it scans all .claude/ context files
  • Intentionally break a reference and confirm Critical detection
  • Test orphaned code detection

🤖 Generated with Claude Code

Adds a new skill that analyzes .claude/ context files (skills, commands,
agents) for drift from the codebase. Features include:

- Broken reference detection (missing files, invalid scripts)
- Semantic mismatch detection (documented vs actual behavior)
- Staleness flags for human review
- Orphaned code detection (significant code lacking docs)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings January 20, 2026 10:41
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new context-drift skill that helps maintain documentation quality by detecting drift between .claude/ context files (skills, commands, agents) and the actual codebase. The skill identifies broken references, semantic mismatches, stale documentation, and orphaned code that lacks documentation.

Changes:

  • Adds comprehensive context-drift detection skill with workflow for scanning, analyzing, and reporting on documentation drift
  • Provides detailed reference guide with extraction patterns, detection heuristics, and severity classification
  • Includes validation checklists, examples, and evaluation prompts for testing the skill

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
.claude/skills/context-drift/SKILL.md Main skill definition including workflow phases (Discovery, Analysis, Orphan Detection, Reporting), output format, examples, and evaluation prompts
.claude/skills/context-drift/REFERENCE.md Technical reference with regex patterns for extraction, detection heuristics for broken references and semantic mismatches, staleness thresholds, and severity classification matrix

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +71 to +73
| At-mentions | `@src/utils.ts` | `@([\w/./-]+\.\w+)` |
| Quoted paths | `"src/utils.ts"` | `"([\w/./-]+\.\w+)"` |
| In bash commands | `cat src/file.ts` | `(cat\|head\|tail\|read) ([\w/./-]+)` |
Copy link

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern for "At-mentions" has an issue with the character class. The pattern @([\w/./-]+\.\w+) contains /./- which is problematic because the dash - should either be escaped or placed at the beginning or end of the character class. Currently, it could be interpreted as a range from / to / which is not valid. Consider changing to @([\w/._-]+\.\w+) to properly match paths with dots, underscores, and hyphens.

Suggested change
| At-mentions | `@src/utils.ts` | `@([\w/./-]+\.\w+)` |
| Quoted paths | `"src/utils.ts"` | `"([\w/./-]+\.\w+)"` |
| In bash commands | `cat src/file.ts` | `(cat\|head\|tail\|read) ([\w/./-]+)` |
| At-mentions | `@src/utils.ts` | `@([\w/._-]+\.\w+)` |
| Quoted paths | `"src/utils.ts"` | `"([\w/._-]+\.\w+)"` |
| In bash commands | `cat src/file.ts` | `(cat\|head\|tail\|read) ([\w/._-]+)` |

Copilot uses AI. Check for mistakes.
Comment on lines +71 to +73
| At-mentions | `@src/utils.ts` | `@([\w/./-]+\.\w+)` |
| Quoted paths | `"src/utils.ts"` | `"([\w/./-]+\.\w+)"` |
| In bash commands | `cat src/file.ts` | `(cat\|head\|tail\|read) ([\w/./-]+)` |
Copy link

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern for "Quoted paths" has the same character class issue as the At-mentions pattern. The pattern "([\w/./-]+\.\w+)" contains /./- which could cause regex parsing issues. Consider changing to "([\w/._-]+\.\w+)" to properly match paths.

Suggested change
| At-mentions | `@src/utils.ts` | `@([\w/./-]+\.\w+)` |
| Quoted paths | `"src/utils.ts"` | `"([\w/./-]+\.\w+)"` |
| In bash commands | `cat src/file.ts` | `(cat\|head\|tail\|read) ([\w/./-]+)` |
| At-mentions | `@src/utils.ts` | `@([\w/._-]+\.\w+)` |
| Quoted paths | `"src/utils.ts"` | `"([\w/._-]+\.\w+)"` |
| In bash commands | `cat src/file.ts` | `(cat\|head\|tail\|read) ([\w/._-]+)` |

Copilot uses AI. Check for mistakes.
| Code block paths | `` `src/utils.ts` `` | `` `([^`]+\.(ts|js|py|go|rs|md))` `` |
| At-mentions | `@src/utils.ts` | `@([\w/./-]+\.\w+)` |
| Quoted paths | `"src/utils.ts"` | `"([\w/./-]+\.\w+)"` |
| In bash commands | `cat src/file.ts` | `(cat\|head\|tail\|read) ([\w/./-]+)` |
Copy link

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern uses escaped pipe characters (cat\|head\|tail\|read) which would match literal text like "cat|head" rather than "cat OR head OR tail OR read". In regex, pipes should not be escaped when used for alternation. The pattern should be (cat|head|tail|read) ([\w/._-]+) instead.

Suggested change
| In bash commands | `cat src/file.ts` | `(cat\|head\|tail\|read) ([\w/./-]+)` |
| In bash commands | `cat src/file.ts` | `(cat|head|tail|read) ([\w/._-]+)` |

Copilot uses AI. Check for mistakes.
## Critical Issues

### .claude/skills/worktree/SKILL.md
- **Line 58**: References `git worktree add -b` but worktree command deprecated
Copy link

Copilot AI Jan 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example claims git worktree add -b is deprecated, but this is inaccurate. The git worktree add -b <branch> command is still valid and not deprecated in Git. Consider using a different example for a broken reference that would actually represent a real drift scenario, such as a file path that doesn't exist.

Suggested change
- **Line 58**: References `git worktree add -b` but worktree command deprecated
- **Line 58**: References script `./scripts/old-deploy.sh` which does not exist

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants