Neo: LLM Security Skill

An AI security co-pilot skill for Claude Code that helps developers, security teams, and non-technical stakeholders identify, test, and fix vulnerabilities in LLM-powered applications.

Features

• Risk-Based Tiering — Automatically classifies projects as Critical/Standard/Exploratory and adjusts testing rigor accordingly
• Attack Library — Comprehensive test cases for prompt injection, jailbreaking, data exfiltration, agent abuse, RAG poisoning, and encoding attacks
• Defense Patterns — Implementation-ready code (Python/TypeScript) for input sanitization, output filtering, prompt hardening, and more
• Promptfoo Integration — Ready-to-use evaluation configs for security testing
• Compliance Mapping — Maps findings to OWASP LLM Top 10, NIST AI RMF, CJIS, SOC2, HIPAA
• Research Mode — Source-tiered research with confidence scoring and anti-hallucination rules
• CI/CD Templates — GitHub Actions workflow for automated security testing

Installation

For Claude Code Users

1. Download the latest .skill file from Releases
2. Copy to your Claude skills directory:

   cp neo-llm-security.skill ~/.claude/skills/

For Project-Specific Installation

Add to your project's .claude/skills/ directory:

mkdir -p .claude/skills
cp neo-llm-security.skill .claude/skills/

Usage

Once installed, Neo activates when you ask about:

• Securing LLM applications
• Prompt injection testing
• Jailbreak defense
• System prompt hardening
• AI compliance (OWASP, NIST, CJIS)
• Security evaluations with promptfoo

Example Interactions

User: "I'm building an AI agent that can access our customer database. Help me secure it."

Neo: [Performs risk assessment, generates threat model, produces promptfoo config]

User: "My system prompt keeps getting extracted. Here it is: [prompt]. Fix it."

Neo: [Analyzes vulnerabilities, provides hardened version with inline explanations]

User: "What's the latest on defending against indirect prompt injection?"

Neo: [Activates research mode, synthesizes sources with confidence levels]

Directory Structure

neo-llm-security/
├── SKILL.md                    # Core skill definition
├── config/
│   ├── tiers.yaml              # Risk tier configurations
│   └── preferences.yaml        # User customization
├── references/
│   ├── THREATS.md              # Attack library
│   ├── DEFENSES.md             # Defense patterns with code
│   └── COMPLIANCE.md           # Framework mappings
├── library/
│   └── test_cases/             # Structured test case library
├── templates/
│   ├── promptfoo/              # Eval configurations
│   ├── ci_cd/                  # CI/CD workflows
│   └── reports/                # Report templates
├── feedback/                   # Effectiveness tracking
└── knowledge/
    └── research_cache/         # Cached research findings

Core Philosophy

• Offensive + Defensive — Find vulnerabilities AND fix them
• Research-driven — Never rely on stale knowledge; express uncertainty when knowledge is thin
• Tool-agnostic — Promptfoo is one weapon, not the whole arsenal
• Natural language first — No syntax memorization required
• Actionable output — Every finding comes with a fix or guided path
• Provenance matters — Every recommendation traces to a source
• Risk-proportionate — Adapts depth and rigor to actual stakes

Running Security Tests

Neo generates promptfoo configurations. To run:

# Install promptfoo
npm install -g promptfoo

# Run evaluation
npx promptfoo@latest eval

# View results
npx promptfoo@latest view

Compliance Support

Neo maps security findings to:

Framework	Coverage	Notes
OWASP LLM Top 10	Deep	Direct test case mapping
NIST AI RMF	Moderate	GOVERN, MAP, MEASURE, MANAGE
CJIS	Moderate	Technical controls only
SOC2	Light	Security trust principle
HIPAA	Light	Technical safeguards only
EU AI Act	Emerging	Risk classification

Important: Neo supports compliance efforts but does not certify compliance. Recommend review by qualified assessors.

Limitations

Neo cannot:

• Execute tests (user runs locally)
• Access production systems
• Certify compliance
• Guarantee zero vulnerabilities

Neo will:

• Tell you when it doesn't know
• Express uncertainty with confidence levels
• Recommend human expert involvement when appropriate

Contributing

Contributions welcome! Areas of interest:

• New attack patterns
• Defense implementations
• Compliance framework mappings
• Test case additions

License

MIT

Acknowledgments

Built with research from:

• OWASP LLM Top 10
• MITRE ATLAS
• JailbreakBench / HarmBench
• Academic security research (Greshake et al., Carlini et al.)
• Promptfoo project