Skip to content

Commit 17ca96a

Browse files
fzipiclaude
andcommitted
docs: add ADRs 0010-0011, second-pass updates across all ADRs
New ADRs: - ADR-0010: Multi-target compilation model — CRSLang compiles to SecLang today, targets Cloud Armor, AWS WAF, Cloudflare in future. Language not constrained by any single target. - ADR-0011: First-class scoring — severity-derived anomaly scoring eliminates setvar boilerplate. Category scores from group membership. Paranoia levels as typed attributes. Cross-ADR updates from design review: - ADR-0001: Add missing fields (request.line, request.filename, files.*, etc.) - ADR-0002: Document regex inlining, pm file references, t:none removal - ADR-0003: Fix Phase 2a/b/c labels, add each() quantifier for multiMatch, add string interpolation note - ADR-0004: Replace skip_to/goto/label with guarded groups - ADR-0006: Add shorthand target-level exclusions, guarded groups with requires clause replacing SecMarker/skipAfter - ADR-0007: Replace skip_to examples with guarded groups - ADR-0008: Update markers section to reference guarded groups - README: Add multi-target compilation, scoring in Phase 3, new ADR table Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 078a2ec commit 17ca96a

10 files changed

Lines changed: 736 additions & 44 deletions

docs/README.md

Lines changed: 26 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -149,8 +149,15 @@ SecLang configuration directives (body limits, PCRE tuning, log paths, etc.) are
149149
explicitly out of scope. Only rule-adjacent metadata (component signature, default
150150
actions, markers, app ID) stays in the language.
151151

152-
See [ADR-0009: Language Base Evaluation](adr/0009-language-base-evaluation.md) and
153-
[ADR-0008: Separation of Configuration](adr/0008-configuration-directives.md).
152+
**Multi-target compilation:** CRSLang is not constrained by any single compilation
153+
target. Today it compiles to SecLang (ModSecurity/Coraza), but the architecture supports
154+
future backends for Google Cloud Armor (CEL), AWS WAF, Cloudflare (Wirefilter), and
155+
others. SecLang generation must be lossless for the CRS ruleset. Features that a target
156+
cannot express are handled by compiler workarounds or clear error messages.
157+
158+
See [ADR-0009: Language Base Evaluation](adr/0009-language-base-evaluation.md),
159+
[ADR-0008: Separation of Configuration](adr/0008-configuration-directives.md), and
160+
[ADR-0010: Multi-Target Compilation](adr/0010-multi-target-compilation.md).
154161

155162
### Phase 1: Typed Field System
156163

@@ -214,13 +221,20 @@ reusable transform chains are defined once and invoked by name.
214221
|---|---|---|
215222
| `transformations: [lowercase, urlDecode]` + `operator: {name: rx, ...}` | `field \|> url_decode() \|> lowercase() \|> matches("...")` | `matches(normalize(field), "...")` |
216223

217-
**Structured actions** — the action bag is replaced with a structured model. If the
218-
custom parser is chosen, `then block { tx.score += 5 }` uses native assignment operators.
219-
If HCL is chosen, effects use structured sub-blocks or string-encoded operations.
224+
**Structured actions and scoring** — the action bag is replaced with a structured model.
225+
If the custom parser is chosen, `then block { tx.score += 5 }` uses native assignment
226+
operators. If HCL is chosen, effects use structured sub-blocks or string-encoded
227+
operations.
220228

221-
| Current | New (custom) | New (HCL) |
222-
|---|---|---|
223-
| `disruptive: block` + `setvar: "tx.score=+10"` | `then block { tx.score += 10 }` | `action = "block"` + `effects { tx_score = "+=10" }` |
229+
Additionally, **anomaly scoring becomes first-class**: severity-derived scoring
230+
eliminates the `setvar` boilerplate from every attack detection rule. A rule just declares
231+
its severity, and the scoring model (defined in globals) handles the rest.
232+
233+
| Current | New |
234+
|---|---|
235+
| `disruptive: block` + `setvar: "tx.score=+10"` | `severity: critical` (score auto-derived) |
236+
| Manual `setvar` per category | Category derived from group membership |
237+
| Complex phase-5 evaluation rules | `scoring_threshold { inbound = 5 }` |
224238

225239
Work:
226240
- Define a `Function` type with signature: name, args, return type
@@ -230,7 +244,8 @@ Work:
230244
- `ctl:` directives become `configure {}` blocks
231245

232246
See [ADR-0002: Pipeline Operator](adr/0002-pipeline-operator.md) (custom parser path),
233-
[ADR-0004: Structured Action Model](adr/0004-structured-actions.md), and
247+
[ADR-0004: Structured Action Model](adr/0004-structured-actions.md),
248+
[ADR-0011: First-Class Scoring](adr/0011-first-class-scoring.md), and
234249
[ADR-0009: Language Base Evaluation](adr/0009-language-base-evaluation.md).
235250

236251
### Phase 4: Text Syntax and Parser
@@ -309,11 +324,13 @@ At every phase:
309324
|-----|-------|-------|--------|
310325
| [0009](adr/0009-language-base-evaluation.md) | 0 | Language Base — HCL, CEL, Expr, or Custom | Proposed |
311326
| [0008](adr/0008-configuration-directives.md) | 0 | Separation of Configuration from Rule Language | Proposed |
327+
| [0010](adr/0010-multi-target-compilation.md) | 0 | Multi-Target Compilation Model | Proposed |
312328
| [0001](adr/0001-typed-field-namespace.md) | 1 | Typed Field Namespace | Proposed |
313329
| [0007](adr/0007-phase-inference.md) | 1 | Phase Inference from Field Types | Proposed |
314330
| [0003](adr/0003-boolean-algebra.md) | 2 | Boolean Algebra Replacing Chains | Proposed |
315331
| [0002](adr/0002-pipeline-operator.md) | 3 | Pipeline Operator for Composition (conditional on 0009) | Proposed |
316332
| [0004](adr/0004-structured-actions.md) | 3 | Structured Action Model | Proposed |
333+
| [0011](adr/0011-first-class-scoring.md) | 3 | First-Class Scoring Model | Proposed |
317334
| [0005](adr/0005-parser-strategy.md) | 4 | Parser Strategy (see 0009 for decision) | Proposed |
318335
| [0006](adr/0006-rule-management.md) | 5 | Rule Management Directives | Proposed |
319336

docs/adr/0001-typed-field-namespace.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,12 @@ Fields use a hierarchical dot-separated namespace:
4141
request.method # was: REQUEST_METHOD
4242
request.uri # was: REQUEST_URI
4343
request.uri.path # was: REQUEST_URI (path component)
44+
request.filename # was: REQUEST_FILENAME
45+
request.basename # was: REQUEST_BASENAME
46+
request.line # was: REQUEST_LINE
4447
request.protocol # was: REQUEST_PROTOCOL
4548
request.body # was: REQUEST_BODY
49+
request.body.length # was: REQUEST_BODY_LENGTH
4650
request.headers # was: REQUEST_HEADERS (entire map)
4751
request.headers["Host"] # was: REQUEST_HEADERS:Host
4852
request.cookies # was: REQUEST_COOKIES (entire map)
@@ -53,8 +57,10 @@ request.args.post # was: ARGS_POST
5357
request.args["id"] # was: ARGS:id
5458
5559
response.status # was: RESPONSE_STATUS
60+
response.protocol # was: RESPONSE_PROTOCOL
5661
response.body # was: RESPONSE_BODY
5762
response.headers # was: RESPONSE_HEADERS
63+
response.content_type # was: RESPONSE_CONTENT_TYPE
5864
5965
client.ip # was: REMOTE_ADDR
6066
client.port # was: REMOTE_PORT
@@ -68,8 +74,13 @@ matched.var # was: MATCHED_VAR
6874
matched.var_name # was: MATCHED_VAR_NAME
6975
matched.vars # was: MATCHED_VARS (map)
7076
77+
files.names # was: FILES_NAMES
78+
files.sizes # was: FILES_SIZES
79+
files.tmpnames # was: FILES_TMPNAMES
80+
7181
multipart.filename # was: MULTIPART_FILENAME
7282
multipart.name # was: MULTIPART_NAME
83+
multipart.part_headers # was: MULTIPART_PART_HEADERS
7384
7485
time.epoch # was: TIME_EPOCH
7586
time.year # was: TIME_YEAR

docs/adr/0002-pipeline-operator.md

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -124,12 +124,12 @@ client.ip |> ip_in_range("10.0.0.0/8", "172.16.0.0/12")
124124
| `le(value)` | Int -> Bool | `@le` |
125125
| `contains(value)` | String -> Bool | `@contains` |
126126
| `contains_word(values...)` | String -> Bool | `@pm` |
127-
| `contains_word_from_file(path)` | String -> Bool | `@pmFromFile` |
127+
| `contains_word_from_file(path)` | String -> Bool | `@pmFromFile` (external word lists) |
128128
| `begins_with(value)` | String -> Bool | `@beginsWith` |
129129
| `ends_with(value)` | String -> Bool | `@endsWith` |
130130
| `within(value)` | String -> Bool | `@within` |
131131
| `ip_in_range(ranges...)` | IP -> Bool | `@ipMatch` |
132-
| `ip_in_range_from_file(path)` | IP -> Bool | `@ipMatchFromFile` |
132+
| `ip_in_range_from_file(path)` | IP -> Bool | `@ipMatchFromFile` (external IP lists) |
133133
| `detect_sqli()` | String -> Bool | `@detectSQLi` |
134134
| `detect_xss()` | String -> Bool | `@detectXSS` |
135135
| `validate_byte_range(range)` | Bytes -> Bool | `@validateByteRange` |
@@ -249,3 +249,22 @@ request.uri | lowercase | matches("pattern")
249249
Start with the functions needed by CRS rules and grow from there.
250250
- **Performance implications** — pipeline representation must compile efficiently to
251251
target engines. Ensure the IR preserves enough information for backends to optimize.
252+
253+
### Notes on File-Based Functions and Regex Assembly
254+
255+
**Regex patterns** from `crs-toolchain` regex assembly (`.ra` files) are **inlined** at
256+
build time. The toolchain compiles them into optimized patterns before CRSLang sees them.
257+
There is no `matches_from_file()` — the final regex is embedded in the rule.
258+
259+
**Pattern match word lists** (`@pmFromFile`) and **IP range lists** (`@ipMatchFromFile`)
260+
remain as external file references via `contains_word_from_file(path)` and
261+
`ip_in_range_from_file(path)`. These lists can be thousands of entries and are data, not
262+
logic — inlining them would make rules unreadable.
263+
264+
### The `t:none` Convention
265+
266+
SecLang's `t:none` transformation (which resets default transformations from
267+
`SecDefaultAction`) has no equivalent in CRSLang and is not needed. Each rule explicitly
268+
states its transforms via the pipeline or named macros, and the `defaults {}` block
269+
(ADR-0008) does not inject hidden transformation chains. The problem `t:none` solved
270+
does not exist in the new model.

docs/adr/0003-boolean-algebra.md

Lines changed: 57 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -280,14 +280,14 @@ rule 901320 (phase: request) {
280280

281281
Use a **layered approach**:
282282

283-
1. **Phase 3a** — implement Option A (side-effects in `then` only). This handles the
283+
1. **Phase 2a** — implement Option A (side-effects in `then` only). This handles the
284284
vast majority of CRS rules. Chain links without intermediate side-effects are already
285285
flattened by the existing normalizer, so this is the natural starting point.
286286

287-
2. **Phase 3b** — add `let` bindings (Option C) for data-flow dependencies. This
287+
2. **Phase 2b** — add `let` bindings (Option C) for data-flow dependencies. This
288288
cleanly handles Category 3 without complicating the boolean expression model.
289289

290-
3. **Phase 3c** — if Category 2 (`ctl` on intermediate links) proves common enough to
290+
3. **Phase 2c** — if Category 2 (`ctl` on intermediate links) proves common enough to
291291
warrant language support, add conditional side-effects (Option B) as an extension.
292292
Before doing so, audit whether these `ctl` patterns can be restructured as separate
293293
rules instead.
@@ -359,6 +359,56 @@ when count(tx.enable) |> eq(1) ...
359359
then pass { init_collection(ip: client.ip + "_" + ua_hash) }
360360
```
361361

362+
### Collection Quantifier: `each()`
363+
364+
SecLang's `multiMatch` action changes how a collection-targeting condition evaluates —
365+
instead of stopping at the first match, it iterates all values and fires side-effects
366+
per match. This is currently modeled as a non-disruptive action, but it is semantically
367+
a condition quantifier.
368+
369+
**Recommendation: `each()` as a condition-level quantifier (Option A).**
370+
371+
```
372+
# Without each(): first match wins, effects fire once
373+
when request.args |> detect_sqli()
374+
375+
# With each(): all values tested, effects fire per match
376+
when each(request.args) |> detect_sqli()
377+
then block {
378+
tx.sqli_score += 5 # incremented per matching argument
379+
log(data: matched.var) # logged per matching argument
380+
}
381+
```
382+
383+
`each()` wraps a map-typed field and signals "iterate all values." Without it, the
384+
default is first-match semantics.
385+
386+
**Alternatives documented:**
387+
388+
- **Option B: Effect-level modifier**`then block (per_match: true) { ... }`. Simpler
389+
to parse but misleading: the reader assumes first-match from the condition until
390+
they notice the modifier.
391+
- **Option C: Separate iteration block** — `for each match { per-match effects } then
392+
block { once-only effects }`. Most expressive (supports both per-match and once-only
393+
effects) but adds a new block type.
394+
- **Option D: Drop it** — if scoring becomes first-class (ADR-0011), per-match scoring
395+
may be handled at the scoring level rather than as a language construct.
396+
397+
`multiMatch` is rarely used in CRS, so Option A is sufficient for the foreseeable
398+
future. Options B/C can be revisited if use cases emerge.
399+
400+
### String Interpolation
401+
402+
SecLang uses `%{TX:score}` and `%{MATCHED_VAR}` for string interpolation in actions
403+
(`logdata`, `msg`, `setvar`). Most of these cases become direct field references or
404+
expressions in CRSLang (e.g., `log(data: matched.var)`).
405+
406+
For cases that require composed strings (log messages, dynamic values), CRSLang needs
407+
a string construction mechanism. The exact form — string interpolation
408+
(`"Score: ${tx.anomaly_score}"`), concatenation (`"Score: " + string(tx.score)`), or
409+
a format function (`format("Score: %d", tx.score)`) — is deferred to the effects model
410+
design in Phase 3. The IR must support composed string values in effect arguments.
411+
362412
### New Capabilities
363413

364414
Boolean algebra enables patterns that are impossible or awkward in SecLang:
@@ -464,10 +514,10 @@ match request {
464514
465515
### Negative
466516
467-
- Three categories of intermediate side-effects require a layered migration (Phase 3a/b/c)
517+
- Three categories of intermediate side-effects require a layered migration (Phase 2a/b/c)
468518
rather than a single clean cutover
469-
- `let` bindings (Phase 3b) add a new language concept not present in SecLang
470-
- Conditional side-effects (Phase 3c, if adopted) complicate the expression model and
519+
- `let` bindings (Phase 2b) add a new language concept not present in SecLang
520+
- Conditional side-effects (Phase 2c, if adopted) complicate the expression model and
471521
require precise execution semantics
472522
- Some deeply chained rules may become long single expressions (mitigated by
473523
line breaks and formatting conventions)
@@ -485,7 +535,7 @@ match request {
485535
(found primarily in initialization and internal-traffic rules like 905111). If they
486536
prove confined to a small set of rules, they can be handled by restructuring those
487537
rules rather than adding Option B to the language. A full CRS audit should quantify
488-
this before committing to Phase 3c.
538+
this before committing to Phase 2c.
489539
- **Category 3 data flow** — `let` bindings change CRSLang from a purely declarative
490540
rule language to one with local variable scoping. This is a significant conceptual
491541
shift. The alternative is to require these patterns to be split into multiple rules

docs/adr/0004-structured-actions.md

Lines changed: 18 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -221,17 +221,29 @@ then:
221221
### Flow Actions
222222
223223
- **`chain`** — eliminated entirely (ADR-0003)
224-
- **`skip`/`skipAfter`** — replaced by rule ordering and labels:
224+
- **`skip`/`skipAfter`/`SecMarker`** — eliminated entirely. These were an
225+
implementation detail of ModSecurity's sequential evaluation model. The actual intent
226+
is **conditional rule activation** — "only run these rules if condition X holds."
227+
228+
This is now expressed as **guarded groups** (ADR-0006):
225229

226230
```
227-
# was: skipAfter:END_RULE_GROUP
228-
goto END_RULE_GROUP # or: skip to END_RULE_GROUP
231+
# was: SecRule TX:DETECTION_PARANOIA_LEVEL "@lt 2" "skipAfter:END-941"
232+
# ... rules ...
233+
# SecMarker "END-941"
229234

230-
label END_RULE_GROUP # was: SecMarker:END_RULE_GROUP
235+
group "xss_pl2" (requires: paranoia >= 2) {
236+
rule 941120 (severity: critical) { ... }
237+
rule 941130 (severity: critical) { ... }
238+
}
231239
```
232240
233-
Note: `skip`/`skipAfter` are rare in CRS and primarily used for backwards
234-
compatibility. Consider deprecating in favor of explicit rule grouping (Phase 6).
241+
The compiler generates the appropriate `skipAfter`/`SecMarker` pairs when compiling
242+
to SecLang. For paranoia-level gating specifically, the `paranoia` attribute on rules
243+
(ADR-0011) allows the compiler to group and gate rules automatically.
244+
245+
No `skip_to()`, `goto`, or `label` exists in CRSLang. The language expresses intent
246+
(conditional activation), not mechanism (skip/marker).
235247
236248
## Alternatives Considered
237249

docs/adr/0006-rule-management.md

Lines changed: 77 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,10 +76,39 @@ exclude rules where severity == critical
7676
exclude rules where tag == "OWASP_CRS/SQL_INJECTION" and phase == request
7777
```
7878
79+
### Target-Level Exclusions (Shorthand)
80+
81+
The most common CRS user customization is "disable rule X for argument Y" — a
82+
target-level exclusion. This is the equivalent of SecLang's
83+
`ctl:ruleTargetRemoveById=942100;ARGS:passwd`. A shorthand syntax avoids the verbosity
84+
of a full `update` block for this common case:
85+
86+
```
87+
# Shorthand: most common exclusion pattern
88+
exclude rule 942100 target request.args["passwd"]
89+
exclude rule 942100 target request.args["username"]
90+
91+
# Multiple targets from one rule
92+
exclude rule 942100 target request.args["passwd"], request.args["token"]
93+
94+
# By tag with target
95+
exclude rules where tag == "OWASP_CRS/SQL_INJECTION" target request.args["search_query"]
96+
```
97+
98+
This compiles to SecLang as:
99+
```
100+
SecRule ... "ctl:ruleRemoveTargetById=942100;ARGS:passwd"
101+
# Or in a REQUEST-900 exclusion file:
102+
SecRuleUpdateTargetById 942100 "!ARGS:passwd"
103+
```
104+
105+
The shorthand covers the vast majority of CRS user customization. For more complex
106+
modifications, the full `update` block syntax is available.
107+
79108
### Update Targets
80109
81110
```
82-
# Remove a specific target from a rule
111+
# Remove a specific target from a rule (full syntax)
83112
update rule 920170 {
84113
remove target request.args["username"]
85114
}
@@ -123,7 +152,10 @@ update rule 920170 {
123152
124153
### Rule Groups
125154
126-
Named groups replace `SecMarker` and provide a scope for batch operations:
155+
Named groups replace `SecMarker` and provide both a scope for batch operations and
156+
**conditional activation** via guard clauses.
157+
158+
**Basic groups** — organize rules and enable batch operations:
127159
128160
```
129161
group sql_injection_checks {
@@ -141,6 +173,49 @@ update group sql_injection_checks {
141173
}
142174
```
143175
176+
**Guarded groups** — replace `skip`/`skipAfter`/`SecMarker` with conditional activation.
177+
The `requires` clause is a boolean expression over rule metadata or TX fields. Rules in
178+
the group only evaluate when the guard is true:
179+
180+
```
181+
# Paranoia level gating (the primary use case for skip/marker in CRS)
182+
group "xss_pl1" (requires: paranoia >= 1) {
183+
rule 941100 (severity: critical) { ... }
184+
rule 941110 (severity: critical) { ... }
185+
}
186+
187+
group "xss_pl2" (requires: paranoia >= 2) {
188+
rule 941120 (severity: critical) { ... }
189+
rule 941130 (severity: critical) { ... }
190+
}
191+
192+
# Custom guard (rare, replaces ad-hoc skip patterns)
193+
group "custom_checks" (requires: tx.enable_custom_checks |> eq(1)) {
194+
rule 100001 { ... }
195+
rule 100002 { ... }
196+
}
197+
```
198+
199+
Guarded groups compile to SecLang as `skipAfter`/`SecMarker` pairs:
200+
201+
```
202+
# CRSLang
203+
group "xss_pl2" (requires: paranoia >= 2) {
204+
rule 941120 ...
205+
}
206+
207+
# Compiled SecLang
208+
SecRule TX:DETECTION_PARANOIA_LEVEL "@lt 2" \
209+
"id:941012,phase:2,pass,nolog,skipAfter:END-xss-pl2"
210+
SecRule ... "id:941120,..."
211+
SecMarker "END-xss-pl2"
212+
```
213+
214+
This replaces `skip_to()`, `goto`, and `label` entirely. CRSLang expresses the intent
215+
(conditional activation), not the mechanism (skip/marker). See also ADR-0004 and
216+
ADR-0011 where paranoia levels as rule attributes enable automatic guard generation.
217+
```
218+
144219
### Unified Selector System
145220

146221
All management directives use the same selector grammar:

0 commit comments

Comments
 (0)