Skip to content

Commit 9b43be3

Browse files
teilomilletclaude
andcommitted
docs: restore SEPA as entropy pooling, note surprisal is current proxy
SEPA is conceptually about entropy pooling — the name and theory are entropy-based. Surprisal (-logprob) is used today because backends only provide logprobs, not full token distributions. Updated docs and manual to keep "Selective Entropy Pooling of Attention" while noting that the implementation currently operates on surprisal values. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 1df1965 commit 9b43be3

2 files changed

Lines changed: 17 additions & 12 deletions

File tree

docs/sepa.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,16 @@ Selective Entropy Pooling of Attention (SEPA) is an adaptive scheduling strategy
44

55
## What SEPA does
66

7-
In the [advantage pipeline](advantages.md), GTPO weights token advantages by token surprisal (`-logprob`) -- high-surprisal tokens get more credit. SEPA modifies execution-token surprisals *before* GTPO weighting by pulling them toward their mean:
7+
In the [advantage pipeline](advantages.md), GTPO weights token advantages by per-token uncertainty -- high-uncertainty tokens get more credit. SEPA modifies execution-token uncertainty values *before* GTPO weighting by pulling them toward their mean:
88

99
```
1010
H_pooled(t) = lambda * mean(H_exec) + (1 - lambda) * H(t) if execution token
1111
H_pooled(t) = H(t) if planning token
1212
```
1313

14-
When `lambda = 0`, SEPA is off and GTPO sees raw surprisals. When `lambda = 1`, all execution tokens have the same surprisal (the mean), so GTPO's differentiation is concentrated entirely on planning tokens.
14+
When `lambda = 0`, SEPA is off and GTPO sees raw values. When `lambda = 1`, all execution tokens have the same uncertainty (the mean), so GTPO's differentiation is concentrated entirely on planning tokens.
15+
16+
Today `uncertainty_kind = "surprisal"` (sampled-token `-logprob`) is the only available signal. When backends provide full token distributions, `shannon_entropy` becomes usable and SEPA will pool true entropy values.
1517

1618
## Why
1719

retrain/retrain.man

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -278,7 +278,7 @@ CONFIGURATION
278278
High-surprisal tokens get more credit.
279279
Controlled by gtpo_beta.
280280
gtpo_hicra GTPO + HICRA planning token amplification.
281-
gtpo_sepa GTPO + SEPA selective surprisal pooling (default).
281+
gtpo_sepa GTPO + SEPA selective entropy pooling (default).
282282
gtpo_sepa_amp GTPO + SEPA with amplification variant.
283283
gtpo_sepa_amp_c GTPO + SEPA amplification with clamping.
284284
entropy_mask Yue et al. surprisal masking. Zeros out advantages
@@ -437,7 +437,7 @@ CONFIGURATION
437437

438438
schedule = "linear"
439439
"linear" or "auto". Auto adapts based on execution-token
440-
surprisal variance decay with linear as fallback floor.
440+
entropy variance decay with linear as fallback floor.
441441

442442
delay_steps = 50
443443
Steps to delay before beginning the SEPA ramp. Must be >= 0.
@@ -861,11 +861,13 @@ ADVANTAGE PIPELINE
861861
or semantic similarity.
862862

863863
SEPA
864-
Selective Surprisal Pooling of Attention.
865-
Pools execution-token surprisals toward their mean, reducing
866-
surprisal variance on execution tokens while preserving diverse
867-
planning behavior. Lambda ramps 0->1 over training, gated by
868-
a correct_rate threshold.
864+
Selective Entropy Pooling of Attention.
865+
Pools execution-token uncertainty values toward their mean,
866+
reducing variance on execution tokens while preserving diverse
867+
planning behavior. Currently operates on surprisal (the only
868+
available signal); will use full entropy when backends provide
869+
token distributions. Lambda ramps 0->1 over training, gated
870+
by a correct_rate threshold.
869871

870872
Post-Process Hooks
871873

@@ -1155,9 +1157,10 @@ GLOSSARY
11551157
weighted credit assignment at the token level.
11561158
HICRA Hierarchical Credit Assignment. Planning token amplification
11571159
via a learned mask.
1158-
SEPA Selective Surprisal Pooling of Attention. Pools execution-token
1159-
surprisals to reduce variance while preserving planning
1160-
diversity.
1160+
SEPA Selective Entropy Pooling of Attention. Pools execution-token
1161+
uncertainty values to reduce variance while preserving
1162+
planning diversity. Uses surprisal today; full entropy
1163+
when backends provide token distributions.
11611164
LoRA Low-Rank Adaptation. Parameter-efficient fine-tuning via
11621165
rank-decomposed weight updates.
11631166
SVD Singular Value Decomposition. Used by LoRA-Squeeze for rank

0 commit comments

Comments
 (0)