OWL — Out-of-Weights Learning

OWL (Out-of-Weights Learning) is a research project and production capability substrate that trains a frozen transformer model to perform new tasks without modifying a single weight ($W_{base}$) or consuming any of the context window.

Instead of traditional fine-tuning (which risks catastrophic forgetting) or in-context learning (which consumes token budget), OWL trains small sparse vectors — called steerers — that are injected directly into the model's residual stream at inference time. The base model remains permanently pristine. Capabilities are stored in modular .owl files that can be loaded, unloaded, or swapped in microseconds.

---

Abstract & Architecture

OWL operates as a parallel execution layer that intercepts the transformer's internal residual stream at multiple depth points during inference. It uses Latent State Routing Maps (LSRM) — a RAM-hosted store mapping activation signatures to sparse steering vectors.

Prompt → [Frozen Layers 0–4] → OWL Hook (Layer 5) ─┐
                                                     ├─ inject ΔA/3
         [Frozen Layers 5–9] → OWL Hook (Layer 10) ─┤
                                                     ├─ inject ΔA/3
         [Frozen Layers 10–14] → OWL Hook (Layer 15)┤
                                                     └─ inject ΔA/3
         [Frozen Layers 15–22] → Output

1. Hashing (Anchor Hook): The model's intermediate activation tensor at the first injection layer is hashed into a 64-bit key via a deterministic random-projection matrix.
2. Lookup: The key queries an in-RAM BK-tree for a matching sparse steerer.
3. Distributed Injection (Satellite Hooks): The steerer ΔA is distributed across multiple injection layers (e.g. 3 layers receiving ΔA/3 each). This prevents cascade divergence and keeps the activations within the model's native manifold.

For more details on the architecture, see the Architectural Blueprint.

---

Empirical Results: Programming Benchmark

We recently evaluated OWL's ability to impart specific programming formatting and syntax skills to a small, unaligned base model (Qwen/Qwen2.5-0.5B). The model was tasked with writing raw Python functions.

• The Problem: The base autoregressive model defaults to conversational output (e.g., wrapping code in markdown blocks like python ... ), which fails raw execution tests.
• The OWL Solution: We trained OWL steerers on just 15 short examples of correctly formatted Python functions.

Benchmark Results:

Metric	Result
Base Model Accuracy	0.00%
OWL-Augmented Accuracy	20.00%
Absolute Improvement	+20.00%

Note: The remaining failures in the OWL model were predominantly due to the model failing to stop generation (e.g. appending new conversational prompts after generating the correct code), demonstrating that the steerers successfully guided the primary logic and formatting.

---

Reproducibility

To replicate our programming benchmark results:

1. Install Dependencies:

   pip install -r requirements.txt

2. Build the Substrate C++ Extension:

   pip install -e .

3. Run the Benchmark Suite:

   python benchmarks/benchmark_owl.py

   The script will evaluate the base model on benchmarks/test_prog.json, train the OWL steerers on benchmarks/train_prog.json for 15 epochs, and then re-evaluate to produce a full audit report.

---

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
from owl import OWLModel, OWLTrainer, OWLConfig

# 1. Load any frozen HuggingFace model
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3-8B")
tokenizer  = AutoTokenizer.from_pretrained("meta-llama/Llama-3-8B")

# 2. Wrap with OWL
config = OWLConfig(num_injection_layers=3, target_sparsity=0.05)
owl    = OWLModel(base_model, config)

# 3. Train OWL steerers (base model weights never change)
trainer = OWLTrainer(owl, config, lr=1e-2)
trainer.train(dataset=my_dataset, epochs=30, tokenizer=tokenizer)
trainer.finalize()

# 4. Save the capability
owl.save("./my_capability")

# 5. Load and use instantly — zero retraining
owl2 = OWLModel.load("./my_capability", base_model)
output = owl2.generate(**inputs)

# 6. Disable OWL to return to the pristine base model
owl2.enabled = False

---

API Reference

OWLConfig

All hyperparameters in one dataclass.

Parameter	Default	Description
lsh_bits	64	Hash key width in bits
num_injection_layers	3	Number of layers receiving distributed injection
max_hamming_distance	4	BK-tree generalisation radius
target_sparsity	0.05	Fraction of d_model dims used per steerer
l1_lambda	1e-4	Sparsity regularisation weight
lr	1e-3	Default AdamW learning rate

OWLModel

Wraps a frozen HuggingFace model.

owl = OWLModel(base_model, config)
owl.enabled = True / False       # toggle injection instantly
owl.save("./directory")          # save substrate + config
owl2 = OWLModel.load("./dir", base_model)

OWLTrainer

Trains steerer parameters while the base model stays frozen.

trainer = OWLTrainer(owl, config, lr=1e-2)
trainer.step(input_ids, labels=labels)      # single step
trainer.train(dataset, epochs=30, ...)      # full loop
trainer.finalize()                          # orthogonalise + commit
trainer.save_checkpoint("./dir")

---

Contributing

See CONTRIBUTING.md.

License

MIT — see LICENSE.