Transformer X-Ray

Transformer X-Ray is an educational, research-style interpretability app that visualizes transformer internals step-by-step.

Stack

• Backend: FastAPI + PyTorch
• Frontend: React + D3.js (Vite)
• Model: Tiny custom transformer (small dimensions for readability)

Features Implemented

• Tokenization view (tokens + IDs)
• Embedding vectors + PCA 2D projection
• Positional encoding view
• Attention math pipeline (Q, K, V, dot-product, scaling, softmax, weighted sum)
• Multi-head toggling + re-run
• Residual stream monitor
• FFN internals (linear1, activation, linear2)
• Logit lens per layer
• Sampling simulator (temperature, top-k, top-p, entropy)
• Head behavior analyzer (entropy, focus, induction-like flag)
• Model comparison mode (tiny transformer vs optional distilgpt2)
• Step-by-step playback + autoplay
• Educational tooltips

Repository Layout

• backend/ FastAPI app, model internals, endpoints
• frontend/ React + D3 visualization app
• docs/ architecture and endpoint docs

Local Setup

1) Backend

cd backend
python -m venv .venv
# Windows PowerShell
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8000

2) Frontend

cd frontend
npm install
npm run dev

Default frontend URL: http://localhost:5173

Quick PowerShell Setup (Windows)

cd "c:\Users\harman deep singh\OneDrive\Desktop\codex transformer"
.\scripts\setup_backend.ps1
.\scripts\setup_frontend.ps1

Quick Run (Windows)

Run each in separate terminals:

.\scripts\run_backend.ps1
.\scripts\run_frontend.ps1

If backend URL differs, set:

# frontend/.env
VITE_API_URL=http://127.0.0.1:8000

API Endpoints

• POST /forward
• POST /attention
• POST /logits
• POST /activations
• POST /compare (optional GPT2 comparison)
• GET /health

See docs/API.md for request/response examples.

Run Tests

cd backend
pytest

GitHub Push Workflow

git init
git add .
git commit -m "Initial commit: Transformer X-Ray full-stack interpretability app"
git branch -M main
git remote add origin https://github.com/<your-username>/transformer-xray.git
git push -u origin main

Notes

• GPT-2 comparison requires transformers package and model download access.
• Tiny transformer is intentionally small and untrained by default for transparent visualization.