Interactive GPT-2 inference explorer with token probability visualization, entropy curves, confidence heatmap, and sampling strategy comparison. Built on nanoGPT.
A web dashboard that lets you watch a neural network think step-by-step as it generates text β showing exactly how confident it is about every single word it chooses.
Most people use language models as black boxes β type a prompt, text comes out. This project opens the black box.
It takes the raw mathematical output of GPT-2 and turns it into an intuitive visual experience. Instead of just showing you the final text, it reveals:
- The probability assigned to every possible next word at each generation step
- Where the model was confident vs uncertain throughout the sequence
- How temperature, top-k, and top-p mathematically reshape what the model produces β shown visually and in real time
Runs entirely on CPU. No GPU required.
Type a prompt, adjust sliders, and watch the model generate with a live token probability chart and confidence heatmap.
Same prompt. Same model. Three different strategies. See how mathematics changes language.
A built-in theory guide with PyTorch code snippets, math explanations, and a preset reference table.
| Feature | What It Does |
|---|---|
| π‘οΈ Confidence Heatmap | Every generated word is color-coded β π’ green (model was certain), π‘ yellow (uncertain), π΄ red (creative leap). Shows at a glance where the model played it safe vs took a risk |
| π Token Probability Chart | Interactive bar chart of the top-10 candidate tokens at every generation step. Scrub the step slider to replay any moment of the generation process |
| π Entropy Curve | Area chart of Shannon entropy at each step. Spikes show exactly where the model faced peak uncertainty β these are the most "interesting" moments |
| π― Perplexity Score | Standard NLP evaluation metric displayed in the metadata bar. Lower = more confident output. Higher = more creative or surprising output |
| π Preset Buttons | One-click parameter configs for four use cases: Code/Factual, Creative Writing, Brainstorming, Max Diversity |
| π Generation History | Logs your last 5 generations with prompt excerpt, perplexity, and settings. One-click reload restores any previous run's exact parameters |
| π₯ Session Export | Downloads a structured session_export.json with your prompt, output, all sampling settings, perplexity, and average entropy |
| Feature | What It Does |
|---|---|
| Parallel Generation | Same prompt β model runs 3 times with Conservative, Balanced, and Creative settings simultaneously |
| Side-by-Side Output | Instantly compare how coherence, repetition, and creativity change across strategies |
| π Winner Badge | Calculates perplexity of all three outputs and automatically awards "Most Creative" to the most diverse result |
| Avg Probability Chart | Bar chart comparing the average top-1 token probability across all three strategies |
A built-in educational reference β explains the math behind every parameter with actual PyTorch code snippets, a comparison table, and a preset guide. No external docs needed.
Every time GPT-2 generates the next word, it produces 50,257 raw scores β one per vocabulary word β called logits. Three parameters control how a final token is sampled from those scores:
Raw Logits β Γ· Temperature β Top-K Filter β Top-P Filter β Softmax β Sample
| Parameter | What It Changes | Low Value | High Value |
|---|---|---|---|
| Temperature | Divides all logits before softmax | Sharpens the distribution β model strongly prefers top tokens β focused, repetitive output | Flattens the distribution β all tokens become more equally likely β creative, unpredictable output |
| Top-K | Keeps only the K highest-probability tokens; zeroes out the rest | Very restricted vocabulary pool | Large candidate pool β more variety, more risk |
| Top-P (Nucleus) | Keeps the smallest set of tokens whose cumulative probability β₯ P | Only the most certain choices | Dynamically expands pool when model is uncertain, contracts when confident |
Why Top-P is smarter than Top-K: When the model is very confident (one word has 90% probability), Top-K=50 still forces 50 candidates. Top-P=0.9 would correctly pick just that one word. When the model is uncertain, Top-P expands the pool automatically. Top-P adapts. Top-K doesn't.
| Preset | Temperature | Top-K | Top-P | Best For |
|---|---|---|---|---|
| Code / Factual | 0.4 | 30 | 0.85 | Code, facts, structured output where predictability is a feature |
| Creative Writing | 0.9 | 50 | 0.95 | Stories, dialogue, natural-sounding conversation |
| Brainstorming | 1.3 | 0 | 0.98 | Ideation where unexpected connections matter more than strict logic |
| Max Diversity | 1.8 | 0 | 1.0 | Stress-testing model limits β highly erratic, often incoherent |
Built on top of Andrej Karpathy's minimalist PyTorch GPT-2 implementation. All original files are left completely untouched.
nanoGPT-visualizer/
β
βββ app.py β
NEW β Gradio 6 Blocks UI, tab handlers, chart builders, session state
βββ inference_engine.py β
NEW β Model loading, Top-P sampling, perplexity & entropy calculations
βββ requirements.txt β
NEW β Project dependencies
β
βββ model.py π§ ORIGINAL β Karpathy's GPT architecture (untouched)
βββ sample.py π§ ORIGINAL β Karpathy's reference generation script (untouched)
βββ train.py π§ ORIGINAL β Karpathy's training loop (untouched)
βββ bench.py π§ ORIGINAL β Benchmarking utilities (untouched)
- Top-P Nucleus Sampling β Implemented from scratch; absent from the original nanoGPT codebase
- Raw Logit Extraction β State-tracking mechanism captures logits at every autoregressive step without disrupting the generation loop
- Perplexity Calculation β
exp(-1/N Β· Ξ£ log P(tokenα΅’))computed from raw logits and actual sampled token IDs - Shannon Entropy per Step β
-Ξ£ P Β· logβ(P)over the full 50,257-word vocabulary distribution at each step - Confidence Heatmap β Custom HTML renderer that maps per-token probability directly to green/yellow/red inline color
- Gradio 6 Blocks Architecture β Multi-tab layout with
gr.State()for history tracking, Plotly for charting, andgr.Filefor export
| Metric | Formula | Low Value | High Value |
|---|---|---|---|
| Perplexity | exp(-1/N Β· Ξ£ log P(tα΅’)) |
Model was confident throughout β output may feel repetitive | Model was frequently surprised β output is diverse or creative |
| Entropy (per step) | -Ξ£ P Β· logβ(P) |
Model strongly preferred one token at that step | Model was torn between many options at that step |
| Avg Top-1 Probability | mean(max(softmax(logitsβ))) |
High uncertainty across the sequence | High confidence across the sequence |
# 1. Clone the repository
git clone https://github.com/YOUR_USERNAME/nanoGPT-visualizer
cd nanoGPT-visualizer
# 2. Install dependencies
pip install -r requirements.txt
# 3. Launch the app
python app.py
# 4. Open your browser
# http://localhost:7860First launch note: GPT-2 weights (~500 MB) download automatically from HuggingFace on first run. All subsequent launches load from local cache instantly.
| Tool | Purpose |
|---|---|
| PyTorch | Core tensor operations and model inference |
| HuggingFace Transformers | Loading pre-trained GPT-2 124M weights |
| Gradio 6 | Interactive multi-tab web dashboard |
| Plotly | Token probability bar chart + entropy area chart |
| tiktoken | OpenAI BPE tokenizer |
| Python 3.x | Core language |
After exploring this visualizer, you will have an intuitive, hands-on understanding of:
- What temperature actually does mathematically β not "makes it creative" but literally divides every logit, sharpening or flattening the softmax curve
- Why Top-P outperforms Top-K β because it adapts the candidate pool size based on the model's real-time confidence at each step
- What perplexity actually measures β the geometric mean of inverse token probabilities; a standard NLP metric, not just a vibes score
- What entropy predicts about output quality β high-entropy steps are where creativity and incoherence both come from; low-entropy steps are grammar and predictable structure
- How autoregressive generation works β one token at a time, each sampled from a distribution conditioned on everything generated before it
- Original GPT architecture and nanoGPT codebase by Andrej Karpathy
- GPT-2 model weights (124M) by OpenAI, served via HuggingFace
- Built as part of an AI/ML internship portfolio project focused on Generative AI and NLP interpretability
Built on nanoGPT by Andrej Karpathy Β Β·Β GPT-2 124M Β Β·Β Runs fully on CPU