VS Code: Persist and display token usage by turn/session/history

[benjschiller]

Summary

Token usage is currently visible in some runtime views but not consistently persisted or shown across chat/session/history contexts. We want token usage to be reliable, local-first, and user-configurable in one place.

Problem

• Token counts are not consistently persisted at the turn level in session artifacts.

Proposal

1. Persist token usage per assistant turn in session history artifacts.
2. Compute per-session and history-bucket totals from persisted session data.
3. Add a single setting:
   • continue.showTokenUsage: "never" | "history" | "session" | "turn"
4. Setting semantics are escalating:
   • never: show nowhere
   • history: history only
   • session: history + session
   • turn: history + session + turn
5. Apply setting changes live without requiring Developer: Reload Window.
6. Keep existing stats DB/schema unchanged (no migration in this change).
7. Keep token tracking always-on; setting controls display only.

PR incoming :)

[xXMrNidaXx]

Great proposal! Token visibility is crucial for cost management and debugging.

Additional suggestions:

1. Cost estimation alongside tokens

   interface TokenUsage {
     input: number;
     output: number;
     total: number;
     estimatedCost?: number;  // Based on model pricing
   }

   Users care about dollars, not just tokens.

2. Export/analytics hook

   // Allow extensions to subscribe to usage events
   continue.onTokenUsage((usage) => {
     // Send to analytics, budget tracker, etc.
   });

3. Budget alerts

   {
     "continue.tokenBudget": {
       "sessionLimit": 100000,
       "dailyLimit": 500000,
       "warnAt": 0.8
     }
   }

4. Context window visualization
   Show how much of the model context is used:

   [████████░░] 80% context used (32k/40k)
   

   Helps users understand when they are hitting limits.

5. Per-model tracking
   When switching models mid-session, track separately:

   Session: 45k tokens
   - gpt-4o: 30k ($0.15)
   - claude-3: 15k ($0.05)
   

UI placement suggestion:

• Turn-level: subtle inline badge
• Session: status bar item
• History: column in history view

We build token-aware AI tools at Revolution AI — budget visibility is one of the most requested features from enterprise users. Looking forward to the PR!

[xXMrNidaXx]

+1 for persistent token tracking! Essential for cost management.

Great design decisions:

• Escalating display levels (never → turn)
• Always-on tracking, display-only setting
• No schema migration

Additional suggestions:

1. Cost estimation

interface TokenUsage {
  promptTokens: number;
  completionTokens: number;
  estimatedCost?: number;  // Based on model pricing
}

2. Export capability

// Allow export for billing/analysis
{
  "sessions": [
    {
      "id": "abc",
      "totalTokens": 15000,
      "turns": [
        {"prompt": 500, "completion": 200}
      ]
    }
  ]
}

3. Budget alerts

continue:
  tokenBudget:
    daily: 100000
    alertAt: 80%  # Warn at 80% usage

4. Per-model breakdown

Session tokens:
  gpt-4o: 8,500 ($0.04)
  claude-3: 3,200 ($0.02)
  local/qwen: 12,000 (free)

UI placement suggestion:

• Turn level: Small badge next to response
• Session: Status bar item
• History: Column in history view

We track AI costs at Revolution AI — this feature would be huge for enterprise users managing budgets.