Skip to content

Commit 0af79b7

Browse files
simionclaude
andcommitted
docs(plans): promote the language registry, cost approved
Moves editor-language-registry from ideas/ to plans/ and rewrites the header from "is this worth it" into a spec. The measurements stay, as the accepted cost rather than the open question: app start unchanged, +40K on the editor chunk, +824K in the .app across 94 lazy chunks. Says what "automatic" means, since that was the question behind the approval: matching is by FILENAME, which is extensions AND whole-filename patterns (Dockerfile, Makefile, justfile, .env.production), so `languageIdForPath`'s special-case list goes away with the catalog. A manual Set-syntax pick still wins, and the content sniffer still answers when the filename says nothing — which is every scratchpad, by definition. Also pins the two things that keep this honest: our Makefile and proto3 grammars are registered ALONGSIDE the registry rather than replaced by it (nothing upstream has a Makefile grammar), and `lib/languages.ts` may not import language-data, since the palette imports it into the main chunk — a source-level test, like cspGuard. No `planned` issue opened: the issue rule says never to open one for a doc I just wrote. Ready to open it on your word, and it needs a README Planned bullet, which is maintainer-only. Refs #244 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F2JzhUsFa9YLjBBvUJQGYj
1 parent 682f276 commit 0af79b7

2 files changed

Lines changed: 121 additions & 102 deletions

File tree

docs/ideas/editor-language-registry.md

Lines changed: 0 additions & 102 deletions
This file was deleted.
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# Language support from the filename, with no per-language edits
2+
3+
**Approved.** Cost accepted (measured below). This replaces the hand-maintained
4+
language catalog with CodeMirror's published registry, so opening a file of
5+
*any* language it knows lights up without anyone editing termic first.
6+
7+
Today adding a language is two edits — an entry in `lib/languages.ts` and a
8+
`case` in `lib/languageExts.ts`. That is how Swift and Groovy/Gradle went in.
9+
After this, neither file has a list to add to.
10+
11+
## What "automatic" means, precisely
12+
13+
Matching is **by filename**, which is both halves of what that implies:
14+
15+
- **Extension**`.swift`, `.gradle`, `.php`, `.zig`.
16+
- **Whole-filename patterns**`Dockerfile`, `Makefile`, `justfile`,
17+
`.env.production`. `LanguageDescription.matchFilename()` handles both, so the
18+
special-case list in `languageIdForPath` goes away with the catalog.
19+
20+
Two things stay in front of it, unchanged, and the precedence order in
21+
`effectiveLanguageId` does not move:
22+
23+
1. **A manual "Set syntax" pick** still wins over everything.
24+
2. **The content sniffer** (`lib/detectSyntax.ts`) still answers when the
25+
filename says nothing — an extension-less file, and every scratchpad
26+
(GH #244), which by definition has no filename at all.
27+
28+
## The registry
29+
30+
`@codemirror/language-data`: ~150 `LanguageDescription`s, each carrying its
31+
extensions, filename patterns, aliases, and a lazy loader.
32+
33+
```ts
34+
const desc = LanguageDescription.matchFilename(languages, "build.gradle");
35+
const support = desc && await desc.load(); // dynamic import, code-split
36+
```
37+
38+
**Our two extras are registered alongside it**, not replaced by it: nothing
39+
upstream has a Makefile grammar (`lib/makeMode.ts`), and the shipped protobuf
40+
mode predates proto3 (`lib/protoMode.ts`). The end state is "the registry plus
41+
our two", expressed as `LanguageDescription.of(...)` entries prepended to the
42+
list so ours win on a tie.
43+
44+
## Accepted cost
45+
46+
Measured by wiring the registry into `EditorPane` behind a live call site and
47+
diffing `npm run build` against the same build without it. (A tree-shaken probe
48+
measures nothing — the first attempt reported an unchanged bundle because
49+
rolldown dropped the reference. Re-measure that way if these numbers are ever
50+
rechecked.)
51+
52+
| | before | after |
53+
| --- | --- | --- |
54+
| main `index` chunk | 2300K | **2300K** (unchanged) |
55+
| editor chunk (first file opened) | 604K | **644K** (+40K) |
56+
| total `dist/` | 7208K | **8032K** (+824K, 94 lazy chunks) |
57+
| new npm deps || 9 (`language-data` + 8 Lezer grammars we lack) |
58+
59+
App start pays nothing, the first opened file pays 40K for the index, and the
60+
`.app` carries ~0.8MB of grammars that are only ever *loaded* for a language
61+
the user actually opens.
62+
63+
**Hold app start at zero.** The registry may only be reachable from the lazily
64+
loaded editor/diff panes. `lib/languages.ts` is imported by the command palette
65+
and the breadcrumb, i.e. the main chunk, so it must not import
66+
`@codemirror/language-data` — pin that with a source-level test, the way
67+
`cspGuard.test.ts` pins the CSP.
68+
69+
## The work
70+
71+
**1. Async loading.** The awkward part. `langForId` returns an `Extension`
72+
synchronously today and the editor mounts with it; the registry returns a
73+
promise. So the editor mounts with no grammar and reconfigures its language
74+
compartment when the load resolves. The compartment is already the live-switch
75+
mechanism (Set-syntax uses it), so the machinery exists.
76+
77+
It needs a **cancellation guard**: bump a generation counter per mount/path
78+
change, capture it before the await, and drop the result if it no longer
79+
matches. A fast tab switch must never land a resolved grammar in a view that
80+
has since been rebuilt — that is the async-mount race the no-StrictMode rule
81+
exists for, and it gets its own test.
82+
83+
**2. Ids become registry names.** `EditTab.syntax` is session-only today, so
84+
there is nothing to migrate — but fix the contract now, because the scratchpad
85+
plan persists a pick in its index. Use the registry's `name` ("TypeScript",
86+
"Makefile"); `languageLabel()` keeps rendering an unknown id verbatim so a
87+
stored name that later disappears degrades to plain text instead of blank.
88+
`lib/detectSyntax.ts` returns those names too.
89+
90+
**3. `lib/languages.ts` gets thin.** It keeps `PLAIN_TEXT`,
91+
`effectiveLanguageId`'s precedence, `languageLabel`, and the note pointing the
92+
LSP work at it as the one place a `languageId` is decided. The `LANGUAGES`
93+
array and `languageIdForPath`'s rules are deleted, not ported.
94+
95+
**4. The picker sources from the registry.** `SyntaxPalette` maps
96+
`LanguageDescription[]` to rows (name + `alias` for the fuzzy search), keeping
97+
Plain Text pinned first. That is a straight simplification.
98+
99+
**5. `lib/languageExts.ts` loses its switch**, keeping only the two custom
100+
descriptions and the registry lookup.
101+
102+
## Testing
103+
104+
- **Unit:** the filename cases the current suite already pins must survive the
105+
swap — `Makefile`, `Dockerfile.dev`, `justfile`, `.env.production`,
106+
`build.gradle` (Groovy) vs `build.gradle.kts` (Kotlin), `README.MD`. Plus the
107+
new invariant: nothing in the main chunk's import graph pulls in
108+
`language-data`.
109+
- **Async race:** mount, switch path before the load resolves, assert the stale
110+
grammar never lands.
111+
- **e2e:** the existing `editor.e2e.ts` syntax cases stay as they are — they
112+
assert *rendered tokens*, so they prove the async path really reaches
113+
CodeMirror. Add one case for a language nobody hand-added (PHP or Lua): open
114+
the file, get highlighting, with zero termic-side registration. That is the
115+
whole feature in one assertion.
116+
117+
## Out of scope
118+
119+
Highlighting inside terminal output; anything LSP (`docs/ideas/lsp.md` owns
120+
that, and should read its `languageId` from the same place); shipping grammars
121+
the registry does not include.

0 commit comments

Comments
 (0)