|
| 1 | +# Language support from the filename, with no per-language edits |
| 2 | + |
| 3 | +**Approved.** Cost accepted (measured below). This replaces the hand-maintained |
| 4 | +language catalog with CodeMirror's published registry, so opening a file of |
| 5 | +*any* language it knows lights up without anyone editing termic first. |
| 6 | + |
| 7 | +Today adding a language is two edits — an entry in `lib/languages.ts` and a |
| 8 | +`case` in `lib/languageExts.ts`. That is how Swift and Groovy/Gradle went in. |
| 9 | +After this, neither file has a list to add to. |
| 10 | + |
| 11 | +## What "automatic" means, precisely |
| 12 | + |
| 13 | +Matching is **by filename**, which is both halves of what that implies: |
| 14 | + |
| 15 | +- **Extension** — `.swift`, `.gradle`, `.php`, `.zig`. |
| 16 | +- **Whole-filename patterns** — `Dockerfile`, `Makefile`, `justfile`, |
| 17 | + `.env.production`. `LanguageDescription.matchFilename()` handles both, so the |
| 18 | + special-case list in `languageIdForPath` goes away with the catalog. |
| 19 | + |
| 20 | +Two things stay in front of it, unchanged, and the precedence order in |
| 21 | +`effectiveLanguageId` does not move: |
| 22 | + |
| 23 | +1. **A manual "Set syntax" pick** still wins over everything. |
| 24 | +2. **The content sniffer** (`lib/detectSyntax.ts`) still answers when the |
| 25 | + filename says nothing — an extension-less file, and every scratchpad |
| 26 | + (GH #244), which by definition has no filename at all. |
| 27 | + |
| 28 | +## The registry |
| 29 | + |
| 30 | +`@codemirror/language-data`: ~150 `LanguageDescription`s, each carrying its |
| 31 | +extensions, filename patterns, aliases, and a lazy loader. |
| 32 | + |
| 33 | +```ts |
| 34 | +const desc = LanguageDescription.matchFilename(languages, "build.gradle"); |
| 35 | +const support = desc && await desc.load(); // dynamic import, code-split |
| 36 | +``` |
| 37 | + |
| 38 | +**Our two extras are registered alongside it**, not replaced by it: nothing |
| 39 | +upstream has a Makefile grammar (`lib/makeMode.ts`), and the shipped protobuf |
| 40 | +mode predates proto3 (`lib/protoMode.ts`). The end state is "the registry plus |
| 41 | +our two", expressed as `LanguageDescription.of(...)` entries prepended to the |
| 42 | +list so ours win on a tie. |
| 43 | + |
| 44 | +## Accepted cost |
| 45 | + |
| 46 | +Measured by wiring the registry into `EditorPane` behind a live call site and |
| 47 | +diffing `npm run build` against the same build without it. (A tree-shaken probe |
| 48 | +measures nothing — the first attempt reported an unchanged bundle because |
| 49 | +rolldown dropped the reference. Re-measure that way if these numbers are ever |
| 50 | +rechecked.) |
| 51 | + |
| 52 | +| | before | after | |
| 53 | +| --- | --- | --- | |
| 54 | +| main `index` chunk | 2300K | **2300K** (unchanged) | |
| 55 | +| editor chunk (first file opened) | 604K | **644K** (+40K) | |
| 56 | +| total `dist/` | 7208K | **8032K** (+824K, 94 lazy chunks) | |
| 57 | +| new npm deps | — | 9 (`language-data` + 8 Lezer grammars we lack) | |
| 58 | + |
| 59 | +App start pays nothing, the first opened file pays 40K for the index, and the |
| 60 | +`.app` carries ~0.8MB of grammars that are only ever *loaded* for a language |
| 61 | +the user actually opens. |
| 62 | + |
| 63 | +**Hold app start at zero.** The registry may only be reachable from the lazily |
| 64 | +loaded editor/diff panes. `lib/languages.ts` is imported by the command palette |
| 65 | +and the breadcrumb, i.e. the main chunk, so it must not import |
| 66 | +`@codemirror/language-data` — pin that with a source-level test, the way |
| 67 | +`cspGuard.test.ts` pins the CSP. |
| 68 | + |
| 69 | +## The work |
| 70 | + |
| 71 | +**1. Async loading.** The awkward part. `langForId` returns an `Extension` |
| 72 | +synchronously today and the editor mounts with it; the registry returns a |
| 73 | +promise. So the editor mounts with no grammar and reconfigures its language |
| 74 | +compartment when the load resolves. The compartment is already the live-switch |
| 75 | +mechanism (Set-syntax uses it), so the machinery exists. |
| 76 | + |
| 77 | +It needs a **cancellation guard**: bump a generation counter per mount/path |
| 78 | +change, capture it before the await, and drop the result if it no longer |
| 79 | +matches. A fast tab switch must never land a resolved grammar in a view that |
| 80 | +has since been rebuilt — that is the async-mount race the no-StrictMode rule |
| 81 | +exists for, and it gets its own test. |
| 82 | + |
| 83 | +**2. Ids become registry names.** `EditTab.syntax` is session-only today, so |
| 84 | +there is nothing to migrate — but fix the contract now, because the scratchpad |
| 85 | +plan persists a pick in its index. Use the registry's `name` ("TypeScript", |
| 86 | +"Makefile"); `languageLabel()` keeps rendering an unknown id verbatim so a |
| 87 | +stored name that later disappears degrades to plain text instead of blank. |
| 88 | +`lib/detectSyntax.ts` returns those names too. |
| 89 | + |
| 90 | +**3. `lib/languages.ts` gets thin.** It keeps `PLAIN_TEXT`, |
| 91 | +`effectiveLanguageId`'s precedence, `languageLabel`, and the note pointing the |
| 92 | +LSP work at it as the one place a `languageId` is decided. The `LANGUAGES` |
| 93 | +array and `languageIdForPath`'s rules are deleted, not ported. |
| 94 | + |
| 95 | +**4. The picker sources from the registry.** `SyntaxPalette` maps |
| 96 | +`LanguageDescription[]` to rows (name + `alias` for the fuzzy search), keeping |
| 97 | +Plain Text pinned first. That is a straight simplification. |
| 98 | + |
| 99 | +**5. `lib/languageExts.ts` loses its switch**, keeping only the two custom |
| 100 | +descriptions and the registry lookup. |
| 101 | + |
| 102 | +## Testing |
| 103 | + |
| 104 | +- **Unit:** the filename cases the current suite already pins must survive the |
| 105 | + swap — `Makefile`, `Dockerfile.dev`, `justfile`, `.env.production`, |
| 106 | + `build.gradle` (Groovy) vs `build.gradle.kts` (Kotlin), `README.MD`. Plus the |
| 107 | + new invariant: nothing in the main chunk's import graph pulls in |
| 108 | + `language-data`. |
| 109 | +- **Async race:** mount, switch path before the load resolves, assert the stale |
| 110 | + grammar never lands. |
| 111 | +- **e2e:** the existing `editor.e2e.ts` syntax cases stay as they are — they |
| 112 | + assert *rendered tokens*, so they prove the async path really reaches |
| 113 | + CodeMirror. Add one case for a language nobody hand-added (PHP or Lua): open |
| 114 | + the file, get highlighting, with zero termic-side registration. That is the |
| 115 | + whole feature in one assertion. |
| 116 | + |
| 117 | +## Out of scope |
| 118 | + |
| 119 | +Highlighting inside terminal output; anything LSP (`docs/ideas/lsp.md` owns |
| 120 | +that, and should read its `languageId` from the same place); shipping grammars |
| 121 | +the registry does not include. |
0 commit comments