Skip to content

feat(type-parsing): use oxc-parser#887

Open
avivkeller wants to merge 5 commits into
mainfrom
typescript
Open

feat(type-parsing): use oxc-parser#887
avivkeller wants to merge 5 commits into
mainfrom
typescript

Conversation

@avivkeller

@avivkeller avivkeller commented Jul 8, 2026

Copy link
Copy Markdown
Member
Screenshot 2026-07-08 at 2 20 39 PM Screenshot 2026-07-08 at 2 21 14 PM

@vercel

vercel Bot commented Jul 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
api-docs-tooling Ready Ready Preview Jul 13, 2026 2:37pm

Request Review

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.44569% with 70 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.28%. Comparing base (4f07fee) to head (52e418d).

Files with missing lines Patch % Lines
src/generators/metadata/utils/resolveTypes.mjs 66.66% 52 Missing ⚠️
src/utils/remark.mjs 62.06% 11 Missing ⚠️
src/generators/man-page/utils/converter.mjs 25.00% 3 Missing ⚠️
src/generators/legacy-html/utils/buildContent.mjs 0.00% 1 Missing ⚠️
src/utils/queries/utils.mjs 50.00% 1 Missing ⚠️
src/utils/type-annotations/hast.mjs 99.11% 1 Missing ⚠️
src/utils/unist.mjs 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #887      +/-   ##
==========================================
+ Coverage   85.19%   85.28%   +0.09%     
==========================================
  Files         180      186       +6     
  Lines       16570    16837     +267     
  Branches     1514     1496      -18     
==========================================
+ Hits        14116    14359     +243     
- Misses       2444     2470      +26     
+ Partials       10        8       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment on lines +43 to +47
current.textRaw = transformNodesToString(
child.children.filter(node => node !== subList)
)
.replace(/\s+/g, ' ')
.replace(/<!--.*?-->/gs, '');
// Shiki tokenizes `vm`, `.`, `Module` separately; the whole range links
const anchors = html.match(/href="vm.html#class-module"/g);
assert.ok(anchors.length >= 1);
assert.match(html.replace(/<[^>]+>/g, ''), /^vm\.Module$/);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't care we are on test and this code will took more computing time so we didn't have any interest of doing that

@avivkeller avivkeller marked this pull request as ready for review July 8, 2026 21:20
@avivkeller avivkeller requested a review from a team as a code owner July 8, 2026 21:20
@cursor

cursor Bot commented Jul 8, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
This is a broad change to how API docs parse and link types across all generators; incorrect parsing or link ranges would affect published documentation, though behavior is covered by new unit tests and MDX is explicitly excluded from the new pass.

Overview
Replaces regex/string rewriting of {Type} prose with first-class typeAnnotation mdast nodes (balanced {...} spans parsed by a new Remark/micromark extension on non-MDX pipelines) and TypeScript-aware resolution via oxc-parser, batched per document in resolveTypeAnnotations.

Resolution and output now follow spec §8: optional whole-value type-map match, then per-identifier links (type map → builtins → MDN → dotted-name heuristic); parse failures warn and render as plain code. Rendering is a single inline <code class="type"> with range-based <a class="type-link"> spans; the web/Recma pipeline can Shiki-highlight the fragment with link decorations.

Downstream generators (metadata, legacy-json, jsx-ast, man-page, legacy-html) consume typeAnnotation instead of link-wrapped `<T>` nodes; the custom typeParser.mjs and text normalizeTypes / visitTextWithTypeNode paths are removed. docs/specification.md bumps to 1.0.1 and rewrites type-annotation rules accordingly.

Reviewed by Cursor Bugbot for commit 52e418d. Bugbot is set up for automated code reviews on this repo. Configure here.

@avivkeller

Copy link
Copy Markdown
Member Author

The diff was too big to leave as a comment

Comment thread src/utils/type-annotations/hast.mjs

@bmuenzenmeyer bmuenzenmeyer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a first pass

Comment thread docs/specification.md

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This changes the version of the spec right? How have we been handling this?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be the first change to the spec, however, it's more of a change in wording that behavior. The only behavioral change is that {string}|{boolean} must now be {string|boolean}. Previously, the spec didn't explicitly say whether to do this or not to do this, now it explicitly says to keep all types with the {...} brackets.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1.0.0 -> 1.1.0 sounds good?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am less concerned if we think its still valid 1.0.0 due to omission - but we should develop some governance over what constitutes a change. imagine semver would work in moooooooost cases

Comment thread package.json Outdated
@avivkeller

Copy link
Copy Markdown
Member Author

@bmuenzenmeyer i ended up patching instead of minoring since we previously claim to support these types and similar ones, and this is more of an improvement on that claim than a new feature of the spec.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want fixes drafted automatically? Bugbot Autofix can create code changes for findings. A team admin can enable Autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 046386f. Configure here.

Comment thread src/generators/metadata/utils/resolveTypes.mjs Outdated
@avivkeller avivkeller mentioned this pull request Jul 11, 2026
@avivkeller avivkeller requested a review from bmuenzenmeyer July 13, 2026 00:05
@avivkeller

Copy link
Copy Markdown
Member Author

Bump @nodejs/web-infra

@avivkeller avivkeller linked an issue Jul 13, 2026 that may be closed by this pull request
@AugustinMauroy

AugustinMauroy commented Jul 13, 2026

Copy link
Copy Markdown
Member

do you know what the impact in terms:

  • of doc-kit npm size ?
  • since it's a fully parsing job it's ask more computing so what happened to perf

Is it possible to achieve this with other parser such as oxc/swc which is lighter that full typescript package

somethign like that:

import { parseSync, Visitor } from "oxc-parser";

function parseTypeValues(values: string[]) {
  // Same trick as today: wrap each type in a synthetic alias.
  const source = values
    .map((value, i) => `type $T${i} = ${value};`)
    .join("\n");

  const result = parseSync("types.ts", source);

  if (result.errors.length > 0) {
    // Same fallback strategy as today:
    // parse each type individually to isolate invalid annotations.
    return fallback(values);
  }

  const identifiersPerAlias = values.map(() => []);

  const visitor = new Visitor({
    TSTypeReference(node) {
      // Promise<T> / vm.Module / Foo.Bar
      record(node.typeName);
    },

    TSTypeQuery(node) {
      // typeof Foo
      record(node.exprName);
    },

    TSImportType(node) {
      // import("fs").Stats
      record(node.qualifier);
    },

    TSThisType(node) {
      record(node);
    },

    TSLiteralType(node) {
      // null
      if (node.literal.type === "NullLiteral") {
        record(node);
      }
    },

    TSStringKeyword: record,
    TSNumberKeyword: record,
    TSBooleanKeyword: record,
    TSAnyKeyword: record,
    TSUnknownKeyword: record,
    TSNeverKeyword: record,
    TSVoidKeyword: record,
    TSUndefinedKeyword: record,
    TSObjectKeyword: record,
    TSBigIntKeyword: record,
    TSSymbolKeyword: record,
  });

  visitor.visit(result.program);

  return identifiersPerAlias;

  function record(node) {
    // node.range / node.span provides source offsets
    // determine which synthetic alias this belongs to
    // convert to offsets relative to the original annotation
    // push { start, end, text, lookup }
  }
}

@avivkeller avivkeller changed the title feat(type-parsing): use typescript feat(type-parsing): use oxc-parser Jul 13, 2026
@avivkeller

Copy link
Copy Markdown
Member Author

@AugustinMauroy nice suggestion! Moved to oxc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

typeParser.mjs should place functions within code blocks.

4 participants