Skip to content

fix: preserve paragraph structure when line comments separate inline text - #9

Merged
sou1118 merged 1 commit into
sou1118:mainfrom
colluca:fix-parbreaks
Aug 4, 2026
Merged

fix: preserve paragraph structure when line comments separate inline text#9
sou1118 merged 1 commit into
sou1118:mainfrom
colluca:fix-parbreaks

Conversation

@colluca

@colluca colluca commented May 14, 2026

Copy link
Copy Markdown
Contributor

Problem

When a // line comment sits on its own line between two sentences with no blank line, the compiled diff output incorrectly introduces a paragraph break between them.

First sentence.
// a comment
Second sentence.

Typst compiles the original correctly as one paragraph. The diff output breaks it into two.

Root cause

typst_syntax tokenises \n// comment\n as three CST nodes: Space("\n"), LineComment(…), Space("\n"). Because markup.exprs() drops LineComment nodes (they carry no semantic content), both surrounding Space("\n") nodes are exposed to the iterator.

typdiff concatenates their raw text into the paragraph buffer, producing "first sentence.\n\nsecond sentence.". When that string is written to the diff output file and compiled by Typst, the bare \n\n is a Parbreak — a paragraph break that was never in the original source.

Note: this does not affect Typst itself, which processes the AST directly through its layout engine and never reconstructs source text from individual nodes.

Fix

Split Expr::Space out of the combined inline arm and skip a Space("\n") node when it would append \n to a buffer already ending with \n. This is always safe: bare \n\n in source is always tokenised as a Parbreak token, never as two consecutive Space nodes — so two adjacent Space("\n") in the expression iterator can only arise from a dropped comment between them.

Expr::Space(_) => {
    let text = node_text(&expr);
    if !(text == "\n" && paragraph_buf.ends_with('\n')) {
        paragraph_buf.push_str(&text);
    }
}
` ` `

## Edge cases

| Source pattern | Before | After |
|---|---|---|
| `text\n// comment\ntext` | paragraph break ✗ | same paragraph ✓ |
| `text\n// c1\n// c2\ntext` | paragraph break ✗ | same paragraph ✓ |
| `text\n\ntext` (blank line) | paragraph break ✓ | paragraph break ✓ |
| `text\n\n// comment\ntext` | paragraph break ✓ | paragraph break ✓ |

@sou1118
sou1118 self-requested a review August 4, 2026 16:54

@sou1118 sou1118 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colluca
Sorry for the slow review, and thank you for the excellent contribution — LGTM!
The root-cause analysis is spot on, and I verified locally that the fix restores the paragraph structure while keeping intentional paragraph breaks intact.
One small note for a follow-up (not blocking): an indented line comment (\n // comment\n) still leaves a whitespace-only line in the output, which Typst treats as a paragraph break. I'll open an issue to track it.

@sou1118 sou1118 changed the title Preserve paragraph structure when line comments separate inline text fix: preserve paragraph structure when line comments separate inline text Aug 4, 2026
@sou1118
sou1118 merged commit 51165f3 into sou1118:main Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants