Follow up to discussion #1399, second of the two issues.
Test file, already attached to the discussion: combining_separate_tj.pdf (888 bytes). It draws a base letter plus a combining mark five times, varying only how the two glyphs are emitted.
What happens
The combining branch in ContentStreamProcessor.RenderGlyph is guarded by attachTo.TextSequence == TextSequence (ContentStreamProcessor.cs:140 on v0.1.15). The same two lines on 0.1.13 (baeac0d) are 113 and 117, with identical text. The combining code itself was never touched: only the meaning of TextSequence changed, so once every Tj increments the counter a base glyph and its mark drawn by two separate Tj can never be in the same sequence.
page.Letters on the attached file:
| how the glyphs are emitted |
0.1.13 |
0.1.14 |
0.1.15 |
(e) Tj then (acute) Tj |
combined |
separate |
separate |
(e+acute) Tj |
combined |
combined |
combined |
[(e) 0 (acute)] TJ |
combined |
separate |
combined |
| total page.Letters.Count |
5 |
8 |
7 |
fee5ce1 repaired the TJ case. The two separate Tj case is still open on 0.1.15.
When it fails, the mark becomes a Letter of its own at x = 63.3 instead of 50, with a width of 6 points against 13.3 for the base, and the base keeps its own bounding box instead of being replaced by the composed letter. page.Text and GetWords() are unchanged on this file, so only callers that walk page.Letters see it.
Note for unit tests
src/UglyToad.PdfPig.Tests/Integration/Documents/Diacritics_export.pdf cannot catch this. Its accented characters are precomposed, U+00ED and U+00F1, and it yields the same 24 letters on both versions, so it never enters the combining branch. A test that covers this needs a document where the mark is a real combining codepoint emitted by its own show operation, which is what the attached file does.
My reading of the code, not measured
The TextSequence equality is now the only effective condition on that branch. The other guard, currentOffset > 0, is always true inside the glyph loop: currentOffset comes from bytes.CurrentOffset (BaseStreamProcessor.cs:337) and MemoryInputBytes defines CurrentOffset as currentOffset + 1 starting from -1 (MemoryInputBytes.cs:25 and :30), so it is at least 1 for the first byte of a string. That matches the first row of the table, where on 0.1.13 the mark combines even though it is the only byte of its own string.
Follow up to discussion #1399, second of the two issues.
Test file, already attached to the discussion: combining_separate_tj.pdf (888 bytes). It draws a base letter plus a combining mark five times, varying only how the two glyphs are emitted.
What happens
The combining branch in ContentStreamProcessor.RenderGlyph is guarded by attachTo.TextSequence == TextSequence (ContentStreamProcessor.cs:140 on v0.1.15). The same two lines on 0.1.13 (baeac0d) are 113 and 117, with identical text. The combining code itself was never touched: only the meaning of TextSequence changed, so once every Tj increments the counter a base glyph and its mark drawn by two separate Tj can never be in the same sequence.
page.Letters on the attached file:
(e) Tjthen(acute) Tj(e+acute) Tj[(e) 0 (acute)] TJfee5ce1 repaired the TJ case. The two separate Tj case is still open on 0.1.15.
When it fails, the mark becomes a Letter of its own at x = 63.3 instead of 50, with a width of 6 points against 13.3 for the base, and the base keeps its own bounding box instead of being replaced by the composed letter. page.Text and GetWords() are unchanged on this file, so only callers that walk page.Letters see it.
Note for unit tests
src/UglyToad.PdfPig.Tests/Integration/Documents/Diacritics_export.pdf cannot catch this. Its accented characters are precomposed, U+00ED and U+00F1, and it yields the same 24 letters on both versions, so it never enters the combining branch. A test that covers this needs a document where the mark is a real combining codepoint emitted by its own show operation, which is what the attached file does.
My reading of the code, not measured
The TextSequence equality is now the only effective condition on that branch. The other guard, currentOffset > 0, is always true inside the glyph loop: currentOffset comes from bytes.CurrentOffset (BaseStreamProcessor.cs:337) and MemoryInputBytes defines CurrentOffset as currentOffset + 1 starting from -1 (MemoryInputBytes.cs:25 and :30), so it is at least 1 for the first byte of a string. That matches the first row of the table, where on 0.1.13 the mark combines even though it is the only byte of its own string.