fix(kana): normalize full-width and invisible chars in romaji answer … - #26301
fix(kana): normalize full-width and invisible chars in romaji answer …#26301Aadityaotaku wants to merge 3 commits into
Conversation
…validation Mobile keyboards (especially on Android) sometimes produce full-width romaji characters (e.g. hitoro instead of hitoro) or inject invisible Unicode characters (zero-width space U+200B, BOM U+FEFF). The previous NFC normalization did not convert full-width latin letters to their standard half-width equivalents, causing correct answers to be rejected as wrong. Fixes: - Replace NFC with NFKC normalization so full-width characters are mapped to their ASCII equivalents before comparison - Strip zero-width space (U+200B-U+200D) and BOM (U+FEFF) characters that some IMEs inject silently Fixes lingdojo#24308
🎉 Thanks for your Pull Request, @Aadityaotaku!We appreciate your contribution to KanaDojo! Pre-merge checklist:
A maintainer will review your PR shortly. In the meantime, make sure all CI checks pass. You can run ありがとうございます! 🙏 |
There was a problem hiding this comment.
Pull request overview
This PR fixes incorrect answer rejection in Kana “Input” mode caused by mobile IMEs producing full-width romaji characters and/or injecting invisible Unicode characters into the user’s input. It updates the input/answer normalization logic in features/Kana/lib/ so comparisons behave consistently across devices.
Changes:
- Switches Unicode normalization from
NFCtoNFKCfor romaji/kana comparisons (enabling full-width → ASCII compatibility folding). - Strips invisible characters (U+200B–U+200D and U+FEFF) from the user input before comparison.
- Applies the updated normalization consistently across reverse-mode, direct-match, and alternative-answer matching paths.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (4)
features/Kana/lib/isKanaInputAnswerCorrect.ts:52
- The comment above the normalization logic still describes NFC/NFD normalization, but the implementation now removes specific invisible Unicode characters and applies NFKC (compatibility) normalization. Updating the comment will prevent future readers from assuming NFC behavior (e.g., no full-width → ASCII mapping).
// Normalize Unicode form and strip surrounding whitespace so that input
// from an IME or copy-paste (which may arrive in a different NFC/NFD form)
// is compared on equal footing with the stored answer.
const normalizedInput = inputValue.trim().replace(/[\u200B-\u200D\uFEFF]/g, '').normalize('NFKC');
features/Kana/tests/isKanaInputAnswerCorrect.test.ts:262
- This test’s comment says it is using full-width uppercase Latin letters (U+FF2B/U+FF21), but the actual input uses the lowercase code points (U+FF4B/U+FF41). Either update the comment or (preferably) switch the input to the mentioned code points to also cover the toLowerCase path.
// K (U+FF2B) A (U+FF21) are full-width Latin letters an IME may emit
expect(
isKanaInputAnswerCorrect({
inputValue: '\uFF4B\uFF41', // ka in full-width
correctChar: 'か',
features/Kana/tests/isKanaInputAnswerCorrect.test.ts:336
- The comment says the input includes a trailing U+FEFF, but the string literal currently does not include it. This makes the test less representative of the stated scenario.
// U+200B before, U+200D in middle, U+FEFF after
expect(
isKanaInputAnswerCorrect({
inputValue: '\u200Bs\u200Di\u200B',
correctChar: 'し',
features/Kana/tests/isKanaInputAnswerCorrect.test.ts:386
- This reverse-mode test name/comments mention “full-width kana input”, but the test input is regular kana with invisible characters around it. Renaming it to match what’s actually being validated will make failures easier to interpret.
it('normalises full-width kana input in reverse mode', () => {
// Full-width a (U+FF41) would not appear in kana reverse mode in practice,
// but NFKC is applied uniformly; verify it does not break a correct kana answer.
expect(
Signed-off-by: Aaditya parab <parabaaditya785@student.sfit.ac.in>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Suppressed comments (2)
features/Kana/tests/isKanaInputAnswerCorrect.test.ts:412
- This test name/comment refers to “full-width kana input” and shows an example full-width Latin 'a', but the actual input under test is hiragana with invisible characters (U+200B/U+200D). Renaming/rewording this keeps the test intent clear and avoids confusion for future debugging.
it('normalises full-width kana input in reverse mode', () => {
// Full-width a (U+FF41) would not appear in kana reverse mode in practice,
// but NFKC is applied uniformly; verify it does not break a correct kana answer.
features/Kana/lib/isKanaInputAnswerCorrect.ts:63
lowerTargetis normalized with NFKC, butnormalizedInput(and reverse-mode comparison) still uses NFC and does not strip IME-injected invisible characters. This means full-width romaji (e.g. hitoro) and U+200B/U+FEFF inputs will still fail to match, and the new normalization tests will fail. Normalize/strip the input (and reverse-mode target) consistently using the same pipeline (strip invisibles + strip whitespace + NFKC), then lowercase for romaji comparisons.
// Normal mode: user types romaji. Compare case- and Unicode-insensitively.
const lowerInput = normalizedInput.toLowerCase();
const lowerTarget = targetChar.toLowerCase().normalize('NFKC');
📝 Description
Fixes a bug where typing correct romaji (e.g.
hitorofor ひとろ) was rejected as wrong on mobile devices.Root cause: Android and other mobile IMEs/keyboards sometimes produce full-width romaji characters (e.g.
hitoroinstead ofhitoro) or inject invisible Unicode characters like zero-width space (U+200B) and BOM (U+FEFF) into the input. The previousNFCnormalization does not convert full-width latin letters to their half-width ASCII equivalents, so the comparison always failed.Changes in
isKanaInputAnswerCorrect.ts:NFC→NFKCso full-width characters (e.g.h→h) are mapped to ASCII equivalents before comparisonU+200B–U+200D) and BOM (U+FEFF) that some IMEs silently injectBoth the normal (kana → romaji) and reverse (romaji → kana) modes are fixed, as well as single-char and multi-char prompt paths.
🔗 Related Issue
Closes #24308
✅ Pre-Submission Checklist
cn()utility where neededmainbranch🎯 Type of Change
fix: Bug fix (non-breaking change which fixes an issue)🧪 How Has This Been Tested?
Test Steps:
hitorohitorostill works correctlyManual Test Checklist:
📸 Screenshots/Videos (if applicable)
Before: Typing
hitoro(or full-width equivalent from mobile) for ひとろ was rejected with "Wrong! Correct answer: hitoro"After: Both
hitoroandhitoroare accepted as correct📦 Additional Context
The fix is a one-line change to the normalization call — switching
.normalize('NFC')to.replace(/[\u200B-\u200D\uFEFF]/g, '').normalize('NFKC')— applied consistently across all four comparison paths in the function.