fix: Hebrew/Arabic diacritics support + highlight corrections

[david-hoze]

Issue filed first: Closes #373

Problem

With "Ignore diacritics" enabled, Hebrew (and Arabic) search is broken in multiple ways:

1. No diacritics stripping — Searching תקון doesn't find תִּקּוּן because Hebrew nikud (and Arabic harakat) aren't removed by the current \p{Diacritic} approach, which only covers Latin combining marks.

2. Wrong highlights — Even when results are found, highlights land on the wrong words (e.g. unrelated word parts scattered in excerpts), because match indices from the normalized (shorter) text are applied to the original (longer) text with nikud, cutting the wrong character spans. see:

3. Unrelated words highlighted — foundWords was populated from MiniSearch's result.terms, which includes fuzzy expansions, so words the user never searched for get highlighted.

Root causes and fixes

Fix 1 — Diacritics stripping (src/tools/utils.ts)

removeDiacritics() uses \p{Diacritic} after NFD normalization, which covers Latin combining marks but not Hebrew nikud (U+0591–U+05C7) or Arabic harakat. These characters aren't decomposed by NFD, so they survive stripping.

→ Add explicit stripping of Hebrew nikud (U+0591–U+05BD, U+05BF–U+05C7) and, when ignoreArabicDiacritics is on, Arabic harakat. Maqaf (U+05BE ־) is deliberately excluded — it's a hyphen, not a diacritic; stripping it would merge words like אֶת־הָאָרֶץ into אתהארץ.

Fix 2 — Match on original text (src/tools/text-processing.ts)

The highlight code ran the match regex on normalized (shorter) text but used the resulting indices to slice the original (longer) text. Indices don't line up, so the wrong span was highlighted.

→ When "Ignore diacritics" is on, build a regex that matches each query word with optional diacritics between base letters and run it on the original text. Then match.index and match[0] refer to the real positions and the actual matched string. The regex does not use \b word boundaries — they're unreliable with Hebrew in many JS engines; the pattern is matched anywhere, and prefix matches (e.g. תק → תקון) still work.

Fix 3 — Highlight from query terms, not result.terms (src/search/search-engine.ts)

foundWords was populated from MiniSearch's result.terms, which includes fuzzy expansions (e.g. תקיש when searching תקון).

→ Use query.query.text, query.getExactTerms(), and query.getTags() for foundWords instead. Only the user's actual search terms are highlighted. Deduplicated with new Set().

Fix 4 — Best match for excerpt (src/tools/text-processing.ts)

The "best match" logic used text.indexOf() on normalized text, producing an index that didn't correspond to the original text.

→ Find the best match by comparing the normalized version of each actual match to the query, then promote that match to the front of the results array.

Testing

• תקון → finds תִּקּוּן in pointed text ✓
• Latin diacritics (café → cafe) still work ✓
• Japanese/Korean unaffected ✓
• Maqaf preserved: אֶת־הָאָרֶץ doesn't merge into אתהארץ ✓

Adherence to contributing guidelines

• Issue first: Closes [Feature request] Correctly clean Arabic diacritics #373.
• No new dependencies. Changes are in existing TS utilities and search logic.
• No UI changes. No Svelte or CSS; existing settings and defaults unchanged.
• Comments: Intent and Unicode ranges commented in code (what, why).
• Philosophy: Improves "smartness" — simple queries like תקון now bring relevant results and correct highlights; no extra interactions or toggles; existing 100-match / 50ms limits keep results fast.
• Style: Only .ts files modified; formatted with Prettier ESLint.