fix: orderable fractional indexing case-sensitivity issue with PostgreSQL

[vladyslavprosolupov]

fix: orderable fractional indexing case-sensitivity issue with PostgreSQL

Problem

When using the orderable feature with PostgreSQL, reordering documents to the "first" position generates keys that break subsequent reordering operations.

Scenario

1. Create two orderable documents → keys are a0, a1
2. Drag a1 before a0 (to make it first)
3. The system generates key Zz for the moved document
4. Result: Ordering is completely broken - cannot reorder any documents anymore

Root Cause

The fractional indexing algorithm uses:

• Uppercase A-Z for "smaller" integer keys
• Lowercase a-z for "larger" integer keys

This relies on ASCII ordering where 'Z' (code 90) < 'a' (code 97), so Zz < a0.

However, PostgreSQL's default collation (en_US.UTF-8) uses case-insensitive comparison, treating 'Z' as 'z'. This means:

• ASCII: Zz < a0 ✓
• PostgreSQL: Zz treated as zz, so zz > a0 ✗

This mismatch causes:

1. Database queries return incorrect adjacent documents
2. The generateKeyBetween function receives arguments in wrong order
3. Subsequent reorder attempts either error out or create more broken keys

Solution

Modified the fractional indexing algorithm to use only characters that sort consistently across all database collations:

Range	Old Encoding	New Encoding
Small keys	A-Z (uppercase)	0-9 (digits)
Large keys	a-z (lowercase)	a-z (lowercase, unchanged)

Key insight: Digits (0-9) always sort before letters in both ASCII ordering and case-insensitive collations.

Before (broken)

decrementInteger('a0') → 'Zz'
'Zz' < 'a0' in ASCII ✓
'Zz' > 'a0' in PostgreSQL (case-insensitive) ✗

After (fixed)

decrementInteger('a0') → '9z'
'9z' < 'a0' in ASCII ✓
'9z' < 'a0' in PostgreSQL ✓
'9z' < 'a0' in MongoDB ✓
'9z' < 'a0' in SQLite ✓

Changes

packages/payload/src/config/orderable/fractional-indexing.js

• Changed integer part encoding to use digits for "small" range
• Maintains backward compatibility with existing a-z keys
• Legacy A-Z keys are still parsed (for backward compatibility) but won't be generated

Backward Compatibility

• ✅ Existing keys starting with a-z continue to work correctly
• ⚠️ Existing keys starting with A-Z (uppercase) will be parsed but may sort incorrectly in case-insensitive databases. Users with such keys should run a migration to regenerate them.

Testing

Verified the algorithm produces correct ordering:

generateKeyBetween(null, null)     // → 'a0'
generateKeyBetween(null, 'a0')     // → '9z' (previously 'Zz')
generateKeyBetween(null, '9z')     // → '9y'

// Sorting works correctly
['9z', 'a0', 'a1'].sort()          // → ['9z', 'a0', 'a1'] ✓

Related

• Fractional Indexing algorithm: https://observablehq.com/@dgreensp/implementing-fractional-indexing
• Original library: https://github.com/rocicorp/fractional-indexing

[DanRibbens]

Thanks for the PR! I actually thought that we fixed this some time ago.

Assuming all existing tests pass, just a few things need to happen before this can be merged.

• remove the excessive comments generated by AI.
• add test in test/sort/int.spec.ts

[vladyslavprosolupov]

Using base 36 because fractional indexing keys use characters 0-9 and a-z.
Base 16 would fail for keys like '9z' (returning just 9) since 'z' is not a valid hex character.

Also changed it in other tests for cohesion, it breaks nothing.

[hohoaisan]

The ordering still broken, when will this issue is resolved :(

[paulpopus]

@hohoaisan sorry about that, will bring this up to the team

[andershermansen]

Some of us generates the order outside of payload admin system and would need a changed function like this to be exported.

[DanRibbens]

Looks good!

[paulpopus]

Exporting the utilities here #15286

[danielwaltz]

To the best of my knowledge and understanding, Postgres's sort actually is case sensitive by default. This feels to me like it may be a case of AI hallucination?

When I have an orderable field in Payload and open the database in my database client and sort by the _order column I see entries that start with Zz above entries with a0.

https://aws.amazon.com/blogs/database/manage-case-insensitive-data-in-postgresql/#:~:text=By%20default%2C%20PostgreSQL%20is%20case,impacts%20how%20they%20are%20sorted.

EDIT: After further digging, I think it may be OS dependent. Postgres seems to defer to the OS on this particular case, and there are differences between macOS and Debian (and perhaps other Linux distros as well).

[github-actions]

🚀 This is included in version v3.73.0