natsorted(..., alg=ns.REAL) silently loses ordering for values above the float maximum (overflow to inf)

[codewitharyan29]

Summary

Under ns.REAL (and ns.FLOAT), numeric substrings are parsed with Python's
float(). Any value larger than the IEEE-754 double maximum (~1.8e308)
overflows to inf. Two distinct, clearly-ordered values that both overflow
therefore compare equal, and natsorted cannot order them — the output
depends on input order (sort stability), not on the actual numeric magnitude.

Reproduction

>>> from natsort import natsorted, ns
>>> natsorted(["1e400", "1e500"], alg=ns.REAL)
['1e400', '1e500']
>>> natsorted(["1e500", "1e400"], alg=ns.REAL)
['1e500', '1e400']

Both calls "succeed", but the two results disagree about whether 1e400 or
1e500 is larger — because both parsed to inf:

>>> from natsort import natsort_keygen
>>> k = natsort_keygen(alg=ns.REAL)
>>> k("1e400"), k("1e500")
(('', inf), ('', inf))

Mathematically 1e500 > 1e400, but the sort treats them as equal.

Why this matters

The failure is silent — no exception, no warning — and the incorrect order is
plausible-looking. Any dataset containing very large numeric tokens (scientific
data, identifiers with large exponents, synthetic stress inputs) can be sorted
incorrectly with no indication that anything went wrong.

Suggested fix

Two reasonable options:

1. Document the limit. Note in the ns.REAL/ns.FLOAT docs that values are
   limited to IEEE-754 double range and overflow to inf, collapsing order.

2. Parse to an arbitrary-precision type when overflow is detected. When
   float() yields inf but the source string is finite, fall back to
   decimal.Decimal (or compare the normalized string form) so ordering is
   preserved. This is heavier but removes the silent-error class entirely.

How this was found

While building and differentially testing a Rust port of natsort. The Rust port
uses f64 and therefore reproduces the exact same behavior (both values become
inf), so this is not a port defect — it is an upstream characteristic that the
differential testing surfaced. Happy to open a PR for the documentation option
if that is preferred.

Environment

• natsort 8.4.0 (latest release on PyPI at time of writing)
• Python 3.12
• Reproducible on a clean install; no other dependencies involved.

[SethMMorton]

I think that just a documentation update would be sufficient. The estimated number of atoms in the observable universe is on the order of 10^80, so I am OK with just pointing out this limitation in the API given that I do not expect numbers this large in realistic data sets.

[SethMMorton]

On the other hand, if you think that inserting the decimal method would not introduce any sort of performance overhead we could go with that. I'm just worried that adding extra operations for a very rare case is not the right trade-off. Dunno - I could be convinced either way.

[codewitharyan29]

That's a fair concern, and I think the answer is: it shouldn't add any
overhead to the common case at all, since the fallback would only trigger
after float() has already returned inf — i.e., only on the already-rare
overflow path itself. Something like:

val = float(s)
if val == float('inf') or val == float('-inf'):
    val = Decimal(s)  # only reached on overflow

The happy path (99.999...% of real-world sorts) still just calls float()
once, same cost as today. The decimal conversion only runs on inputs that
were already going to be "wrong" under the current behavior, so there's no
regression for normal data — only a fix for the edge case.

I haven't benchmarked it, but happy to put together a quick PR + benchmark
if that would help make the call. Appreciate you taking a look at this!