Skip to content

fix findTriTriDistance in case of finite upDistLimitSq#6357

Merged
Fedr merged 2 commits into
masterfrom
fix/findTriTriDistance
Jul 2, 2026
Merged

fix findTriTriDistance in case of finite upDistLimitSq#6357
Fedr merged 2 commits into
masterfrom
fix/findTriTriDistance

Conversation

@Fedr

@Fedr Fedr commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

findTriTriDistance could return a wrong (overestimated) distance when a non-default upDistLimitSq was given.

In the edge-pair loop, sd.dir is the vector between the closest points of the two edges and is not unit length. The projections p, s, t are dot products with sd.dir, so they carry an extra factor of |sd.dir|, and sqr( p - s + t ) overestimates the actual squared distance between the separating planes by |sd.dir|². This inflated value was compared against upDistLimitSq and, on early exit, stored in res.distSq — so with a finite limit the function could stop early and report a distance larger than the true one (and larger than what the same call without a limit returns).

The fix divides the squared plane distance by sqr( sd.dir ) (guarding against a zero vector), making the early-exit value a true squared distance.

Added a regression test with two triangles ~277.5 apart: the distance computed with upDistLimitSq slightly above the true squared distance must match the unlimited result; before the fix the early exit returned an inflated value.

@Fedr Fedr marked this pull request as ready for review July 2, 2026 11:36
@Fedr Fedr changed the title fix findTriTriDistance in case not default upDistLimitSq fix findTriTriDistance in case of finite upDistLimitSq Jul 2, 2026
@Fedr Fedr merged commit 3bbe1ed into master Jul 2, 2026
42 checks passed
@Fedr Fedr deleted the fix/findTriTriDistance branch July 2, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants