Scale the input once in OneD_SymbolicAggregateApproximation - #721
Open
VenishPaneliya wants to merge 3 commits into
Open
Scale the input once in OneD_SymbolicAggregateApproximation#721VenishPaneliya wants to merge 3 commits into
VenishPaneliya wants to merge 3 commits into
Conversation
`OneD_SymbolicAggregateApproximation._transform` scales `X` and then
hands it to `SymbolicAggregateApproximation._transform`, which scales it
again. `_scale` uses the fitted `mu_` and `std_`, so it is not
idempotent and the average symbols are computed from
`((X - mu) / std - mu) / std`.
With `scale=True` on data that is not already standardised the average
symbols collapse. For a series with mean 10 and standard deviation 5,
1d-SAX puts all 16 segments in a single symbol while SAX uses four:
SAX [3 3 2 3 2 3 0 2 2 3 1 1 3 0 2 1]
1d-SAX [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
Let the parent do the scaling for the average, and scale explicitly for
the slopes, which need the scaled series too.
`scale=False`, the default, is unchanged, and the slope symbols are
unchanged in both cases - only the average symbols under `scale=True`
move, and they now match SAX exactly.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #721 +/- ##
==========================================
+ Coverage 94.35% 95.31% +0.95%
==========================================
Files 83 83
Lines 7990 7996 +6
==========================================
+ Hits 7539 7621 +82
+ Misses 451 375 -76 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
Hi @VenishPaneliya , thanks for the contribution, that's a nice catch!!! Would you mind opening a related issue and reference it in a changelog entry? That would make both the issue and the fix more visible!!! Happy to help in anyway if needed |
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OneD_SymbolicAggregateApproximation._transformscalesXand then passes it toSymbolicAggregateApproximation._transform, which scales it again:_scaleuses the fittedmu_andstd_, so it is not idempotent — the average symbols end up computed from((X - mu) / std - mu) / std. The slopes are fine, since they are derived separately.With
scale=Trueon data that is not already standardised, the average symbols collapse. For a series with mean 10 and standard deviation 5:Every segment lands in the same symbol, so the average half of the representation carries no information.
inverse_transformalso unscales only once, so the round-trip does not return to the input scale.This lets the parent do the scaling for the average, and scales explicitly for the slopes, which need the scaled series as well.
Scope:
scale=False, the default, is unchanged — average and slope symbols are identical before and after.scale=Truethe slope symbols are also unchanged; only the average symbols move, and they now matchSymbolicAggregateApproximationexactly.scale=True. That output is degenerate today rather than merely different, which is why it seemed worth correcting rather than leaving.The existing
test_1dsaxexercisesscale=Truebut only asserts thatdistanceanddistance_1d_saxagree with each other, so it passes either way. The added test compares the average symbols againstSymbolicAggregateApproximation; it fails onmainand passes here.tests/test_piecewise.pygoes 4 -> 5 passing, andtests/test_estimators.pyfor the SAX/piecewise estimators is 127 passed, 3 skipped.