Skip to content

[Enhancement] Skip predicate column vacuuming when TTL is negative or usage is empty#71290

Open
seokyun-ha-toss wants to merge 2 commits intoStarRocks:mainfrom
seokyun-ha-toss:disable-predicate-column-vacuum
Open

[Enhancement] Skip predicate column vacuuming when TTL is negative or usage is empty#71290
seokyun-ha-toss wants to merge 2 commits intoStarRocks:mainfrom
seokyun-ha-toss:disable-predicate-column-vacuum

Conversation

@seokyun-ha-toss
Copy link
Copy Markdown

@seokyun-ha-toss seokyun-ha-toss commented Apr 4, 2026

Why I'm doing:

I'm using StarRocks with enable_predicate_columns_collection=false and have truncated the _statistics_.predicate_columns table. However, the vacuum process with TTL is still running and generating useless audit logs. I want to suppress predicate column vacuuming by disabling predicate column TTL.

What I'm doing:

When statistic_predicate_columns_ttl_hours is set to a negative value (e.g. -1), skip vacuuming entirely.

Additionally, skip vacuuming id2columnUsage when it is already empty before removal:

    public void vacuum() {
        long ttlHour = Config.statistic_predicate_columns_ttl_hours;
        if (ttlHour < 0) { // HERE
            return;
        }
        LocalDateTime ttlTime = TimeUtils.getSystemNow().minusHours(ttlHour);
        Predicate<ColumnUsage> outdated = x -> x.getLastUsed().isBefore(ttlTime);

        long before = id2columnUsage.size();
        if (before > 0 && id2columnUsage.values().removeIf(outdated)) { // HERE
            long after = id2columnUsage.size();
            LOG.info("removed {} objects from predicate columns because of ttl {}", before - after,
                    Config.statistic_predicate_columns_ttl_hours);
        }

        // If the process crashed before vacuum the storage, the storage may be different from in-memory state,
        // but it doesn't matter. Because we will remove them finally.
        getStorage().vacuum(ttlTime);
    }

Already, reflect the changes on documents also.

Fixes #71291

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5
    • 3.4

@seokyun-ha-toss seokyun-ha-toss requested a review from a team as a code owner April 4, 2026 08:02
@github-actions github-actions bot added behavior_changed title needs [type] documentation Improvements or additions to documentation labels Apr 4, 2026
@seokyun-ha-toss seokyun-ha-toss changed the title Disable predicate column vacuum [improve] Skip predicate column vacuuming when TTL is negative or usage is empty Apr 4, 2026
@seokyun-ha-toss seokyun-ha-toss changed the title [improve] Skip predicate column vacuuming when TTL is negative or usage is empty [Enhancement] Skip predicate column vacuuming when TTL is negative or usage is empty Apr 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip predicate column vacuuming when TTL is negative or usage is empty

1 participant