backup: support SHOW BACKUPS with backup ids #160137

kev-cao · 2025-12-24T21:58:02Z

This commit teaches SHOW BACKUPS to display the new UX with backup IDs, along with BEFORE/AFTER filtering.

Release note (sql change): Users can now set the use_backups_with_ids session setting to enable a new SHOW BACKUPS IN experience. When enabled, SHOW BACKUPS IN <collection> displays all backups in the collection. Results can be filtered by backup end time using OLDER THAN <timestamp> or NEWER THAN <timestamp> clauses.

Example usage:

SET use_backups_with_ids = true;
SHOW BACKUPS IN '<collection>' OLDER THAN '-1w' NEWER THAN '-2w';

blathers-crl · 2025-12-24T21:58:07Z

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

msbutler

Nearly there!

msbutler · 2026-01-08T19:47:31Z

pkg/sql/sessiondatapb/session_data.proto

  google.protobuf.Duration deadlock_timeout = 33 [(gogoproto.nullable) = false,
    (gogoproto.stdduration) = true];
+
+  // UseBackupsWithIDs, when true, enables the use of backup IDs in SHOW BACKUP


i think the commit could use a release note which explains how to use this new UX via the session var

msbutler · 2026-01-08T19:50:59Z

pkg/backup/show.go

+	} else {
+		p.BufferClientNotice(
+			ctx,
+			pgnotice.Newf("no specific time range provided, defaulting to 24 hours"),


to last 24 hours, yeah?

oh, i was under the impression that if the user specifies just BEFORE X (i.e. no AFTER), we return all backups taken before X; and if they just specify AFTER Y, we return all backups after Y.... no 24 hour time window.

Yeah the original impetus for having the default 24 hour window was in consideration of BEFORE. If the user wanted to see backups from before some disaster event at time t, I imagine t would be pretty recent. If they were to run SHOW BACKUPS IN <coll> BEFORE <t>, I didn't want them to get every backup in the collection before that time. So hence the 24 hour window.

And since I was already having a 24 hour window for BEFORE, it made sense to do the same with AFTER.

kev-cao · 2026-01-09T19:57:03Z

Screenshot:

msbutler

nearly there!

pkg/backup/show.go

msbutler · 2026-01-12T14:50:11Z

pkg/backup/BUILD.bazel

        "//pkg/util",
        "//pkg/util/admission",
        "//pkg/util/admission/admissionpb",
+        "//pkg/util/besteffort",


given how cursed negative intervals are, perhaps the example we give docs should be date times?

Hmm, yeah that's fair. I'm sad about it, it's so human-readable to just say "give me backups from 1 week ago".

pkg/backup/show.go

pkg/backup/show_test.go

msbutler · 2026-01-12T15:24:30Z

pkg/backup/show_test.go

+			)
+		})
+
+		t.Run("AFTER", func(t *testing.T) {


could you add a test that uses both BEFORE AND AFTER?

Hmm, so the reason I didn't write one for both is because of the small window of time we have for backups. These backups are all taken within a second of each other and since doing a BEFORE/AFTER rounds to second-precision, the query is unable to really query for a subset of the backups. You'd end up either getting all backups, or just doing a repeat of the tests above, which I didn't find to be a very interesting test. One solution would be to take the backups with sleeps in between, but that'd increase the length of the test.

So for the BEFORE and AFTER test coverage, I figured the unit tests on ListRestorableBackups would get us the coverage we need.

I see, we can leave it as is.

Added the test — after some more testing, you can sort of get some coverage if you just run the test enough times. So while most cases don't really give you interesting results, it still gives us coverage over time.

This commit updates the `SHOW BACKUP` time filtering syntax from `AFTER/BEFORE` to `NEWER/OLDER THAN`. Epic: CRDB-57536 Informs: cockroachdb#159647 Release note: None

kev-cao · 2026-01-12T16:14:41Z

New PGNotice:

msbutler

Nice job! i've come around to your approach. thanks for entertaining my push back.

msbutler · 2026-01-13T16:11:17Z

pkg/backup/backupinfo/backup_index.go

+				pendingEmit = nextEntry
+				return nil
+			}
+			if !nextEntry.end.After(pendingEmit.end) {


nit: I find "not after" a bit harder to reason about. To aid the reader, the comment could instead be written in a positive way:

"the pending emit has an end time less than or equal to the new entry. flush pending emit."

msbutler · 2026-01-13T16:22:52Z

pkg/backup/backupinfo/backup_index.go

+		ctx, store, newerThan, olderThan,
+		func(index parsedIndex) error {
+			if len(filteredIdxs) > 0 {
+				last := &filteredIdxs[len(filteredIdxs)-1]


nit: i think it is more idiomatic and readable to dereference the array element, instead of using pointer shenanigans. i.e. instead just do:

lastIdx := len(filteredIdx) if filteredIdx[lastIdx].end.Equal(index.end) .... filteredIdxs[lastIdx] = index

msbutler · 2026-01-13T16:24:02Z

pkg/backup/backupinfo/backup_index.go

+				// in descending order with ties broken by ascending start time, keeping
+				// the last one ensures that we keep the non-compacted backup.
+				if last.end.Equal(index.end) && last.fullEnd.Equal(index.fullEnd) {
+					*last = index


should we double check our invariant with a test only assertion? i.e. index.StartTime > last.StartTIme

Hmm, I think it might be better to just have a unit test on listIndexesWithinRange. Probably something that needs to be added anyway.

Actually, thinking about it some more, there's already test cases that cover this in ListRestorableBackups. I'll just add the test-only assertion.

kev-cao · 2026-01-13T21:36:36Z

TFTR!

bors r=msbutler

160137: backup: support SHOW BACKUPS with backup ids r=msbutler a=kev-cao This commit teaches SHOW BACKUPS to display the new UX with backup IDs, along with BEFORE/AFTER filtering. Epic: [CRDB-57536](https://cockroachlabs.atlassian.net/browse/CRDB-57536) Resolves: #159647 Release note (sql change): Users can now set the `use_backups_with_ids` session setting to enable a new `SHOW BACKUPS IN` experience. When enabled, `SHOW BACKUPS IN <collection>` displays all backups in the collection. Results can be filtered by backup end time using `OLDER THAN <timestamp>` or `NEWER THAN <timestamp>` clauses. Example usage: SET use_backups_with_ids = true; SHOW BACKUPS IN '<collection>' OLDER THAN '-1w' NEWER THAN '-2w'; 160922: jobs: skip stale pts check for cancel requested jobs r=dt a=msbutler The stale pts check will load in job progress/payload, even if the job is already in a cancel requested state. If the cancel requested queue is backed up, we have seen a single job processed over a hundred times, which simply leads to more contention. This patch reduces this contention by quickly checking if the job is in a cancel requested state before payload/progress reading. Informs: #158979 Release note: none Co-authored-by: Kevin Cao <39608887+kev-cao@users.noreply.github.com> Co-authored-by: Michael Butler <butler@cockroachlabs.com>

craig · 2026-01-13T22:07:49Z

Build failed (retrying...):

local_roachtest_fips

160137: backup: support SHOW BACKUPS with backup ids r=msbutler a=kev-cao This commit teaches SHOW BACKUPS to display the new UX with backup IDs, along with BEFORE/AFTER filtering. Epic: [CRDB-57536](https://cockroachlabs.atlassian.net/browse/CRDB-57536) Resolves: #159647 Release note (sql change): Users can now set the `use_backups_with_ids` session setting to enable a new `SHOW BACKUPS IN` experience. When enabled, `SHOW BACKUPS IN <collection>` displays all backups in the collection. Results can be filtered by backup end time using `OLDER THAN <timestamp>` or `NEWER THAN <timestamp>` clauses. Example usage: SET use_backups_with_ids = true; SHOW BACKUPS IN '<collection>' OLDER THAN '-1w' NEWER THAN '-2w'; Co-authored-by: Kevin Cao <39608887+kev-cao@users.noreply.github.com>

craig · 2026-01-13T23:27:49Z

Build failed:

unit_tests

This commit teaches SHOW BACKUPS to display the new UX with backup IDs, along with BEFORE/AFTER filtering. Epic: CRDB-57536 Resolves: cockroachdb#159647 Release note (sql change): Users can now set the `use_backups_with_ids` session setting to enable a new `SHOW BACKUPS IN` experience. When enabled, `SHOW BACKUPS IN <collection>` displays all backups in the collection. Results can be filtered by backup end time using `OLDER THAN <timestamp>` or `NEWER THAN <timestamp>` clauses. Example usage: SET use_backups_with_ids = true; SHOW BACKUPS IN '<collection>' OLDER THAN '2026-01-09 12:13:14' NEWER THAN '2026-01-04 15:16:17';

kev-cao · 2026-01-14T05:23:00Z

bors r=msbutler

craig · 2026-01-14T05:59:22Z

Build succeeded:

kev-cao requested review from a team as code owners December 24, 2025 21:58

kev-cao requested review from asg0451, msbutler, nameisbhaskar, shailendra-patel and xinhaoz and removed request for a team December 24, 2025 21:58

kev-cao requested review from a team, DrewKimball, andyyang890 and mw5h and removed request for a team, DrewKimball, andyyang890, asg0451, mw5h, nameisbhaskar, shailendra-patel and xinhaoz December 24, 2025 21:58

kev-cao force-pushed the backup/show-backups-with-index branch 3 times, most recently from d050901 to e7d8452 Compare January 8, 2026 18:03

msbutler reviewed Jan 8, 2026

View reviewed changes

kev-cao force-pushed the backup/show-backups-with-index branch 3 times, most recently from 3f30822 to f04686e Compare January 9, 2026 19:55

kev-cao requested a review from msbutler January 9, 2026 20:01

kev-cao force-pushed the backup/show-backups-with-index branch from f04686e to 24a7600 Compare January 9, 2026 21:43

msbutler reviewed Jan 12, 2026

View reviewed changes

backup: update SHOW BACKUP syntax to NEWER/OLDER than

45bb092

This commit updates the `SHOW BACKUP` time filtering syntax from `AFTER/BEFORE` to `NEWER/OLDER THAN`. Epic: CRDB-57536 Informs: cockroachdb#159647 Release note: None

kev-cao force-pushed the backup/show-backups-with-index branch from 24a7600 to e349e1d Compare January 12, 2026 16:13

kev-cao force-pushed the backup/show-backups-with-index branch from e349e1d to c8ed42e Compare January 12, 2026 16:34

msbutler approved these changes Jan 13, 2026

View reviewed changes

kev-cao force-pushed the backup/show-backups-with-index branch 3 times, most recently from f2fd283 to 214f7bd Compare January 13, 2026 18:58

kev-cao force-pushed the backup/show-backups-with-index branch from 214f7bd to b81ac03 Compare January 14, 2026 03:27

craig bot merged commit 2eeff2e into cockroachdb:master Jan 14, 2026
25 of 26 checks passed

celeste-cockroachdb bot added the target-release-26.2.0 label Jan 14, 2026

backup: support SHOW BACKUPS with backup ids #160137

backup: support SHOW BACKUPS with backup ids #160137

Uh oh!

Conversation

kev-cao commented Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blathers-crl bot commented Dec 24, 2025

Uh oh!

msbutler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kev-cao Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kev-cao commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

msbutler left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kev-cao commented Jan 12, 2026

Uh oh!

msbutler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kev-cao commented Jan 13, 2026

Uh oh!

craig bot commented Jan 13, 2026

Uh oh!

craig bot commented Jan 13, 2026

Uh oh!

kev-cao commented Jan 14, 2026

Uh oh!

craig bot commented Jan 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kev-cao commented Dec 24, 2025 •

edited

Loading

kev-cao Jan 8, 2026 •

edited

Loading

kev-cao commented Jan 9, 2026 •

edited

Loading