Skip to content

fix: preserve vector DDL errors after embedding retries - #1353

Open
willlling wants to merge 4 commits into
oceanbase:masterfrom
willlling:fix/seek-511-vector-aux-schema
Open

fix: preserve vector DDL errors after embedding retries#1353
willlling wants to merge 4 commits into
oceanbase:masterfrom
willlling:fix/seek-511-vector-aux-schema

Conversation

@willlling

@willlling willlling commented Sep 1, 2026

Copy link
Copy Markdown

Summary

  • retry transient CURLE_RECV_ERROR failures in embedding tasks with the existing bounded exponential backoff
  • refresh the parent vector-index DDL status before health decisions
  • while the parent waits for an auxiliary-table child DDL, let the child result be authoritative instead of replacing its original error with a secondary OB_TABLE_NOT_EXIST
  • retain the normal base-table health check and all auxiliary-schema checks outside child-wait states

Verification

  • release build: make -j80
  • normal hybrid vector-index create/rebuild path passed
  • added a no-mock mysqltest against a real HTTP 503 endpoint with model_max_retries=2
  • the test verifies at least six seconds of retry backoff, the post-retry product error, and complete auxiliary-table cleanup
  • old binary: test fails with MySQL 1146 (Table doesn't exist)
  • fixed binary: test passes with the original child failure preserved as MySQL 4016, and zero residual index tables

Issue: SEEK-511

@CLAassistant

CLAassistant commented Sep 1, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@willlling
willlling force-pushed the fix/seek-511-vector-aux-schema branch 3 times, most recently from 0ebc604 to 65f0dd3 Compare September 2, 2026 06:25
@hnwyllmm
hnwyllmm requested a balanced review from Copilot September 2, 2026 07:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Task matching remains overly broad, a completion race can misclassify valid schema, and cleanup coverage is nondeterministic.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Corrects vector refresh/rebuild error classification while HNSW auxiliary tables are being created.

Changes:

  • Checks active vector DDL tasks when the hidden index-id table is absent.
  • Returns transient or schema errors based on task state.
  • Adds race and cancellation mysqltests.
File summaries
File Description
ob_vector_refresh_index_executor.cpp Adds DDL-task-based error classification.
ob_vector_refresh_index_executor.h Declares the task-check helper.
vector_index_refresh_while_creating.test Tests refresh/rebuild during creation.
vector_index_refresh_while_creating.result Adds expected race-test output.
vector_index_create_failure_cleanup.test Tests cancellation cleanup and recreation.
vector_index_create_failure_cleanup.result Adds expected cleanup-test output.
Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 3
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +210 to +214
"SELECT COUNT(*) AS CNT FROM oceanbase.%s WHERE object_id = %lu "
"AND ddl_type IN (%d, %d)",
OB_ALL_DDL_TASK_STATUS_TNAME, data_table_id,
share::ObDDLType::DDL_CREATE_VEC_INDEX,
share::ObDDLType::DDL_REBUILD_INDEX))) {
false, /*with_hidden_flag*/
true /*is_built_in_index*/))) {
if (OB_TABLE_NOT_EXIST == ret) {
ret = check_vector_index_ddl_task_exist(base_table_id, has_ddl_task);

connection conn_create;
send create vector index idx_create_fail on t_vec_create_fail(c2) with (distance=l2, type=hnsw);
sleep 5;
@willlling
willlling force-pushed the fix/seek-511-vector-aux-schema branch 2 times, most recently from ab604ee to fd9efa3 Compare September 2, 2026 08:05
@willlling
willlling force-pushed the fix/seek-511-vector-aux-schema branch from fd9efa3 to 26b5c9d Compare September 2, 2026 08:14

@wangyunlai-seekdb wangyunlai-seekdb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed commit 26b5c9d.

I reviewed the force-pushed fixes for the existing Copilot threads. This review only reports an additional issue that is not covered there.

[Major] Register the new regressions in mysqltest CI

Location: tools/deploy/mysqltest_config.yaml:258-279

Neither vector_index.vector_index_create_failure_cleanup nor vector_index.vector_index_refresh_while_creating is listed in runtime_configs.psmall.test-set. discover_cases() executes only configured names and silently ignores unconfigured suite cases. The current successful run 33607668840 ran 68+68+68+67 = 271 configured cases, and neither new name appears in its logs. Add both cases to the vector-index list and rerun the slices so this fix is enforced by PR and future CI.

@willlling willlling changed the title Fix vector refresh during index creation fix: retry transient embedding receive errors Sep 3, 2026
@willlling willlling changed the title fix: retry transient embedding receive errors fix: preserve vector DDL errors after embedding retries Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants