Server prepare request hangs for 1000s due to OB_TX_NOLOGCB (-6268), leading to rollback communication failure

[cms-cms]

Environment

• Version: observer (OceanBase seekdb)
• Revision: 1-f84169fbd80d020887d038cdb0892703e28833f3
• Build Branch: HEAD
• Build Time: Aug 10 2026 10:45:34
• Build Flags: RelWithDebInfo
• Is this an RD-provided temporary version?: No
• Reproducibility: To be determined
• Test Changes: To be determined
• Other environmental anomalies/changes: To be determined

Description

During a PayTrade server prepare operation, the request hung for approximately 1000 seconds. The client subsequently attempted a rollback, which failed due to a communication link failure (SocketTimeoutException). The root cause on the observer side is identified as error code -6268 (OB_TX_NOLOGCB), indicating the transaction context could not obtain an idle transaction log callback.

Steps to Reproduce / Scenario

1. A PayTrade workload was running.
2. A serverPrepare request was sent.
3. The request hung for ~1000 seconds without a response from the server.
4. The client's socketTimeout (configured to 1,000,000 ms) was triggered.
5. The client attempted a rollback, which failed with a Communications link failure during rollback error.

Client Error (JDBC):

MySQLNonTransientConnectionException:
Communications link failure during rollback.
Transaction resolution unknown.
CommunicationsException
The last packet successfully received from the server was
1,000,110 milliseconds ago.
The last packet sent successfully to the server was
1,000,110 milliseconds ago.

Caused by: java.net.SocketTimeoutException: Read timed out
SQLState: 08S01

Call Stack:

ServerPreparedStatement.serverPrepare
PayTrade.prepare(PayTrade.java:118)
Transaction.doTransaction(Transaction.java:190)
TransactionTask.runMixTransations(TransactionTask.java:303)

Relevant Business Info:

trade_no=1000000000000000009
user_id=3
session id=9

The rollback error is a symptom, not the root cause. The primary issue is the server-side hang.

Observer-Side Core Evidence

Database logs clearly show the OB_TX_NOLOGCB error.

Key Log Snippets:

iter_idle_pool_ (ob_tx_log_cb_mgr.cpp:429)
errcode=-6268
no idle log cb group

get_log_cb_ (ob_tx_ctx_log_cb_helper.cpp:153)
ret="OB_TX_NOLOGCB"
free_cbs_.get_size=0

prepare_log_cb_
failed to get log_cb

~ObMvccWriteGuard (ob_mvcc_ctx.cpp:377)
errcode=-6268
failed to submit log if necessary

Another transaction freeze path:

submit_redo_log_for_freeze_ (ob_tx_ctx.cpp:1347)
fail to submit redo log for freeze(ret=-6268)

traverse_tx_to_submit_redo_log (ob_trans_ctx_mgr_v4.cpp:835)
failed to submit log(ret=-6268)

Error code -6268 (OB_TX_NOLOGCB) appeared approximately 244 times in the relevant rotated logs. The timing aligns: the client began waiting around 18:10:16, and observer logs show OB_TX_NOLOGCB and freeze redo submission failures at 18:08:35 and 18:11:32 respectively.

Environment & Connection Details

Database Connection:

obclient -h% -P 2831 -uroot@sys -A -Dtest

JDBC Core Configuration:

jdbc:oceanbase://:2831/test
user=root@sys
socketTimeout=1000000
useServerPrepStmts=true

Current Preserved State:

host:%
port: 2831
observer pid: 1423622

Log Directory

• Client Error Log: /data/2/mk_resource/seekdb-gitlab/obtrade_lite/runs/pipeline-189961-job-919130/scenario/singlejob/obtrade_lite/log/client.error.log
• Full Client Log: /data/2/mk_resource/seekdb-gitlab/obtrade_lite/runs/pipeline-189961-job-919130/scenario/singlejob/obtrade_lite/log/client.log
• Application Server Log: /data/2/mk_resource/seekdb-gitlab/obtrade_lite/runs/pipeline-189961-job-919130/scenario/singlejob/obtrade_lite/log/appserver.console.log
• Configuration: /data/2/mk_resource/seekdb-gitlab/obtrade_lite/runs/pipeline-189961-job-919130/scenario/singlejob/obtrade_lite/conf/conf.xml
• Database Logs: /data/1/seekdb-gitlab/obtrade_lite-log/pipeline-189961-job-919130/
• Key File: seekdb.log.20260810181037576

Key Traces/Transactions:

trace: YB427F000001-000658AE538C8991-0-0
txid: 451310

trace: YB427F000001-000658AE543D4902-0-0

Suggested Focus for Investigation

1. Why did the transaction log callback pool reach free_cbs_.get_size=0?
2. Is there a callback leak, prolonged occupation, or failure to recycle promptly?
3. Does OB_TX_NOLOGCB cause a server prepare request to hang indefinitely without returning an error?
4. Investigate the redo submission and callback lifecycle for transaction ID 451310 and the related traces.
5. Does batch freeze exacerbate the callback exhaustion problem?
6. Why did the request hang for the full 1000 seconds instead of returning a diagnosable error promptly?

This case presents a more complete causal chain than previous bigtrans issues and is recommended for priority review by transaction/PALF/log callback related developers.