-
Notifications
You must be signed in to change notification settings - Fork 709
*: add detailed intro to the TiDB X architecture #22245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
Hi @dbsid. Thanks for your PR. I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Summary of ChangesHello @dbsid, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces detailed documentation for the new TiDB X architecture, outlining its fundamental shift towards a cloud-native, 'Share-Everything' design. It addresses limitations of the classic TiDB architecture by implementing innovations such as object storage as the single source of truth, a 'Compute and Compute' separation for workload isolation, and an LSM Forest storage engine, all aimed at enhancing scalability, stability, cost-effectiveness, and performance predictability. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces a new documentation page for the TiDB X architecture. The content provides a good high-level overview of the new architecture, its motivations, and key innovations. My review focuses on improving clarity, correcting some typos and grammatical errors, and ensuring consistency in terminology, as per the repository's style guide. I've provided several suggestions to enhance readability and technical accuracy.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Review CommentsTerminology Suggestion: Share-Everything → Shared-StorageThe term Share-Everything is imprecise and creates an awkward comparison (as @likidu noted - comparing a distributed database to an SOA is not apples-to-apples). Recommendation: Consider using Shared-Storage instead:
Alternative framing for the intro:
Structural Issues
Content Issues
PR DescriptionPlease fill in the required What is changed, added or deleted? section before moving out of draft. Overall the content is well-structured and provides good technical depth. The diagrams help illustrate the architectural differences clearly. |
|
/ok-to-test |
|
|
||
| The architecture diagram highlights a sophisticated separation of duties, ensuring that different types of work do not interfere with each other. The top "Isolated SQL Layer" consists of separate groups of compute nodes, which allows for multi-tenancy or workload isolation where different applications can have dedicated compute resources while sharing the same underlying data. Beneath this, the "Shared Services" layer breaks down heavy database tasks into independent microservices for operations like compaction, analyze, and DDL. By offloading expensive background operations—such as adding an index, Online DDL, or massive data imports—to the Shared Services layer, the system ensures these heavy jobs never compete for CPU or memory with the "Compute" nodes serving online user traffic. This guarantees predictable performance for critical applications and allows each component—Gateway, SQL Compute, Cache, and Background Services—to scale independently based on specific bottlenecks. | ||
|
|
||
| # Key innovations of TiDB X |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are some comparisons with classic TiDB in this section, such as “6TB+ data per TiKV node or 200k+ SST files,” which are duplicated in the section “The Architectural Ceiling: Challenges Hard to Overcome.” It would be better to remove the redundant descriptions.
|
|
||
| ## Rapid Elastic Scalability (5x-10x Faster) | ||
|
|
||
| In TiDB X, data resides in shared object storage with fully isolated LSM-trees for each Region. The system eliminates the need for physical data migration or compaction when adding or removing TiKV nodes. The result is a 5x–10x improvement in scaling speed compared to classic TiDB, maintaining stable latency for online traffic. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are many key advantages. It is recommended to end with a summary section that includes a comparison between TiDB Classic and TiDB X in key areas, making it easier for users to remember.
Co-authored-by: Frank945946 <[email protected]>
|
@dbsid: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
lilin90
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll continue my review soon.
| @@ -0,0 +1,111 @@ | |||
| --- | |||
| title: TiDB X Architecture | |||
| summary: Learn how TiDB X's shared-storage architecture delivers cloud-native scalability and cost optimization | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| summary: Learn how TiDB X's shared-storage architecture delivers cloud-native scalability and cost optimization | |
| summary: Learn how the shared-storage, cloud-native TiDB X architecture delivers elastic scalability, predictable performance, and optimized total cost of ownership. |
|
|
||
| # TiDB X Architecture | ||
|
|
||
| TiDB X represents a fundamental architectural evolution from classic TiDB's Shared-Nothing design to a cloud-native Shared-Storage architecture. By leveraging object storage as the single source of truth, TiDB X introduces "Separation of Compute and Compute" design that isolates online transactional workloads from heavy background tasks. This architecture enables elastic scalability, predictable performance, and optimized Total Cost of Ownership (TCO) for AI-era workloads. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reason:
- Since this doc is about intro to TiDB X, the first intro should tell what it is clearly to help users quickly get it if they don't want to read details.
- The phrase "single source of truth" is marketing-oriented, not recommended in official docs.
- Clarify what is being separated (compute workloads), not just repeating the slogan.
- Use lowercase “total cost of ownership (TCO)” on first occurrence (common style-guide practice).
Definition reference:
- https://www.pingcap.com/press-release/pingcap-launches-tidb-x-new-ai-capabilities/
- https://www.pingcap.com/blog/introducing-tidb-x-a-new-foundation-distributed-sql-ai-era/
| TiDB X represents a fundamental architectural evolution from classic TiDB's Shared-Nothing design to a cloud-native Shared-Storage architecture. By leveraging object storage as the single source of truth, TiDB X introduces "Separation of Compute and Compute" design that isolates online transactional workloads from heavy background tasks. This architecture enables elastic scalability, predictable performance, and optimized Total Cost of Ownership (TCO) for AI-era workloads. | |
| TiDB X is a new distributed SQL architecture that makes cloud-native object storage the backbone of TiDB. This architecture enables elastic scalability, predictable performance, and optimized total cost of ownership (TCO) for AI-era workloads. | |
| TiDB X represents a fundamental evolution from classic TiDB's shared-nothing architecture to a cloud-native shared-storage architecture. By leveraging object storage as the shared persistent storage layer, TiDB X introduces a separation of compute workloads that isolates online transactional processing from resource-intensive background tasks. |
First-time contributors' checklist
What is changed, added or deleted? (Required)
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?