(WIP)TiDB x Architecture #22245
(WIP)TiDB x Architecture #22245
Conversation
ti-chi-bot
bot
added
the
do-not-merge/work-in-progress
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
bot
added
contribution
|
Hi @dbsid. Thanks for your PR. I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
ti-chi-bot
bot
added
the
missing-translation-status
Summary of ChangesHello @dbsid, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces detailed documentation for the new TiDB X architecture, outlining its fundamental shift towards a cloud-native, 'Share-Everything' design. It addresses limitations of the classic TiDB architecture by implementing innovations such as object storage as the single source of truth, a 'Compute and Compute' separation for workload isolation, and an LSM Forest storage engine, all aimed at enhancing scalability, stability, cost-effectiveness, and performance predictability. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
ti-chi-bot
bot
added
the
size/L
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new documentation page for the TiDB X architecture. The content provides a good high-level overview of the new architecture, its motivations, and key innovations. My review focuses on improving clarity, correcting some typos and grammatical errors, and ensuring consistency in terminology, as per the repository's style guide. I've provided several suggestions to enhance readability and technical accuracy.
tidb-x-architecture.md
Outdated
|
|
||
| The motivation of TiDB X is documented in the blog [The Making of TiDB X: Origins, Architecture, and What’s to Come](https://www.pingcap.com/blog/tidbx-origins-architecture/) | ||
|
|
||
| TiDB Classic has faced several challenges in large-scale production environments, primarily stemming from its "Share-nothing" architecture. |
There was a problem hiding this comment.
For consistency, technical terms like "Share-Nothing" should be capitalized consistently throughout the document.
| TiDB Classic has faced several challenges in large-scale production environments, primarily stemming from its "Share-nothing" architecture. | |
| TiDB Classic has faced several challenges in large-scale production environments, primarily stemming from its "Share-Nothing" architecture. |
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
|
||
| # TiDB X Introduction | ||
|
|
||
| TiDB X represents a fundamental architectural shift in the evolution of TiDB, transitioning from a classic "Share-Nothing" distributed database to a modern, "Share-Everything" Service-Oriented Architecture (SOA). Designed for the AI era and massive cloud scalability, TiDB X leverages Object Storage (e.g., Amazon S3) as the single source of truth. |
There was a problem hiding this comment.
single source of truth of what? We should elaborate it.
|
|
||
| # TiDB X Introduction | ||
|
|
||
| TiDB X represents a fundamental architectural shift in the evolution of TiDB, transitioning from a classic "Share-Nothing" distributed database to a modern, "Share-Everything" Service-Oriented Architecture (SOA). Designed for the AI era and massive cloud scalability, TiDB X leverages Object Storage (e.g., Amazon S3) as the single source of truth. |
There was a problem hiding this comment.
We called "share-nothing" a distributed database, but "share-everything" a SOA. It seems to me not apple to apple comparison.
|
|
||
| In the classic architecture, every TiKV node runs a single, massive RocksDB instance. This means all data from thousands of different "Regions" (logical data shards) is mixed together into one giant "single LSM-tree" structure. Because data is mixed, operations like moving a Region, scaling in/out, or importing data require rewriting massive amounts of existing data (compaction) to separate or merge it. This consumes huge CPU and I/O resources and impacts online traffic. The single LSM-tree is protected by a global mutex. As data size grows (3TB+) or file count increases (100k+ SST files), contention on this global lock will impact both the read and write operations. | ||
|
|
||
| While TiDB X retains the logical region concept from TiDB Classic, it fundamentally redesigns the storage engine by shifting from a single LSM tree to an LSM Forest. Instead of one giant tree for all data, TiDB X assigns each region its own separate, independent LSM Tree. The most critical benefit of this physical isolation is the elimination of compaction overhead during cluster operations (scale-in, scale-out, region movement, load data). Operations on one Region (like a heavy write or a split) are isolated to its specific tree. There is no global mutex lock contention. |
There was a problem hiding this comment.
TiDB Classic -> classic TiDB architecture? We better be consistent with how we call it.
2 participants
First-time contributors' checklist
What is changed, added or deleted? (Required)
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?