Open-Source OceanBase Reference Architectures
Architecture
This document provides only typical architectures for reference. For open-source databases, customers can select the required solution architecture based on their specific services. For commercial databases, technical solutions and support are provided by the database vendors.
The OceanBase database uses a shared-nothing distributed cluster architecture. All nodes are completely equal, and each node has its own SQL engine, storage engine, and transaction engine. The database runs on clusters consisting of common PC servers and offers core features such as high scalability, high availability, high performance, low cost, and high compatibility with mainstream databases. A cluster consists of multiple nodes, which are distributed across multiple availability zones, with each node belonging to exactly one zone. Each node in the cluster runs a service process called observer. All nodes are functionally equivalent, and these processes communicate with one another over TCP/IP. Applications do not directly connect to database nodes. Instead, they connect to the OceanBase Database Proxy (ODP), also known as OBProxy, which forwards SQL requests to the appropriate nodes.
Scenarios
Open-source OceanBase is a typical example of a natively distributed HTAP database. Based on the principle of "one copy of data, one engine," it supports both online transaction processing (OLTP) and online analytical processing (OLAP) workloads. The database uses a multi-replica strong consistency mechanism based on the Multi-Paxos protocol. Using common servers, it can achieve equipment room-level disaster recovery (DR) with zero data loss. That is, the Recovery Point Objective (RPO) is 0, and the Recovery Time Objective (RTO) is less than 8 seconds, meeting the level-6 DR standards of the financial industry. Through transparent horizontal scaling, the database supports clusters with thousands of nodes, more than 3 PB of data in a single cluster, and trillions of rows in a single table. The LSM-Tree-based high-compression engine reduces storage costs by 70% to 90%. The database natively supports a multi-tenant architecture, allowing a single cluster to serve multiple independent service workloads, with data and resource isolation between tenants. OceanBase is highly compatible with MySQL (the community edition supports only MySQL mode). It has set world records in both TPC-C and TPC-H, making it a preferred solution for replacing commercial distributed databases and powering financial-grade core systems.
Working Mechanism
Figure 1 shows the OceanBase principle. Table 1 describes the main threads of the observer process.
In terms of data organization, the data in a table can be horizontally split into multiple partitions based on the partitioning rules. Hash, range, and list partitioning, as well as subpartitioning, are supported. Each physical partition corresponds to a storage-layer object called a tablet, which is used to store ordered data records. Modifications to records in a tablet are written as redo logs to the corresponding log stream (LS). Each LS and its associated tablets have multiple replicas, which are typically distributed across different availability zones. Among multiple replicas, only one replica accepts the modification operation, which is called the leader. The other replicas are called followers. The leader and followers maintain data consistency through a distributed consensus protocol based on Multi-Paxos. If the node hosting the leader fails, a follower is elected as the new leader and continues to provide services.
The observer process adopts a single-process, multi-threaded architecture, with its threads categorized into tenant worker threads and background threads. Table 1 describes the main threads.
|
Thread |
Description |
|---|---|
|
SQL/transaction worker (tenant worker thread) |
Thread for processing SQL and transaction requests. It is deployed by tenant. Each tenant has its own set of worker threads to implement tenant-level resource isolation. |
|
Network I/O |
Thread for processing network I/O. It is not tenant-specific and is responsible for RPC communication between nodes and data transmission and reception for external connections. |
|
Disk I/O |
Thread for processing disk I/O. It is tenant-specific and responsible for reading and writing data files. |
|
Clog writer |
Thread for writing redo logs (clogs). It is tenant-specific and ensures the persistence of transaction modifications. |
|
Election worker |
Election thread, which is tenant-specific and is responsible for electing the leader among Multi-Paxos replicas. |
|
DAG thread |
Tenant-specific thread used to execute background tasks such as partition dump, merge, and migration. |
|
Misc timer |
Includes multiple background timer threads, which are tenant-specific and responsible for clearing resources. |
|
RootService dedicated thread |
Handles cluster-level management (such as load balancing and replica balancing performed by the RootService module), which is not tenant-specific. |
To simplify the management of multiple service databases deployed on a large scale and reduce resource costs, OceanBase provides the multi-tenant feature. Each observer process can have multiple virtual containers that belong to different tenants, which are called resource units and include CPU and memory resources. The resource units of each tenant on multiple nodes form a resource pool. After a cluster is initialized, a system tenant named sys is automatically created to store cluster metadata.
Typical Component Architectures
OceanBase's mainstream reference architecture is a three-replica deployment across three data centers (DCs) in the same city, as shown in Figure 2.
Three DCs in the same city form a cluster, with each DC being a zone. The network latency between DCs typically ranges from 0.5 ms to 2 ms. In the event of a DC disaster, the remaining two replicas still form a majority, and redo logs can be synchronized to ensure an RPO of 0. However, this deployment cannot handle city-level disasters.
Common reference architectures include:
- Intra-city, three-DC, three-replica deployment: Provides DC-level DR with an RPO of 0, which is a mainstream production deployment solution.
- Three-city, five-DC, five-replica deployment: Spans three cities with a five-replica cluster. If any IDC or city is unavailable, the remaining replicas still form a majority, enabling city-level DR with an RPO of 0. This is a financial-grade remote multi-active solution.
- Intra-city, two-DC primary-standby deployment: Forms an independent cluster in each DC, with one primary and one standby; the clusters are asynchronously synchronized through redo logs, similar to traditional primary-secondary database replication.
- Two-city, three-DC primary-standby deployment: Forms a five-replica cluster with two DCs in the primary city and builds an independent three-replica cluster in the standby city as a standby database with asynchronous synchronization; if the primary city suffers from a disaster, the standby city takes over services.
- Intra-city, three-DC arbitration service deployment: Places full-featured replicas in two DCs and deploys the arbitration service (no log synchronization required) in the third DC to reduce costs, enabling DC-level DR with an RPO of 0.
- Three-city, five-DC arbitration service deployment: Places full-featured replicas in two nearby cities and deploys the arbitration service in the third city, enabling city-level DR with an RPO of 0 while reducing cross-city bandwidth overhead and deployment costs.
- Two-city, three-DC arbitration service deployment: Places full-featured replicas in two DCs of the primary city and deploys the arbitration service in the standby city, reducing deployment costs and cross-city bandwidth overhead.
In multi-DC and multi-city deployment scenarios, the IDC and region attributes of a zone must reflect the actual deployment to ensure that the automatic DR and optimization policies in the cluster work better.