Principles

In MySQL OLTP applications, a large number of DML statements (Insert, Update, and Delete) are concurrently executed on the key data structures in the lock_sys->mutex global lock, causing severe lock contention and performance deterioration. To solve this problem, Kunpeng BoostKit provides a fine-grained hash bucket lock to replace the global lock, which reduces lock conflicts and improves concurrency.

Each table or row in the MySQL database can be considered as a resource, and a transaction can request access to resources. However, concurrent transactions may cause a conflict of access to resources. The Lock-sys is designed to orchestrate access to tables and rows.

The Lock-sys maintains a separate queue for each resource. When a request for a resource is initiated, the Lock-sys queries whether the resource is occupied in the corresponding queue. No matter whether the requested resource is occupied, the Lock-sys inserts a lock request into the corresponding queue and marks the lock request as WAITING or GRANTED. To support concurrent operations, the queue needs to be locked during query and insertion.

In a database, the concepts of lock and latch are different.

A lock is used to lock database objects, such as tables and rows.
A latch is used to protect the memory data structure.

Access to all queues was managed by a latch. This means that even if only one queue is accessed, all other queues are locked, which is inefficient in high-concurrency scenarios. To solve this problem, a fine-grained latch is introduced.

The queues are divided into a fixed number of shards based on the original global latch. Each shard is protected by its own mutex. To efficiently latch all shards, the global latch in the new feature is designed as a read/write latch. Before a queue is accessed, the shared global latch and then the mutex of the corresponding shard must be obtained. This implementation is similar to the process for accessing a MySQL database record, where an intent lock is added to the table and then the record is locked. In certain special scenarios where all queues need to be latched, only the exclusive global latch needs to be obtained. The general idea is that one or two Lock-sys queues are involved in most operations and are independent of other queues. The following figure shows the relationship between the global latch and its managed objects.

The following figure shows how the new latching mode improves efficiency. The left part illustrates the access without lock optimization. If transaction A and transaction B need to access queue 1 and queue 2 respectively, transaction B will be blocked because both transactions need to latch the global latch. With the new feature (as indicated by the right part of the figure), transaction A and transaction B can request the shared global latch, and then request mutex 1 and mutex 2 respectively. That is, transaction A and transaction B can be performed at the same time, which improves the concurrency.

Accessing two queues to obtain two records involves the following steps:

S-latch the global latch.
Identify the two pages to which the records belong.
Identify the two hash buckets which contain the queues for the given pages.
Identify the IDs of the shards which contain these two buckets.
Latch mutexes for the two shards in the order of their addresses.

All of the preceding steps (except step 2, as we usually know the page already) are accomplished by using the following code:

     
          locksys::Shard_latches_guard guard{*block_a, *block_b};

For the "stop the world" operation, x-latch the global latch by using the following code:

     
          locksys::Exclusive_global_latch_guard guard{};

To use friend guard classes, like Shard_latches_guard, this class does not expose too many public functions.