Key Function Process

In the parallel framework, there are the leader thread and worker threads.

The make_pq_leader_plan function of the leader thread determines whether statements can be executed in parallel, generates the leader's execution plan based on the original execution plan, and calls the ParallelScanIterator iterator to execute the statements.

• Init

  In the init process, the leader thread calls the add_scan table that can be executed in parallel to split the table into multiple data shards and put all shards in the same queue. Then, mysql_thread_create is called to create multiple worker threads. The worker threads cyclically obtain shards from the queue until all shards are executed.

• Read

  During the read process of the leader thread, the gather module is called to obtain data from the message queue. If necessary, extra aggregate operations are performed, such as COUNT and SUM. Finally, the data is transmitted to the upper layer and then to the client.

• End

  The end operations of an iterator include releasing the memory and returning the read data status.

The worker threads are created and started by the leader thread. The number of worker threads to be started depends on the parallelism degree. The worker thread calls make_pq_worker_plan to generate its own execution plan. In this process, a replaceable iterator in the original execution plan is replaced with the parallel iterator PQblockScanlterator. Then, the read function of PQblockScanIterator is called. This function calls the interaction interface with the InnoDB storage engine to obtain data from InnoDB. Then the send_data function is called to send the data to the message queue for the leader thread.