libaio Implementation Reference

For drive files, file reading is synchronous. As a result, when a thread reads files, the thread is blocked. To improve performance and drive throughput, the program creates several independent drive read/write threads and uses mechanisms such as semaphore to implement inter-thread communication (with locks). Obviously, if there are many threads and locks, more resources will be preempted, which deteriorates the overall system performance.

The libaio provides an asynchronous mechanism for reading and writing drive files, preventing file read and write from being blocked. With the epoll mechanism, a thread can run without block and process multiple file read and write requests at the same time, improving the overall server performance. The following is an implementation example of libaio+epoll:

Branches such as variable initialization and exception determination are deleted from the code.

/* Callback function of AIO*/
void aio_callback(io_context_t ctx, struct iocb *iocb, long res, long res2)
{
    /*If the operation is failure, -res is errorno. If the operation is successful, res indicates the number of bytes that are actually read.
       iocb->u.c.buf is the read content.*/
    ···
}

/*
 *Use libaio and epoll to initiate asynchronous read requests to different offset positions of the same file.
 *For example, initiate 128 asynchronous read requests (reading 1024 bytes) to the offset positions 0, 1024 x 1, 1024 x 2, and 1024 x 127 of the file.
 */
int libaio_epoll_read_file()
{
    efd = eventfd(0, EFD_NONBLOCK);
    fd = open(TEST_FILE, O_RDWR | O_CREAT | O_DIRECT, 0644);
    io_setup(8192, &ctx);
    for (int i = 0; i < NUM_EVENTS; i++) {
        posix_memalign(&buf, ALIGN_SIZE, RD_WR_SIZE);
        io_prep_pread(iocbs[i], fd, buf, RD_WR_SIZE, i * RD_WR_SIZE);
        io_set_eventfd(iocbs[i], efd);
        io_set_callback(iocbs[i], aio_callback);
    }
    io_submit(ctx, NUM_EVENTS, iocbs);
    epfd = epoll_create(1);
    epevent.events = EPOLLIN | EPOLLET;
    epevent.data.ptr = NULL;
    epoll_ctl(epfd, EPOLL_CTL_ADD, efd, &epevent);
    while (i < NUM_EVENTS) {
        epoll_wait(epfd, &epevent, 1, -1);
        read(efd, &finished_aio, sizeof(finished_aio));
        while (finished_aio > 0) {
            r = io_getevents(ctx, 1, NUM_EVENTS, events, &tms);
            for (j = 0; j < r; ++j) {
                //Call the callback function of the corresponding event to process data.
                ((io_callback_t)(events[j].data))(
                    ctx, events[j].obj, events[j].res, events[j].res2);
            }
        }
    }
}