Testing the vKAE Performance Using httpress in the Nginx Application Scenario

Test Commands

Two groups of performance tests using httpress on the VM are performed, covering the synchronous/asynchronous modes of the open source Nginx and the synchronous/asynchronous modes of Nginx with vKAE enabled.

On the client, run the following commands to use httpress to perform pressure tests on HTTPS persistent and short connections:

• HTTPS persistent connection:

  httpress -n 2000000 -c 200 -t 10 -k https://Server_IP_address:Server_port_number/index${nodeIndexNum}.html

• HTTPS short connection:

  httpress -n 20000 -c 200 -t 10 https://Server_IP_address:Server_port_number/index${nodeIndexNum}.html

• -n specifies the number of requests, -t specifies the number of threads, -c specifies the number of connections, and -k enables (or disables) the use of persistent connections. The -n, -t, and -c parameters can be adjusted to optimal values based on actual situations, achieving the largest RPS value.
• ${nodeIndexNum}, a variable, specifies the index of the page to be tested as required.
• On the client, HTTP does not use encryption and decryption algorithms, while HTTPS uses the SSL/TLS algorithm based on HTTP, and the SSL/TLS handshake process involves encryption and decryption calculation. Because asymmetric encryption and decryption calculation is complex and time-consuming, the HPRE device only accelerates the RSA asymmetric encryption and decryption algorithm, that is, vKAE hardware acceleration only works for HTTPS persistent and short connections. Therefore, in the vKAE acceleration scenario, only HTTPS test results are analyzed.

On the server, perform the following operations.

1. One thread is used as an example. Create an Nginx configuration file named sync_nginx_1_worker.conf in the /root/test/ directory.

   vim sync_nginx_1_worker.conf

2. Press i to enter the insert mode and add the following content to the file.

   The following is the content of the configuration file for enabling KAE + Nginx synchronous mode without parameter tuning.

   # For more information on configuration, see:
   #   * Official English Documentation: http://nginx.org/en/docs/
   #   * Official Russian Documentation: http://nginx.org/ru/docs/
   
   user root;
   
   worker_processes 1;
   worker_cpu_affinity
   1
   ;
   
   # error_log /var/log/nginx/error.log debug;
   error_log /var/log/nginx/error.log;
   pid /run/nginx.pid;
   
   # Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
   include /usr/share/nginx/modules/*.conf;
   
   events {
       worker_connections 1024;
   }
   
   http {
       log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent "$http_referer" '
                         '"$http_user_agent" "$http_x_forwarded_for"';
       # access_log off;
       # access_log  /var/log/nginx/access.log  main;
   
       sendfile            on;
       tcp_nopush          on;
       tcp_nodelay         on;
       keepalive_timeout   65;
       types_hash_max_size 4096;
   
       include             /usr/local/nginx/conf/mime.types;
       default_type        application/octet-stream;
   
       # Load modular configuration files from the /etc/nginx/conf.d directory.
       # See http://nginx.org/en/docs/ngx_core_module.html#include
       # for more information.
       include /etc/nginx/conf.d/*.conf;
   
       server {
           listen       8080;
           listen       [::]:8080;
           server_name  _;
           root         /usr/share/nginx/html;
   
           # Load configuration files for the default server block.
           include /etc/nginx/default.d/*.conf;
   
           error_page 404 /404.html;
               location = /40x.html {
           }
   
           error_page 500 502 503 504 /50x.html;
               location = /50x.html {
           }
       }
   
   # Settings for a TLS enabled server.
   #
      server {
          listen       8090 ssl http2 so_keepalive=off;
          listen       [::]:8090 ssl http2 so_keepalive=off;
          #listen       8090 ssl http2 so_keepalive=off asynch;
          #listen       [::]:8090 ssl http2 so_keepalive=off asynch;
          server_name  _;
   
          # ssl_asynch on;
   
          ssl_certificate      /usr/local/nginx/server_2048.crt;
         ssl_certificate_key  /usr/local/nginx/server_2048.key;
   
          root         /usr/share/nginx/html;
   
          ssl_session_cache    shared:SSL:1m;
          ssl_session_timeout  5m;
          ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
   
          ssl_ciphers  "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256    EECDH+aRSA+RC4 EECDH EDH+aRSA !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS !RC4";
          #ssl_ciphers  "RSA-PSK-AES128-CBC-SHA256 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS !RC4";
          # ssl_ciphers  AES256-GCM-SHA384;
          ssl_prefer_server_ciphers  on;
          # ssl_session_tickets  off;
   
          error_page 404 /404.html;
              location = /40x.html {
          }
   
          error_page 500 502 503 504 /50x.html;
              location = /50x.html {
          }
      }
   
   }

   

   • In the configuration file, the HTTP port number is 8080 and the HTTPS port number is 8090.
   • worker_processes specifies the number of threads on the server, and worker_cpu_affinity specifies NUMA affinity. You can change the number of threads on the server as required and change the NUMA affinity accordingly.
   • To use four server threads, modify the worker_processes and worker_cpu_affinity parameters as follows:

     worker_processes 4;
     worker_cpu_affinity
     1
     10
     100
     1000;

   • To configure a configuration file for enabling KAE + Nginx asynchronous mode without parameter tuning, create a configuration file named async_nginx_x_worker.conf (x is the number of threads to be used). Copy the preceding content to async_nginx_x_worker.conf, and delete the comment signs (#) at the start of the following three lines:

     #listen       8090 ssl http2 so_keepalive=off asynch;
     #listen       [::]:8090 ssl http2 so_keepalive=off asynch;
     
     # ssl_asynch on;

     Add comment signs (#) to the start of the following two lines:

     listen       8090 ssl http2 so_keepalive=off;
     listen       [::]:8090 ssl http2 so_keepalive=off;

   • If you want to use the Nginx configuration file with tuned parameters, you can use the configuration file in 9.
3. Press Esc, type :wq!, and press Enter to save the file and exit.
4. On the client, use httpress to perform a pressure test on HTTPS short connections.
   In this example, the server uses one thread and four threads, and the number of bound cores on the client is limited as required. Use httpress to perform a pressure test on HTTPS short connections on the client. The test commands are as follows:

   • KAE enabled + Nginx synchronous mode + One thread used by the server
     • On the server:

       nginx -s stop || true; sleep 1; OPENSSL_CONF=/home/openssl.cnf /usr/share/nginx/sbin/nginx -c /root/test/sync_nginx_1_worker.conf; sleep 1

     • On the client:

       taskset -c 64-254 httpress -c 64 -t 32 -n 64000 https://Server_IP_address:8090/index.html

   • KAE enabled + Nginx asynchronous mode + One thread used by the server
     • On the server:

       nginx -s stop || true; sleep 1; OPENSSL_CONF=/home/openssl.cnf /usr/share/nginx/sbin/nginx -c /root/test/async_nginx_1_worker.conf; sleep 1

     • On the client:

       taskset -c 64-254 httpress -c 64 -t 32 -n 64000 https://Server_IP_address:8090/index.html

   • KAE enabled + Nginx synchronous mode + Four threads used by the server
     • On the server:

       nginx -s stop || true; sleep 1; OPENSSL_CONF=/home/openssl.cnf /usr/share/nginx/sbin/nginx -c /root/test/sync_nginx_4_worker.conf; sleep 1

     • On the client:

       taskset -c 64-254 httpress -c 64 -t 32 -n 128000 https://Server_IP_address:8090/index.html

   • KAE enabled + Nginx asynchronous mode + Four threads used by the server
     • On the server:

       nginx -s stop || true; sleep 1; OPENSSL_CONF=/home/openssl.cnf /usr/share/nginx/sbin/nginx -c /root/test/async_nginx_4_worker.conf; sleep 1

     • On the client:

       taskset -c 64-254 httpress -c 64 -t 32 -n 128000 https://Server_IP_address:8090/index.html

   • KAE disabled + Nginx synchronous mode + One thread used by the server
     • On the server:

       nginx -s stop || true; sleep 1; /usr/share/nginx/sbin/nginx -c /root/test/sync_nginx_1_worker.conf; sleep 1

     • On the client:

       taskset -c 64-254 httpress -c 64 -t 32 -n 64000 https://Server_IP_address:8090/index.html

   • KAE disabled + Nginx asynchronous mode + One thread used by the server
     • On the server:

       nginx -s stop || true; sleep 1; /usr/share/nginx/sbin/nginx -c /root/test/async_nginx_1_worker.conf; sleep 1

     • On the client:

       taskset -c 64-254 httpress -c 64 -t 32 -n 64000 https://Server_IP_address:8090/index.html

   • KAE disabled + Nginx synchronous mode + Four threads used by the server
     • On the server:

       nginx -s stop || true; sleep 1; /usr/share/nginx/sbin/nginx -c /root/test/sync_nginx_4_worker.conf; sleep 1

     • On the client:

       taskset -c 64-254 httpress -c 64 -t 32 -n 128000 https://Server_IP_address:8090/index.html

   • KAE disabled + Nginx asynchronous mode + Four threads used by the server
     • On the server:

       nginx -s stop || true; sleep 1; /usr/share/nginx/sbin/nginx -c /root/test/async_nginx_4_worker.conf; sleep 1

     • On the client:

       taskset -c 64-254 httpress -c 64 -t 32 -n 128000 https://Server_IP_address:8090/index.html

Test Results and Analysis

Table 1 shows the RPS results of the httpress pressure tests on HTTPS short connections on an 8C16G VM.

Conclusions are drawn under the circumstance that the CPU usage reaches 100%:

• For 8C16G VMs, the performance in the synchronous and asynchronous modes is almost the same when vKAE is disabled. After vKAE is enabled and before the CPU reaches the performance bottleneck, vKAE gains better performance in the asynchronous mode than in the synchronous mode, and the RPS in the asynchronous mode is 40% higher than that in the synchronous mode.
• For 8C16G VMs, before the CPU reaches the performance bottleneck, the computing power ratio in the synchronous mode is 226%, and that in the asynchronous mode is 329%, which proves that the performance improvement of the asynchronous mode is higher than that of the synchronous mode.

Conclusion and Application Suggestions

For VMs of the 8C16G specification:

• When vKAE is disabled, the performance of the Nginx synchronous and asynchronous modes are almost the same. After vKAE is enabled, the Nginx asynchronous mode demonstrates better performance especially when processing HTTPS requests. Therefore, in scenarios where high-performance HTTPS services are required, you are advised to use the combined configuration of vKAE and the Nginx asynchronous mode.
• The parameter tuning of the Nginx configuration file is also key to performance improvement. You can properly adjust Nginx parameters such as worker_processes and worker_cpu_affinity to further explore hardware capability potential and improve the overall performance of the system.