Documentation
我要评分
获取效率
正确性
完整性
易理解
提交关闭

Deploying Nginx on VMs

Deploy Nginx for KAE to enable the Nginx synchronous or asynchronous mode.

  1. Install the dependencies on the VM.
    yum install -y openssl openssl-devel pcre pcre-devel zlib zlib-devel gcc make
  2. If the server is connected to the Internet, run the wget command to download the Nginx source code and then upload the Nginx source code to the /home directory on the VM.
    wget https://nginx.org/download/nginx-1.21.5.tar.gz --no-check-certificate
  3. Deploy Nginx.
    tar -zxvf nginx-1.21.5.tar.gz
cd nginx-1.21.5/
chmod 755 configure
./configure --prefix=/usr/local/nginx --user=nginx --group=nginx --with-http_ssl_module --with-http_v2_module --with-http_realip_module --with-http_stub_status_module --with-http_gzip_static_module --with-pcre --with-stream --with-stream_ssl_module --with-stream_realip_module
make -j 60 && make install
    • In the command, -j 60 can make full use of the multi-core feature of the CPUs to accelerate the compilation.
    • You can run the lscpu command to query the number of CPU cores.
  4. Generate an OpenSSL certificate.

    For details, see Generating an OpenSSL Certificate in the Nginx Porting Guide.

    If the message "unable to find 'distinguished_name' in config" is displayed during OpenSSL certificate generation, the command conflicts with the export OPENSSL_CONF=/home/openssl.cnf command used for testing the KAE performance in 7.c. See Failed to Generate an OpenSSL Certificate When Deploying Nginx on a VM During vKAE Deployment to rectify the fault.

  5. Check the Nginx installation directory.
    ls /usr/local/nginx
  6. Check that the Nginx version is the target version.
    /usr/local/nginx/sbin/nginx -v
  7. Configure and start the open source Nginx when KAE is disabled.
    1. Open the Nginx configuration file. 
      cd /usr/local/nginx/conf
vim nginx.conf
    2. Press i to enter the insert mode and copy the following content to the Nginx configuration file.

      The following is the content of the open source Nginx configuration file nginx.conf, which is not tuned. KAE is not enabled as well.

      user  root;
worker_processes  auto;

#worker_processes  10;
#worker_cpu_affinity 
#10000000000000000000000000000000000000000000000000000000000000000000000000000000000
#100000000000000000000000000000000000000000000000000000000000000000000000000000000000
#1000000000000000000000000000000000000000000000000000000000000000000000000000000000000
#10000000000000000000000000000000000000000000000000000000000000000000000000000000000000
#;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    server {
        listen       10000;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   html;
            index  index.html index.htm;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }

    # HTTPS server
    #
    server {
        listen       20000 ssl;
        server_name  localhost;

        ssl_certificate      /usr/local/nginx/server_2048.crt;
        ssl_certificate_key  /usr/local/nginx/server_2048.key;

        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;

        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers  on;

        location / {
            root   html;
            index  index.html index.htm;
        }
    }

}

      The HTTP listening port number is 10000, and the HTTPS listening port number is 20000.

    3. Press Esc, type :wq!, and press Enter to save the file and exit.
    4. Run the open source Nginx and check whether Nginx is started. 
      /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf
ps -ef | grep nginx

      If the Nginx threads are returned, Nginx has been started.

      In the nginx.conf configuration file, worker_processes is set to auto, and the number of created Nginx threads is 8, which is equal to the number of cores of the VM specification 8C16G. Set the number of worker_processes as required.

      Commands for restarting and exiting Nginx.

      • Restart Nginx. 
        sudo systemctl restart nginx
      • Gracefully restart Nginx. 
        sudo nginx -s reload
      • Exit Nginx. 
        /usr/local/nginx/sbin/nginx -s quit
        or 
        /usr/local/nginx/sbin/nginx -s stop
  8. Configure KAE enabled + Nginx synchronous mode.
    1. Create a configuration file named nginx_kae.conf in the usr/local/nginx/conf directory. 
      vim nginx_kae.conf
    2. Press i to enter the insert mode and copy the following content to the Nginx configuration file.
      The following content of the Nginx configuration file nginx.conf is for configuring enabled KAE + Nginx synchronous mode. The Nginx parameters are tuned. 
      user  root;
worker_processes auto;
#4-7
#worker_cpu_affinity
#10000
#100000
#1000000
#10000000
#;
#daemon off;
error_log  /dev/null;

worker_rlimit_nofile 102400;
events {
        use epoll;
        worker_connections 102400;
        accept_mutex off;
        multi_accept on;
}


http {
        include       mime.types;
        default_type  application/octet-stream;
        #log_format  main  '$remote_addr - $remote_user [$time_local] $request_time "$request" '
        #        '$status $body_bytes_sent $request_length $bytes_sent "$http_referer" '
        #        '"$http_user_agent" "$http_x_forwarded_for"';
        #access_log  logs/access.log  main;
        access_log  off;

        sendfile      on;
        tcp_nopush    on;
        tcp_nodelay   on;
        server_tokens off;
        sendfile_max_chunk 512k;
        keepalive_timeout  65;
        keepalive_requests 20000;
        client_header_buffer_size 4k;
        large_client_header_buffers 4 32k;
        server_names_hash_bucket_size 128;
        client_max_body_size 100m;
        open_file_cache max=102400 inactive=40s;
        open_file_cache_valid 50s;
        open_file_cache_min_uses 1;
        open_file_cache_errors on;
        #gzip  on;

    server {
        listen       10000 reuseport;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   html;
            index  index.html index.htm;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }
    # HTTPS server
    #
    server {
        listen       20000 ssl reuseport;
        server_name  localhost;

        ssl_certificate  /usr/local/nginx/server_2048.crt;
        ssl_certificate_key  /usr/local/nginx/server_2048.key;

        ssl_session_cache    shared:SSL:1m;
        ssl_session_timeout  5m;
        ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers  AES256-GCM-SHA384;
        ssl_prefer_server_ciphers  on;
        ssl_session_tickets  off;
        location / {
            root   html;
            index  index.html index.htm;
        }
        access_log  off;
    }

}
    3. Press Esc, type :wq!, and press Enter to save the file and exit.
    4. Run the configuration file for enabling KAE + Nginx synchronous mode with parameters tuned.

      To run the configuration file for enabling KAE + Nginx synchronous mode with parameters tuned, you only need to add OPENSSL_CONF=/home/openssl.cnf before the Nginx execute command.

      /usr/local/nginx/sbin/nginx -s stop || true; sleep 1;
OPENSSL_CONF=/home/openssl.cnf /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx_kae.conf
  9. Configure KAE enabled + Nginx asynchronous mode.

    To enable KAE + Nginx asynchronous mode, you need to download the Nginx source code that adapts to the asynchronous mode. The source code supports synchronous or asynchronous modes and adapts to the KAE or Intel QAT hardware acceleration.

    1. Download the Nginx source code (version 0.4.9 in GitHub) that adapts to the asynchronous mode, and compile and install Nginx. 
      cd /home
git clone https://github.com/intel/asynch_mode_nginx.git
cd /home/asynch_mode_nginx/
yum install gcc gcc-c++ make libtool zlib zlib-devel pcre pcre-devel perl-devel perl-ExtUtils-Embed perl-WWW-Curl wget -y
./configure --prefix=/usr/share/nginx --add-dynamic-module=modules/nginx\_qat\_module --with-cc-opt="-DNGX\_SECURE\_MEM -Wno-error=deprecated-declarations" --with-http\_ssl\_module --with-http\_v2\_module
make -j60 && make install

      In steps 7 and 8, the open source Nginx is used for testing and is in the /usr/local/nginx directory. To avoid conflicts, Nginx in asynchronous mode is installed in the /usr/share/nginx directory.

    2. Create a file named nginx_kae_async.conf in the /root directory. 
      vim nginx_kae_async.conf
    3. Press i to enter the insert mode and copy the following content to the nginx_kae_async.conf file.

      The following content of the Nginx configuration file nginx.conf is for enabling KAE and the Nginx asynchronous mode. The Nginx parameters are tuned and KAE is enabled. Change the number of Nginx processes as required. Generally, setting worker_processes to auto will occupy all cores of the VM. The HTTP port number is 10000, and the HTTPS port number is 20000.

      # For more information on configuration, see:
#   * Official English Documentation: http://nginx.org/en/docs/
#   * Official Russian Documentation: http://nginx.org/ru/docs/

user root;
worker_processes auto;

#worker_processes  10;
#worker_cpu_affinity 
#10000000000000000000000000000000000000000000000000000000000000000000000000000000000
#100000000000000000000000000000000000000000000000000000000000000000000000000000000000
#1000000000000000000000000000000000000000000000000000000000000000000000000000000000000
#10000000000000000000000000000000000000000000000000000000000000000000000000000000000000
#;

events  {
    use epoll;
    worker_connections 102400;
    accept_mutex off;
    multi_accept on;
}

error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

include /usr/share/nginx/modules/*.conf;

http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    # access_log off;
    # access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65s;
    types_hash_max_size 4096;

    include             /usr/local/nginx/conf/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/conf.d/*.conf;
        access_log  off;
        server_tokens off;
        sendfile_max_chunk 512k;
        keepalive_requests 20000;
        client_header_buffer_size 4k;
        large_client_header_buffers 4 32k;
        server_names_hash_bucket_size 128;
        client_max_body_size 100m;
        open_file_cache max=102400 inactive=40s;
        open_file_cache_valid 50s;
        open_file_cache_min_uses 1;
        open_file_cache_errors on;

    server {
        listen       10000;
        listen       [::]:10000;
        location / {
            root html;
            index index.html index.htm;
        }
        error_page 500 502 503 504  /50x.html;
        location = /50x.html {
            root html;
        }
    }

# Settings for a TLS enabled server.
#
   server {
       listen 20000 ssl http2 asynch;
       listen [::]:20000 ssl http2 asynch;
       server_name localhost;
       ssl_asynch on;
       ssl_certificate /usr/local/nginx/server_2048.crt;
       ssl_certificate_key /usr/local/nginx/server_2048.key;
       ssl_session_cache shared:SSL:1m;
       ssl_session_timeout 5m;
       ssl_protocols  TLSv1 TLSv1.1 TLSv1.2;
       ssl_ciphers  "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256    EECDH+aRSA+RC4 EECDH EDH+aRSA !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS !RC4";

       ssl_prefer_server_ciphers  on;

       location / {
            root html;
            index index.html index.htm;
      }

   }
    gzip on;
    gzip_buffers 4 16k;
    gzip_comp_level 9;
    gzip_disable "MSIE [1-6]\.";
    gzip_http_version 1.1;
    gzip_min_length 500k;
    gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml;
    gzip_vary on;
    proxy_buffer_size 1024k;
    proxy_buffers 16 1024k;
    proxy_busy_buffers_size 2048k;
    proxy_temp_file_write_size 2048k;
 }
    4. Press Esc, type :wq!, and press Enter to save the file and exit.
    5. Reuse the OpenSSL certificate path (/usr/local/nginx/conf/mime.types) after the open source Nginx installation.

      Copy the mime.types, server_2048.crt, and server_2048.key files to a new path.

      To create an OpenSSL certificate by yourself, run the following command in the new path:

      openssl genrsa -des3 -out server_2048.key 2048
openssl rsa -in server_2048.key -out server_2048.key
openssl req -new -key server_2048.key -out server_2048.csr
openssl rsa -in server_2048.key -out server_2048.key
openssl x509 -req -days 365 -in server_2048.csr -signkey server_2048.key -out server_2048.crt
    6. Run the configuration file for enabling KAE + Nginx asynchronous mode with parameters tuned.

      To run the configuration file for enabling KAE + Nginx asynchronous mode with parameters tuned, you only need to add OPENSSL_CONF=/home/openssl.cnf before the Nginx execute command.

      /usr/share/nginx/sbin/nginx -s stop || true; sleep 1;
OPENSSL_CONF=/home/openssl.cnf /usr/share/nginx/sbin/nginx -c /root/nginx_kae_async.conf
Parent topic: Deployment