Abnormal Database Connection or Active/Standby Master Switchover

Symptom

The database connection or active/standby Master switchover is abnormal, and the following error message is displayed:

FATAL:  remaining connection slots are reserved for non-replication superuser connections

Key Process and Cause Analysis

The value of max_connections in the postgresql.conf configuration file of the database is too small. The administrator needs to manually change the value.

Conclusion and Solution

Use PuTTY to log in to the server as the DonauKit O&M user, for example, donau_guest.
Run the following command to switch to the root user and enter the password as prompted:
su - root
Change the maximum number of database connections. In the HA scenario of databases, you can modify the configuration file on any database node.
1. Run the following command to open the postgresql.conf configuration file:
  vi /usr/local/postgresql/data/postgresql.conf
  In the preceding command:
  
  /usr/local/postgresql/data is the database PGDATA path. In the Kylin environment, the path is /var/lib/pgsql/data.
  In the HA scenario, the directory is a shared directory. Enter the command based on the actual directory where the configuration file is located.
2. Press i to enter the edit mode and modify configuration items in the max_connections file.
  Change the maximum number of database connections as required. Ensure that the maximum number of database connections is greater than the configured number of Donau Scheduler connections. The following is an example:
```
max_connections = 1000                # (change requires restart)
```
  For details about the Donau Scheduler database configurations, see the db.connectNum configuration item in the /opt/batch/master/conf/master.properties configuration file. By default, this configuration item is not displayed. If this configuration item is not explicitly set, the default value 200 is used.
3. Press Esc, type :wq!, and press Enter to save the settings and exit.
After the configuration file is modified, reload it.
- Non-HA scenario of databases:
  systemctl reload postgresql.service
- HA scenario of databases:
  1. Run the following command to query the active database node:
    pcs status
    
    If information similar to the following is displayed, the database is running on the node whose host name is ccsdb01:
```
pgsql      (ocf::heartbeat:pgsql): Started ccsdb01
```
  2. Use PuTTY to log in to the active database node as the DonauKit O&M user, for example, donau_guest.
  3. Run the following command to switch to the root user and enter the password as prompted:
    su - root
  4. Run the following command to switch to user postgres:
    su - postgres
  5. Run the following command to reload the configuration file:
    pg_ctl reload
After the database service is restarted, check whether the configuration takes effect.
1. Use PuTTY to log in to the database node as the DonauKit O&M user, for example, donau_guest.
2. Run the following command to switch to the root user and enter the password as prompted:
  su - root
3. Run the following command to switch to user postgres:
  su - postgres
4. Run the following command to access the database console:
  psql -p 15432
  
  Enter the password of the postgres database administrator.
```
Password for user postgres:
```
  15432 indicates the database port number. Replace it with the actual value.
5. Run the following command to query the maximum number of connections:
  show max_connections;
  
  The following is an example:
```
 max_connections
-----------------
 1000
(1 row)
```
6. Run the following command to exit the database:
  \q