Starting the Storage Service
This section describes how to deploy the Storage Service.
- Obtain the binary file and configuration file from the compilation server.
mkdir -p /opt/3fs/{bin,etc} mkdir -p /var/log/3fs rsync -avz meta:/home/3fs/build/bin/storage_main /opt/3fs/bin rsync -avz meta:/home/3fs/configs/{storage_main_launcher.toml,storage_main.toml,storage_main_app.toml} /opt/3fs/etc rsync -avz meta:/home/3fs/deploy/systemd/storage_main.service /usr/lib/systemd/system rsync -avz meta:/usr/lib64/libfdb_c.so /usr/lib64 - Update the storage_main_app.toml configuration file.
- Open /opt/3fs/etc/storage_main_app.toml.
vim /opt/3fs/etc/storage_main_app.toml
- Press i to enter the insert mode and modify the configuration file as follows.
allow_empty_node_id = true node_id = 10001 # Update the node ID. Note that the node ID of each storage node must be unique.
- Press Esc to exit the insert mode. Type :wq! and press Enter to save the file and exit.
- Open /opt/3fs/etc/storage_main_app.toml.
- Update the storage_main_launcher.toml configuration file.
- Open /opt/3fs/etc/storage_main_launcher.toml.
vim /opt/3fs/etc/storage_main_launcher.toml
- Press i to enter the insert mode and modify the configuration file as follows.
... cluster_id = "stage" ... [mgmtd_client] mgmtd_server_addresses = ["RDMA://192.168.65.10:8000"] ...
- Press Esc to exit the insert mode. Type :wq! and press Enter to save the file and exit.
- Open /opt/3fs/etc/storage_main_launcher.toml.
- Update the storage_main.toml configuration file.
- Open /opt/3fs/etc/storage_main.toml.
vim /opt/3fs/etc/storage_main.toml
- Press i to enter the insert mode and modify the configuration file as follows.
... [server.base.groups.listener] filter_list = ['enp133s0f0np0'] # Enter the RDMA NIC name. listen_port = 8000 ... [server.base.groups.listener] filter_list = ['enp133s0f0np0'] # Enter the RDMA NIC name. listen_port = 9000 ... [server.mgmtd] mgmtd_server_address = ["RDMA://192.168.65.10:8000"] ... [common.monitor.reporters.monitor_collector] remote_ip = "192.168.65.10:10000" ... [server.targets] target_paths = ["/storage/data0/3fs","/storage/data1/3fs","/storage/data2/3fs","/storage/data3/3fs","/storage/data4/3fs","/storage/data5/3fs","/storage/data6/3fs","/storage/data7/3fs"] ...
- Press Esc to exit the insert mode. Type :wq! and press Enter to save the file and exit.
- Open /opt/3fs/etc/storage_main.toml.
- Update the configuration of each storage node to the Mgmtd Service.
/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "set-config --type STORAGE --file /opt/3fs/etc/storage_main.toml"
- Start the service.
systemctl start storage_main
- Check the service status.
systemctl status storage_main

- List nodes in the cluster.
/opt/3fs/bin/admin_cli -cfg /opt/3fs/etc/admin_cli.toml "list-nodes"
Command output:
# Id Type Status Hostname Pid Tags LastHeartbeatTime ConfigVersion ReleaseVersion # 1 MGMTD PRIMARY_MGMTD meta 2281735 [] N/A 1(UPTODATE) 250228-dev-1-999999-923bdd7c # 100 META HEARTBEAT_CONNECTED meta 2281842 [] 2025-03-12 17:01:32 1(UPTODATE) 250228-dev-1-999999-923bdd7c # 10001 STORAGE HEARTBEAT_CONNECTED storage1 3294593 [] 2025-03-12 17:38:13 1(UPTODATE) 250228-dev-1-999999-923bdd7c # 10002 STORAGE HEARTBEAT_CONNECTED storage2 476286 [] 2025-03-12 17:38:12 1(UPTODATE) 250228-dev-1-999999-923bdd7c # 10003 STORAGE HEARTBEAT_CONNECTED storage3 2173767 [] 2025-03-12 17:38:12 1(UPTODATE) 250228-dev-1-999999-923bdd7c
If no storage nodes are displayed, wait for 1 to 3 minutes.
Parent topic: Deploying the Storage Service