Running and Verification

Use PuTTY to log in to the server as the root user.
Obtain the case file from the following link:
https://www2.mmm.ucar.edu/WG2bench/conus_2.5_v3/1-RST/

Three target files: wrfrst_d01_2005-06-04_06_00_00, namelist.input, and wrfbdy_d01

Run the following commands on PuTTY to create the /path/to/WRFTEST directory and upload the case to the directory:

mkdir -p /path/to/WRFTEST
cd /path/to/WRFTEST
wget https://www2.mmm.ucar.edu/WG2bench/conus_2.5_v3/1-RST/RST/ -r -c -np -nH --cut-dirs 2 --restrict-file-names=nocontrol -e robots=off --reject "index.html*"
wget https://www2.mmm.ucar.edu/WG2bench/conus_2.5_v3/wrfbdy_d01.gz -r -c -np -nH --cut-dirs 2 --restrict-file-names=nocontrol -e robots=off --reject "index.html*"
wget https://www2.mmm.ucar.edu/WG2bench/conus_2.5_v3/1-RST/namelist.input -r -c -np -nH --cut-dirs 2 --restrict-file-names=nocontrol -e robots=off --reject "index.html*"

Run the following command to place namelist.input in the test directory and delete unnecessary directories:
```
mv 1-RST/namelist.input ./
```

Run the following commands to generate a case and delete unnecessary directories:

cat 1-RST/RST/rst_6hr* | gunzip -c > wrfrst_d01_2005-06-04_06_00_00
rm -rf 1-RST

Run the following command to decompress the wrfbdy_d01 file:
```
gunzip wrfbdy_d01.gz
```

Run the following command to modify the namelist.input file:

Open the namelist.input file.
```
vim namelist.input
```

Press i to enter the insert mode and modify the following information:

&time_control
run_days                            = 0,
run_hours                       = 3,
run_minutes                         = 0,
run_seconds                         = 0,
start_year                          = 2005,
start_month                         = 06,
start_day                           = 04,
start_hour                          = 06,
start_minute                        = 00,
start_second                        = 00,
end_year                            = 2005,
end_month                           = 06,
end_day                             = 04,
end_hour                            = 12,
end_minute                          = 00,
end_second                          = 00,
interval_seconds                = 10800
input_from_file                     = .true.,
history_interval                    = 180,
frames_per_outfile                  = 1,
nocolons                            = .true.,
restart                             = .true.,
restart_interval                    = 360,
io_form_history                     = 11,
io_form_restart                     = 11,
io_form_input                       = 2,
io_form_boundary                    = 2,
io_form_auxhist2                    = 2,
debug_level                         = 0
/
&domains
time_step                           = 15,
time_step_fract_num                 = 0,
time_step_fract_den                 = 1,
max_dom                             = 1,
s_we                                = 1,
e_we                                = 1501,
s_sn                                = 1,
e_sn                                = 1201,
s_vert                              = 1,
e_vert                              = 35,
dx                                  = 2500,
dy                                  = 2500,
grid_id                             = 1,
parent_id                           = 0,
i_parent_start                      = 0,
j_parent_start                      = 0,
parent_grid_ratio                   = 1,
parent_time_step_ratio              = 1,
feedback                            = 1,
smooth_option                       = 0
lagrange_order                      = 2
interp_type                         = 2
extrap_type                         = 2
t_extrap_type                       = 2
use_surface                         = .true.
use_levels_below_ground             = .true.
num_metgrid_levels                  = 40,
eta_levels                          = 1.00000 , 0.99258 , 0.98275 , 0.96996 , 0.95372 ,
0.93357 , 0.90913 , 0.87957 , 0.84531 , 0.80683 ,
0.76467 , 0.71940 , 0.67163 , 0.62198 , 0.57108 ,
0.51956 , 0.46803 , 0.42030 , 0.37613 , 0.33532 ,
0.29764 , 0.26290 , 0.23092 , 0.20152 , 0.17452 ,
0.14978 , 0.12714 , 0.10646 , 0.08761 , 0.07045 ,
0.05466 , 0.03981 , 0.02580 , 0.01258 , 0.00000
/
&physics
mp_physics                          = 4,
mp_zero_out                         = 2,
mp_zero_out_thresh                  = 1.e-9,
ra_lw_physics                       = 1,
ra_sw_physics                       = 1,
radt                                = 10,
sf_sfclay_physics                   = 1,
sf_surface_physics                  = 2,
bl_pbl_physics                      = 1,
bldt                                = 0,
cu_physics                          = 0,
cudt                                = 5,
isfflx                              = 1,
ifsnow                              = 0,
icloud                              = 1,
surface_input_source                = 1,
num_soil_layers                     = 4,
maxiens                             = 1,
maxens                              = 1,
maxens2                             = 1,
maxens3                             = 1,
ensdim                              = 1,
/
&dynamics
w_damping                           = 1,
diff_opt                            = 1,
km_opt                              = 4,
khdif                               = 0,
kvdif                               = 0,
non_hydrostatic                     = .true.,
use_baseparam_fr_nml                = .t.,
/
&bdy_control
spec_bdy_width                      = 5,
spec_zone                           = 1,
relax_zone                          = 4,
specified                           = .true.,
nested                              = .false.,
/
&namelist_quilt
nio_tasks_per_group = 0,
nio_groups = 1,
/

Press Esc, type :wq!, and press Enter to save the file and exit.

Run the following command to copy all files in the /path/to/WRF/WRF-3.8.1/run directory to the /path/to/WRFTEST directory:
```
cp /path/to/WRF/WRF-3.8.1/run/* /path/to/WRFTEST/
```

Run the following commands to load environment variables:

export PATH=/path/to/GNU/bin:/path/to/OPENMPI/bin:$PATH
export LD_LIBRARY_PATH=/path/to/GNU/lib64:/path/to/OPENMPI/lib:/path/to/NETCDF/lib:/path/to/PNETCDF/lib:/path/to/HDF5/lib:/path/to/MATH/lib:$LD_LIBRARY_PATH

Run the following command to create the hostfile file:
1. Create hostfile.
```
vi hostfile
```
2. Press i to enter the insert mode and add the following content:
```
Node1
Node2
```
3. Press Esc, type :wq!, and press Enter to save the file and exit.

Run the following command to start the WRF program.

clush --hostfile hostfile "sync;echo 3 > /proc/sys/vm/drop_caches"

If an IB network is used, run the following command:

time -p `which mpirun` --allow-run-as-root -x PATH=$PATH -x LD_LIBRARY_PATH=$LD_LIBRARY_PATH -x LD_PRELOAD=/path/to/MATH/lib/libmathlib.so -x OMP_NUM_THREADS=4 -map-by ppr:32:node:pe=4 --bind-to core --hostfile hostfile --mca pml ucx ./wrf.exe

If an RoCE network is used, run the following command:

time -p mpirun --allow-run-as-root -hostfile hostfile -x PATH -x LD_LIBRARY_PATH -x LD_PRELOAD=/path/to/MATH/lib/libmathlib.so -x OMP_NUM_THREADS=4 -map-by ppr:24:node:pe=4 -bind-to core -display-map -mca pml ucx -mca btl ^vader,tcp,openib,uct -x UCX_TLS=self,sm,rc -x UCX_NET_DEVICES=mlx5_0:1 -x UCX_IB_GID_INDEX=5 ./wrf.exe

--hostfile hostfile: specifies the list of node names to be used.
-x OMP_NUM_THREADS=4: specifies the number of threads to be used.
-map-by ppr:32:node:pe=4: specifies how processes and threads are bound. Each node has 32 processes, and each process has four threads. (The 128-core CPU on a single node is used as an example.)
UCX_NET_DEVICES: specifies the RoCE network port.
UCX_IB_GID_INDEX: specifies the RoCE network type.

Run the following command to check whether the program has stopped properly:
```
less rsl.out.0000
```

Parent topic: WRF 3.8.1 Porting Guide (CentOS 7.6)