Installing OmniData

Distribute the OmniData package to all nodes, decompress the package, and install and initialize OmniData.

1. Copy the OmniData binary package prepared in Obtaining Software to all nodes using the file transfer function.
   1. Use the batch distribution function to select the offload nodes (ceph1, ceph2, and ceph3) and configure a service flow.
   2. Choose File Transfer, click Add a Task, set the task parameters according to Table 1, and click Save.

      Table 1 Task 1 parameters

      Parameter	Description
      Task Name	Transferring the OmniData installation package
      Task Description	Copying the OmniData installation package to offload nodes
      Transfer Direction	From Local to Remote
      Local Path	Directory where the local installation package is stored
      Remote Path	/home/omm/omnidata-software

2. Copy the OmniData third-party dependencies prepared in Obtaining Software to offload nodes using the file transfer function.
   Choose File Transfer, click Add a Task, set the task parameters according to Table 2 to Table 4, and click Save.

   Table 2 Task 2 parameters

   Parameter	Description
   Task Name	Copying the OmniData lib package
   Task Description	Copying the OmniData lib package to offload nodes
   Transfer Direction	From Local to Remote
   Local Path	Directory where the local installation package is stored
   Remote Path	/home/omm/omnidata-software

   Table 3 Task 3 parameters

   Parameter	Description
   Task Name	Copying Ceph dependencies
   Task Description	Copy the Ceph dependencies to all offload nodes. (Skip this step for an HDFS cluster.)
   Transfer Direction	From Local to Remote
   Local Path	Directory where the local installation package is stored
   Remote Path	/home/omm/omnidata-software

   Table 4 Task 4 parameters

   Parameter	Description
   Task Name	Copying the librgw file
   Task Description	Copying the librgw dependency to all offload nodes (Skip this step for an HDFS cluster.)
   Transfer Direction	From Local to Remote
   Local Path	Directory where the local installation package is stored
   Remote Path	/home/omm/omnidata-software

3. Decompress the OmniData package boostkit-omnidata-server-1.5.0-aarch64.tar.gz and copy the third-party dependencies to the corresponding directories.
   Select Command Execution, click Add a Task, set the task parameters according to Table 5 to Table 8, and click Save.

   Table 5 Task 5 parameters

   Parameter	Description
   Task Name	Decompressing the OmniData package
   Task Description	Decompressing the OmniData installation package and third-party dependencies
   Run Directory	/home/omm/omnidata-software
   Run Command	unzip BoostKit-omnidata_1.5.0.zip; tar -zxvf BoostKit-omnidata_1.5.0.tar.gz;tar -zxvf boostkit-omnidata-server-1.5.0-aarch64.tar.gz -C /home/omm/omnidata-install; unzip boostkit-omnidata-server-1.5.0-aarch64-lib.zip;
   Whether to Check Return Code	Yes
   Expected Return Code	0

   Table 6 Task 6 parameters

   Parameter	Description
   Task Name	Copying OmniData dependencies
   Task Description	Copying third-party dependencies to the corresponding OmniData directories
   Run Directory	/home/omm/omnidata-software
   Run Command	mv /home/omm/omnidata-software/boostkit-omnidata-server-1.5.0-aarch64-lib/* /home/omm/omnidata-install/omnidata/lib;rmdir /home/omm/omnidata-software/boostkit-omnidata-server-1.5.0-aarch64-lib;cd /home/omm/omnidata-install;cp -f ./haf-offload/lib/jar/haf-1.3.0.jar ./omnidata/lib/;
   Whether to Check Return Code	Yes
   Expected Return Code	0

   Table 7 Task 7 parameters

   Parameter	Description
   Task Name	Copying the feature combination dependency package to the corresponding OmniData directory
   Task Description	Copy the feature combination dependency package to the corresponding OmniData directory (boostkit-omniop-bindings-1.3.0-aarch64.jar. Skip this step if feature combination is not involved.)
   Transfer Direction	From Local to Remote
   Local Path	Directory where the local installation package is stored
   Remote Path	/home/omm/omnidata-install/omnidata/lib

   Table 8 Task 8 parameters

   Parameter	Description
   Task Name	Copying Ceph dependencies
   Task Description	Copying Ceph dependencies to the corresponding OmniData directories (Skip this step for an HDFS cluster.)
   Run Directory	/home/omm/omnidata-software
   Run Command	cd /home/omm/omnidata-install;chmod 700 ./haf-offload/lib/; cp /home/omm/omnidata-software/librgw_jni.so ./haf-offload/lib/;cp /usr/lib64/ceph/libceph-common.so.0 ./haf-offload/lib/;chmod -R 500 ./haf-offload/lib/;cp /home/omm/omnidata-software/hdfs-ceph-3.2.0.jar ./omnidata/lib/;
   Whether to Check Return Code	Yes
   Expected Return Code	0

4. Install the OmniData service on the offload nodes (ceph1, ceph2, and ceph3).
   Select Command Execution, click Add a Task, set the task parameters according to Table 9, and click Save.

   Table 9 Task 9 parameters

   Parameter	Description
   Task Name	Installing the OmniData service
   Task Description	Installing the OmniData service
   Run Directory	/home/omm/omnidata-install/omnidata-setup
   Run Command	sh omnidata_install.sh
   Whether to Check Return Code	Yes
   Expected Return Code	0

5. Create configuration files for the OmniData service on the offload nodes (ceph1, ceph2, and ceph3).
   Select Command Execution, click Add a Task, set the task parameters according to Table 10, and click Save.

   Table 10 Task 10 parameters

   Parameter	Description
   Task Name	Creating a configuration file
   Task Description	Creating configuration files for the OmniData service
   Run Directory	/home/omm/omnidata-install/omnidata/deploy
   Run Command	sh generate_config.sh -s Register_HOSTNAME:Register_Port
   Whether to Check Return Code	Yes
   Expected Return Code	0

   

   • If the big data engine is Spark, run sh generate_config.sh -s Register_IP:Register_Port.
   • The script parameter -s is used to specify the IP address/domain name and port number of a third-party registered service used by OmniData. (The default IP address and port number of the ZooKeeper node are 2181.)
6. Export the service flow and save it to your local PC. You can import service flows to quickly install other components.

   Click Export in the upper right corner, name the template OmniData Installation and Configuration Workflow, and click Save.

7. Execute the service flow.
   

   • You can develop your own OmniData plugins and ensure the security of the plugins.
   • OmniData can have the Hive UDF plugin. You can add your own UDF plugin. To add a plugin, perform the following operations:
     1. Add a task similar to task 2 of step 2 in Installing OmniData, for example, copying the Hive UDF plugin.
     2. In task 5 of step 3 in Installing OmniData, add the unzip boostkit-omnidata-hive-udf-loader-1.6.1-1.4.0-aarch64.zip command to the beginning of the commands.
     3. In task 6 of step 3 in Installing OmniData, add the mv /home/omm/omnidata-software/boostkit-omnidata-hive-udf-loader-1.6.1-1.4.0-aarch64 /home/omm/omnidata-install/omnidata/plugin command to the beginning of the commands.