Related Concepts

Understand the following concepts before using OmniRuntime features.

• OmniVec: An efficient off-heap memory data organization method. It supports zero-copy data read and has no serialization overhead.
• Omni operators: High-performance operators, which use native code (C/C++) to replace physical operators at the bottom layer of big data, increasing the computing speed.
• In-memory shuffle: During shuffle, data is cached in the memory instead of being directly written to drives. In this way, the drive I/O overhead is reduced to improve the data processing efficiency.
• OCKD process: After OmniShuffle is installed, you can use the OCKD process to start or stop OmniShuffle.
• Remote Shuffle Service (RSS): The shuffle service is deployed on a node outside the Spark cluster and shuffles data remotely.
• External Shuffle Service (ESS): The shuffle service is deployed on the compute nodes of the Spark cluster and shuffles data on the compute nodes.