Detailed Control Plane Development

Preparations

Create a directory for developing the OVS control plane in a custom code project.

The following is an example. For details, see Custom Code Project Directory.

ovs_project/
├── CMakeLists.txt                   # Top-level CMake configuration file
├── src_control_plane                # Directory storing the user's OVS control plane source code
    ├── CMakeLists.txt               # CMake configuration file of the OVS control plane
    └── ...                          # Source code of the OVS control plane
└── src_dsl                          # Subdirectory storing user data plane source code
    ├── CMakeLists.txt               # CMake configuration file in the data plane subdirectory
    └── ovs_project.hdr              # Data plane Hydra source file

Copy the hydra_info.h header file generated during firmware compilation to src_control_plane.
For details, see Compilation Procedure.
```
cd {$project_path}/ovs_project
cp src_dsl/build/hydra_info.h src_control_plane/
```
Include the OVS control plane header files (hook.h and libapi.h) provided by the programming framework into the code.
Copy hook.h and libapi.h in flexda_sdk/include/control_plane/ to src_control_plane.
```
cd {$sdk_path}/flexda_sdk/
cp include/control_plane/* {$project_path}/ovs_project/src_control_plane/
```

Procedure 1: Developing OVS Control Plane Code

Develop the OVS control plane source code and implement the hook functions defined in hook.h using the code and the interfaces provided in libapi.h.

The global initialization and deinitialization hook functions are optional, and other hook functions are mandatory. If the implementation of the mandatory hook functions is not provided in the source code, OVS will fail to start after the compilation and installation of the product.

Prior to developing the code, identify the packet types that require hardware flow entry issuance and their characteristics based on the service scenarios and the Hydra data plane code. Then, extract information from the Hydra data plane code and hydra_info.h to determine the target table of flow entry issuance for each packet type, match fields (keys) within the table, and modifications to the flow table actions.

The following takes the scenario of processing input VLAN traffic and executing VLAN POP and inner RSS actions as an example.

Hydra data plane code

#include "hydra_model.hdr"
#include "hydra_parser.hdr"
#include "hydra_externs.hdr"
#include "hydra_builtin.hdr"
#define VPORT_TYPE_BOND 3
 
control MyMainPipe(in hdr_fix_headers hdr) {
    rte_vlan_hdr_t vlan_key;
    bit<16> inner_eth;
    @main
    table vlan_exact {
        key = {
            vlan_key         : exact;
            inner_eth        : exact;
        }
        actions = {
            rte_of_pop_vlan;
            rte_port_id;
            flexda_inner_rss;
        }
        size = 0x100000;
        default_action = flexda_no_action;
        aging_time = 0;
    }
    apply {
        if ((hdr_input_port_id_get() >> 12) == VPORT_TYPE_BOND) { // Determine that the packet is an RX packet.
            if (hdr.vlan.isValid()) { // Determine that the packet is a VLAN RX packet.
                if (hdr.ethernet.isValid()) {
                    inner_eth = hdr.ethernet.eth_type; // Extract the VLAN type.
                }
                vlan_key.vlan_tci = (bit<16>)((((bit<16>)hdr.vlan.pri) << 13) | ((bit<16>)hdr.vlan.vid)); // Extract the VLAN ID.
                if (vlan_exact.apply().miss) { // Flow table matching
                    hdr_ovs_pipeline_action_upcall();
                }
            } 
        }
    }
}
 
control MyPostPipe(in hdr_fix_headers hdr) {
    apply {}
}
 
Switch(
        TX_OVS(hdr_fix_parser(),MyMainPipe(),MyPostPipe()),
        RX_OVS(hdr_fix_parser(),MyMainPipe(),MyPostPipe())
) main;

hydra_info.h generated on the Hydra data plane

... // Definitions of some Hydra structures
enum hydra_group_type { // table id
  HYDRA_TABLE_INVALID = 0x0000,
  HYDRA_TABLE_MYMAINPIPE_VLAN_EXACT,
};
enum hydra_default_action_type { // Enumeration of the Hydra default action type
  DEFAULT_FLOW_ACTION_TYPE_UPCALL = 359,
  DEFAULT_FLOW_ACTION_TYPE_CT = 357,
  DEFAULT_FLOW_ACTION_TYPE_INNER_RSS = 364,
};
enum hydra_action_type { // Enumeration of the Hydra custom action type
  HYDRA_ACTION_INVALID = 0x1000,
};
enum hydra_item_type { // Enumeration of the Hydra custom key type
  HYDRA_ITEM_INVALID = 0x2000,
  HYDRA_ITEM_MYMAINPIPE_VLAN_EXACT_INNER_ETH,
};
int table_patterns_mymainpipe_vlan_exact [2] = { // Key type contained in the table
  RTE_FLOW_ITEM_TYPE_VLAN,
  HYDRA_ITEM_MYMAINPIPE_VLAN_EXACT_INNER_ETH,
};
int table_actions_mymainpipe_vlan_exact [4] = { // Action type contained in the table
  RTE_FLOW_ACTION_TYPE_COUNT,
  RTE_FLOW_ACTION_TYPE_OF_POP_VLAN,
  RTE_FLOW_ACTION_TYPE_PORT_ID,
  DEFAULT_FLOW_ACTION_TYPE_INNER_RSS,
};
... //

When the Hydra data plane processes VLAN RX traffic, it extracts the VLAN type (inner_eth) and VLAN ID from the packet header as the key to perform matching in the vlan_exact table. The executable actions include rte_of_pop_vlan, rte_port_id, and flexda_inner_rss.

hydra_info.h defines the following enumeration values:

Table enumeration value for vlan_exact: HYDRA_TABLE_MYMAINPIPE_VLAN_EXACT
Key enumeration value for inner_eth: HYDRA_ITEM_MYMAINPIPE_INNER_ETH_TYPE
Key enumeration value for vlan_key: RTE_FLOW_ITEM_TYPE_VLAN

VLAN RX traffic first hits an OVS software flow table configured with the RTE_FLOW_ACTION_TYPE_OF_POP_VLAN and RTE_FLOW_ACTION_TYPE_PORT_ID actions. When assembling a hardware flow entry, the DEFAULT_FLOW_ACTION_TYPE_INNER_RSS action, which is not included in the software flow table, must be added.

The preceding analysis can be summarized as follows:

**Table 1** Example of control plane code analysis before development
Packet Type	Characteristic	Table	Key	Action
VLAN RX	The OVS flow table contains the VLAN POP action.	HYDRA_TABLE_MYMAINPIPE_VLAN_EXACT	RTE_FLOW_ITEM_TYPE_VLAN	Add DEFAULT_FLOW_ACTION_TYPE_INNER_RSS.
VLAN RX	The OVS flow table contains the VLAN POP action.	HYDRA_TABLE_MYMAINPIPE_VLAN_EXACT	HYDRA_ITEM_MYMAINPIPE_INNER_ETH_TYPE	Add DEFAULT_FLOW_ACTION_TYPE_INNER_RSS.

Define global variables for the OVS runtime as required. Handle the initialization and deinitialization of these global variables within the implementation of the initialization and deinitialization hook functions.

flexda_custom_api_t g_api = {0};
/*... Other global variables ...*/
int flexda_custom_init(flexda_custom_api_t api)
{
    g_api = api;
    /*... Initialization of other global variables ...*/
    return 0;
}
int flexda_custom_deinit(void)
{
    /*... Deinitialization of other global variables ...*/
    return 0;
}

Based on service requirements, define the OVS control plane packet context (the same packet uses a single packet context across different flow entry issuing processes), and implement the hook functions for the initialization and deinitialization of the custom packet context.

The packet context must contain, at a minimum, the packet type flags and the original packet header information for each packet type.

// Refer to the following code to customize a packet context in the single-layer VLAN RX scenario.
struct custom_packet_ctx {
    // Packet scenario flag
    uint8_t vlan_poped;  // VLAN RX packet flag
    /* ... Other flags ...*/

    // Packet header information
    struct flexda_sub_hdr out_header;  // Outer L4, L3, and L2 packet header information
    /* ... Other packet header information ...*/

    // Packet contexts required by other services
    /* ... Other contexts ... */ 
};

int flexda_custom_init_ctx(void **ctx_addr)
{
    // Allocate memory for the custom packet context.
    struct custom_packet_ctx *ctx = (struct custom_packet_ctx *)flexda_calloc(1, sizeof(struct custom_packet_ctx));
    if (ctx == NULL) {
        return -EINVAL;
    }
    *ctx_addr = (void *)ctx;
    ctx->vlan_poped = 0;
    /* ... Other context initialization and assignment ...*/
    return 0;
}

int flexda_custom_deinit_ctx(void **ctx_addr)
{
    if (ctx_addr == NULL || *ctx_addr == NULL) {
        return -EINVAL;
    }
    struct custom_packet_ctx *ctx = (struct custom_packet_ctx *)*ctx_addr;
    // Free the memory allocated for the custom packet context.
    flexda_free(ctx);
    *ctx_addr = NULL;
    return 0;
}

Based on the service scenarios, implement the logic for updating the context and extracting packet headers within the hook function implementations using the following code instructions:

The flexda_custom_update_ctx function must, at a minimum, read the input parameter info to determine the packet type and pass this type into ctx.

The flexda_custom_extract_hdr function must, at a minimum, read the input parameter ctx to identify the packet type, parse the original packet header information for this type, and write the information into ctx.

// Refer to the following code. In this example, if the OVS flow table includes the VLAN_POP action, the packet scenario is identified as VLAN RX, and the vlan_poped flag is set to 1.
int flexda_custom_update_ctx(void *ctx, flexda_custom_info_t info)
{
    struct custom_packet_ctx *custom_ctx = (struct custom_packet_ctx *)ctx;
    struct rte_flow_action *actions = info.src_actions;
    for (int i = 0; i < 15; i++) {
        switch (actions[i]->type) {
            case RTE_FLOW_ACTION_TYPE_OF_POP_VLAN:
                custom_ctx->vlan_poped = 1;
                break;
            /*** Logic for updating the context flag in other scenarios ***/
            default:
                break;
        }
    }
    /*** Logic for updating the context in other scenarios ***/
    return 0;
}

// Refer to the following code. If vlan_poped is 1, extract the packet header information required to construct a key for the VLAN RX packet.
int flexda_custom_extract_hdr(void *ctx, flexda_custom_info_t info)
{
    struct custom_packet_ctx *custom_ctx = (struct custom_packet_ctx *)ctx;
    struct flexda_sub_hdr *out_header = &custom_ctx->out_header;  // Default packet header information structure provided by the framework, which is used together with the packet parsing API
    struct dp_packet *pkt = info.pkt;
    struct flexda_extract_ctx pkt_parser;  // Default packet parsing structure provided by the framework, which is used together with the packet parsing API
    pkt_parser.pkt_itr = (uint8_t *)flexda_dp_packet_data(pkt);
    pkt_parser.pkt_end = (uint8_t *)flexda_dp_packet_data(pkt) + flexda_dp_packet_size(pkt);
    if (custom_ctx->vlan_poped == 1) { 
        // Call the default packet header parsing API provided by the framework to parse and save the packet header information in the custom packet context.
        if (!flexda_extract_hdr(&pkt_parser, out_header)) { // Parsing failure
                return -EINVAL;
        }
        return 0; // Parsing success
    }
    /*** Logic for extracting packet header information in other scenarios ***/
    return false;// The flag bit is not 1.
}

Based on the service scenarios and the data plane logic implemented in the Hydra code, implement the hook functions for constructing the match key, action, and attribute.

The flexda_custom_construct_attr function must, at a minimum, read the input parameter ctx to identify the packet type and the corresponding original header information. It shall then write the table ID associated with that packet type (refer to 1 for the identification results) into attr->group.
The flexda_custom_construct_key function must, at a minimum, read the input parameter ctx to identify the packet type and the corresponding original header information. It shall then invoke relevant interfaces to write all match fields (refer to 1 for the identification results) of this packet type into hw_key.
The flexda_custom_construct_action function must, at a minimum, read the input parameter ctx to identify the packet type and the corresponding original header information. It shall then invoke relevant interfaces to add, delete, or convert the flow table actions in hw_action (refer to 1 for the identification results).

When specifying a table in flexda_custom_construct_attr, use the enum hydra_group_type in hydra_info.h. For details, see Group, Action, and Item Enumerations.
When inserting a key in flexda_custom_construct_key, find the key corresponding to the specific table in hydra_info.h. For details, see Table Key Information. Based on the matching logic defined in the Hydra code, invoke the appropriate interfaces to insert the required key. Use an enumeration value in Table Key Information for the key type.
When inserting an action in flexda_custom_construct_action, identify the supported action list for the specific table in hydra_info.h. For details, see Table Action Information. Insert or convert a Hydra action in the flow table according to service requirements. Use an enumeration value in Table Action Information for the action type, and assemble the action data according to the structures defined in Hydra Action Information.

// Note: The default group value of attr is 1 when the following function is called.
// In the following example code: When processing a VLAN RX packet (vlan_poped is set to 1), issue a flow entry to the VLAN exact table (HYDRA_TABLE_MYMAINPIPE_VLAN_EXACT) defined in the Hydra data plane.
int flexda_custom_construct_attr(void *ctx, flexda_custom_info_t info, struct rte_flow_attr *attr)
{
    struct custom_packet_ctx *custom_ctx = (struct custom_packet_ctx *)ctx;
    // Assign the Hydra table enumeration value to the group value of attr based on the custom context to specify the target table.
    if (custom_ctx->vlan_poped == 1) {
      attr->group = HYDRA_TABLE_MYMAINPIPE_VLAN_EXACT;
    }
    /*** Specifying the target table in other scenarios ***/
    return 0;
}

// Note: hw_key is empty when the following function is called. You need to insert all keys required by the Hydra data plane.
// In the following example code: When processing a VLAN RX packet (vlan_poped is set to 1), extract the vlan and encap fields from out_header, and invoke relevant interfaces to insert the required keys into hw_key. (The Hydra data plane code performs matching based on the vlan and encap fields when processing the VLAN RX packet.)
int flexda_custom_construct_key(void *ctx, flexda_custom_info_t info, struct rte_flow_item *hw_key)
{
    struct custom_packet_ctx *custom_ctx = (struct custom_packet_ctx *)ctx;
    struct flexda_sub_hdr *out_header = &custom_ctx->out_header;
    // Control the key construction process based on context conditions. The constructed key must match the key definitions specified for the table in Hydra code.
    if (custom_ctx->vlan_poped == 1) { // vlan pop
        
        struct rte_flow_item_vlan vlan = {0};
vlan.tci = custom_ctx->out_header.l2.vlan.vlan_id;
        flexda_insert_rte_key(hw_key, -1, &vlan, sizeof(rte_flow_item_vlan ), RTE_FLOW_ITEM_TYPE_VLAN);
        // Issue the VLAN eth_type key. Use the key insertion API that matches the custom key types defined in Hydra, and provide the length and type enumeration values that meet the Hydra code requirements.
        uint16_t 
encap = custom_ctx->out_header.l2.vlan.encap;
        flexda_insert_hydra_key(hw_key, -1, &encap, sizeof(uint16_t), HYDRA_ITEM_MYMAINPIPE_INNER_ETH_TYPE);
        /*** Other key construction code for matching the hardware flow table in the Hydra data plane VLAN POP scenario ***/
        return 0;
    }
    /*** Key construction code for hardware flow entry issuance in other scenarios ***/
    return -EINVAL;
}

// Note: hw_action used in this call is the concatenated action after flow entry normalization.
// In the following example code: When processing a VLAN RX packet (vlan_poped is set to 1), invoke the interface to add the RSS action (HWOFF_FLOW_ACT_DEF_INNER_RSS) supported by Hydra into hw_action.
int flexda_custom_construct_action(void *ctx, flexda_custom_info_t info, struct rte_flow_action *hw_action)
{
    struct custom_packet_ctx *custom_ctx = (struct custom_packet_ctx *)ctx;
    // Control the action construction process based on context conditions. The constructed action must match the action definitions specified for the table in Hydra code.
    
    if (custom_ctx->vlan_poped == 1) {
         
        uint8_t rss = 1;
        
        // Use the matching action insertion API with the custom action types defined in Hydra, and provide the length and type enumeration values that meet the Hydra code requirements.
        flexda_insert_hydra_action(hw_action, -1, &rss, sizeof(uint8_t), DEFAULT_FLOW_ACTION_TYPE_INNER_RSS);
    }
    /*** Other action editing code ***/
    return 0;
}

Implement the logic to allow OVS offload requests from tunnel ports.

int flexda_custom_check_tunnel(const char *devname)
{
    // If the OVS offload request comes from the tunnel port named gre, offloading is allowed.
    if (strcmp(devname, "gre") == 0) {
        return 0;
    }
    // If the request is from other ports (non-gre/vxlan/geneve/tap), offloading is not allowed.
    return -EINVAL;
}

Procedure 2: Compilation and Building

If necessary, package the OVS control plane development directory and send it to the compilation environment.
Develop a CMake configuration file for the OVS control plane, include the header files and source files in the src_control_plane directory, and set the compilation product name to custom_openvswitch_plugin.so.
Check whether the DPDK version in the compilation environment is correct.
For details about the DPDK version requirements, see Table 2.
```
pkg-config --modversion libdpdk
```
Confirm that the OS and architecture in the compilation environment are correct.
For details about the OS requirements, see Table 1.
```
uname -r
```

Perform compilation and building.

cd {project_path}
mkdir build; cd build; cmake ..; make;

Obtain the generated product custom_openvswitch_plugin.so from the build directory.

Procedure 3: Installing the Compilation Product

Upload custom_openvswitch_plugin.so to the /usr/lib64 directory in the operating environment.
Change the execute permissions to 550.

After installing or updating custom_openvswitch_plugin.so, restart the OVS software for the new version to take effect.
You need to grant root permissions for custom_openvswitch_plugin.so. If not, the loading of this dynamic library by libhiflexda_ovs_adapter.so may fail due to permission validation errors.
You are advised to set the permissions on custom_openvswitch_plugin.so to 550. Higher permissions will increase the risk of tampering.
The use of symbolic links for custom_openvswitch_plugin.so is not recommended to avoid potential security risks or access restrictions.

Special Scenarios

ovs_example.zip is used as an example (prepared by the user). Perform the following operations to develop the control plane:

Decompress the example code in the flexda_sdk directory.

The following shows the directory structure. geneve_single is used as an example.

flexda_sdk/example/geneve_single/
├── CMakeLists.txt                   # Top-level CMake configuration file
└── src_control_plane                # Subdirectory storing user control plane source code
    ├── CMakeLists.txt               # CMake configuration file in the control plane subdirectory
    ├── context.c                    # Control plane source code in C
    └── context.h                    # Control plane header file in C
└── src_dsl                          # Subdirectory storing user data plane source code
    ├── CMakeLists.txt               # CMake configuration file in the data plane subdirectory
    └── geneve_ovs.hdr               # Data plane Hydra source file

Compile the firmware by referring to Special Scenarios.

Copy the hydra_info.h header file generated during firmware compilation to src_control_plane.

cd {$sdk_path}/flexda_sdk/example/geneve_single/
cp src_dsl/build/hydra_info.h src_control_plane/

(Optional) If the current environment (firmware compilation environment) differs from the OVS control plane compilation environment, package flexda_sdk and send it from the current environment to the OVS control plane compilation environment.

# Execute the following commands in the current environment:
zip -r flexda_sdk.zip {$sdk_path}/flexda_sdk
scp flexda_sdk.zip {$username}@{$cplane_compile_env_address}:{$sdk_path}/
# After the sending is successful, log in to the OVS control plane compilation environment and run the following commands:
cd {$sdk_path}
unzip flexda_sdk.zip
cd flexda_sdk/example/geneve_single/

Go to the OVS control plane project directory and perform the compilation and building.

cd src_control_plane
# Ensure that the permissions for the directories and files generated by cmake/make are not too high.
umask 077
# Generate the build file.
mkdir build; cd build; cmake ..; 
# Run the make command to compile the dynamic library.
make

The compilation is successful if the following information is displayed:

[100%] Built target custom_openvswitch_plugin

Install the dynamic link library generated after compilation in the operating environment.

scp custom_openvswitch_plugin.so {$username}@{$run_env_address}:{$path}/
# Log in to the operating environment and install the dynamic link library.
chmod 550 custom_openvswitch_plugin.so
cp {$path}/custom_openvswitch_plugin.so /usr/lib64

Parent topic: Control Plane Development Reference