Instructions

Interface Definition

Initialize Linear+ReLU, Linear+SiLU, and Linear+GELU. During the construction, the tensor information of the input matrix, weight matrix, and output matrix needs to be passed. The bias matrix is optional. If it is not passed, the second constructor will be called for initialization.

• LinearActivationLayerFWD(const TensorInfo &aInfo, const TensorInfo &bInfo, const TensorInfo &cInfo, const TensorInfo &biasInfo, float alpha = 1.0f, float beta = 0.0f, ActivationFunction kind = KuDNN::ActivationFunction::RELU, int numthreads = 0)->void
• LinearActivationLayerFWD(const TensorInfo &aInfo, const TensorInfo &bInfo, const TensorInfo &cInfo, float alpha = 1.0f, float beta = 0.0f, ActivationFunction kind = KuDNN::ActivationFunction::RELU, int numthreads = 0)->void

To perform operator computation, the memory addresses for storing the input and output must be passed. a, b, c, and bias are the memory addresses of the input matrix, weight matrix, output matrix, and bias matrix, respectively. Whether to pass the address of the bias matrix depends on whether the bias is passed during construction.

• void Run(const void *a, const void *b, void *c, const void *bias, int numThreads = 0)->void
• Run(const void *a, const void *b, void *c, int numThreads = 0)->void

ValidateInput verifies the input parameters of Linear+ReLU, Linear+SiLU, and Linear+GeLU, and is automatically triggered during operator construction.

• ValidateInput(const TensorInfo &srcInfo, const TensorInfo &weiInfo, const TensorInfo &dstInfo, const TensorInfo &biasInfo, float alpha = 1.0f, float beta =0.0f, ActivationFunctionkind=KuDNN::ActivationFunction::RELU, int numthreads = 0)-> KuDNN::Status
• ValidateInput(const TensorInfo &srcInfo, const TensorInfo &weiInfo, const TensorInfo &dstInfo, float alpha = 1.0f, float beta =0.0f,ActivationFunctionkind=KuDNN::ActivationFunction::RELU, int numthreads = 0)-> KuDNN::Status

Supported Data Types

• postops supports the FP16, FP32, and BF16 data types. (The Shape, Type, and Layout parameters must be passed during TensorInfo object initialization. The following lists the data types supported by Type.)

  Table 4 Type supported during TensorInfo object initialization

  srcInfo	weiInfo	dstInfo	biasInfo
  KuDNN::Element::TypeT::F16(fp16)	KuDNN::Element::TypeT::F16(fp16)	KuDNN::Element::TypeT::F16(fp16)	KuDNN::Element::TypeT::F16(fp16)
  KuDNN::Element::TypeT::F32(fp32)	KuDNN::Element::TypeT::F32(fp32)	KuDNN::Element::TypeT::F32(fp32)	KuDNN::Element::TypeT::F32(fp32)
  KuDNN::Element::TypeT::BF16(bf16)	KuDNN::Element::TypeT::BF16(bf16)	KuDNN::Element::TypeT::BF16(bf16)	KuDNN::Element::TypeT::BF16(bf16)

• A maximum of 5 dimensions are supported. The supported sequential data layouts include a, ab, abc, abcd, and abcde.

  They correspond to KuDNN::Layout::A, KuDNN::Layout::AB, KuDNN::Layout::ABC, KuDNN::Layout::ABCD, and KuDNN::Layout::ABCDE, respectively.

  Table 5 Layout supported during TensorInfo object initialization

  Dimension	srcInfo	weiInfo	dstInfo	biasInfo
  2D	ab/ba	ab/ba	ab/ba	ab/ba
  3D	abc/acb	abc/acb	abc/acb	abc/acb
  4D	abcd/abdc	abcd/abdc	abcd/abdc	abcd/abdc
  5D	abcde/abced	abcde/abced	abcde/abced	abcde/abced

Examples

Linear + ops operation with two-dimensional data of the FP32 type and ops of the RES_IDENTIAL type:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

using SizeType = KuDNN::SizeType;
using Shape = KuDNN::Shape;
using Type KuDNN::Element::TypeT
// Tensor initialization
const KuDNN::TensorInfo srcTensor = {{110, 20}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
const KuDNN::TensorInfo weiTensor = {{20, 200}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
const KuDNN::TensorInfo dstTensor = {{110, 200}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
const KuDNN::TensorInfo biasTensor = {{110, 200}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
float alpha = 0.0f;
float beta = 0.0f;
KuDNN::ActivationFunction algKind = KuDNN::ActivationFunction::RELU;
int numThreads = 0;
SizeType srcSize = srcTensor.GetTotalTensorSize();
SizeType dstSize = weiBTensor.GetTotalTensorSize();
SizeType weiSize = dstCTensor.GetTotalTensorSize();
SizeType biasSize = biasTensor.GetTotalTensorSize(); 
// Allocate memory for storing the input arguments and result.
float * src = malloc(srcSize * sizeof(float));
float * wei = malloc(weiSize * sizeof(float));
float * dst = malloc(dstSize * sizeof(float));
float * bias = malloc(biasSize * sizeof(float));
// Construct the operator KuDNN::LinearActivationLayerFWD.
linearActivationLayerFwd(srcTensor, weiTensor, dstTensor,
biasTensor, alpha, beta, algKind, numThreads);
// Run the operator.
linearActivationLayerFwd.Run(src, wei, dst, bias, numThreads);