Instructions

Interface Definition

Initializes the Linear + resext, Linear + resmul, and Linear + residential operations. During construction, the tensor information of the Linear input matrix, Linear weight matrix, Linear output matrix, Res input matrix, and LinearRes output matrix needs to be passed. The bias matrix is optional. If it is not passed, the second constructor will be called for initialization.

LinearResFWD(const TensorInfo &srcInfo, const TensorInfo &weiInfo, const TensorInfo &dstInfo, const TensorInfo &biasInfo, const TensorInfo &resInfo, const TensorInfo &linearResInfo, float alpha = 1.0f, float beta = 0.0f, float gamma = 1.0f, ResOpsFunction kind = ResOpsFunction::RES_IDENTIAL, int numthreads)->void
LinearResFWD(const TensorInfo &srcInfo, const TensorInfo &weiInfo, const TensorInfo &dstInfo, const TensorInfo &resInfo, const TensorInfo &linearResInfo, float alpha = 1.0f, float beta = 0.0f, float gamma = 1.0f, ResOpsFunction kind = ResOpsFunction::RES_IDENTIAL, int numthreads)->void

**Table 1** Input parameters of LinearResFWD
Parameter	Data Type	Description	Value Range
srcInfo	KuDNN::TensorInfo	Source tensor information.	{shape, type, layout}
weiInfo	KuDNN::TensorInfo	Weight tensor information.	{shape, type, layout}
dstInfo	KuDNN::TensorInfo	Destination tensor information.	{shape, type, layout}
biasInfo	KuDNN::TensorInfo	Bias tensor information.	{shape, type, layout}
resInfo	KuDNN::TensorInfo	Res tensor information.	{shape, type, layout}
linearResInfo	KuDNN::TensorInfo	Linear + Res output tensor information.	{shape, type, layout}
alpha	float	Linear parameter.	Default value: 1.0
beta	float	Linear parameter.	Default value: 0.0
gamma	float	Res operation parameter.	Default value: 1.0 (when kind is set to RES_EXT, and gamma is multiplied by the res matrix. In all other cases, this parameter has no effect.)
kind	enum class	Activation function type.	Default value: KuDNN::ResOpsFunction::RES_IDENTIAL Other values include RES_EXT and RES_MU.
numthreads	int	Number of threads	Default value: 0

To perform operator computation, the memory addresses for storing the input and output must be passed. a, b, c, and bias are the memory addresses of the input matrix, weight matrix, output matrix, and bias matrix, respectively. Whether to pass the address of the bias matrix depends on whether the bias is passed during construction.

Run(const void *a, const void *b, void *c, const void *bias, const void *res, void *linearRes, float gamma = 1.0f, int numThreads = 0)->void
Run(const void *a, const void *b, void *c, const void *res, void *linearRes, float gamma = 1.0f, int numThreads = 0)->void

**Table 2** Input parameters of Run
Parameter	Data Type	Description	Value Range
a	void *	Input pointer	Pointer with a size of MxKxtype.GetSize()
b	void *	Weight pointer	Pointer with a size of KxNxtype.GetSize()
c	void *	Output pointer	Pointer with a size of MxNxtype.GetSize()
bias	void *	Bias pointer	Pointer with a size of MxNxtype.GetSize()
res	void *	Input pointer of the ops operation	If the ops operation is multiplication, res is a pointer with a size of NxOxtype.GetSize(). If the ops operation is addition, res is a pointer with a size of MxNxtype.GetSize().
linearRes	void *	Pointer to the result of the linear+res operation	If the ops operation is multiplication, res is a pointer with a size of MxOxtype.GetSize(). If the ops operation is addition, res is a pointer with a size of MxNxtype.GetSize().
(Optional) numThreads	int	Number of threads	By default, if 0 is passed, the maximum number of threads returned by GetMaxNumThreads is used.

ValidateInput is used to validate the input parameters of Linear + resext, Linear + resmul, and Linear + residential. It also has two versions: with bias and without bias. It is automatically triggered during operator construction.

ValidateInput(const TensorInfo &srcInfo, const TensorInfo &weiInfo, const TensorInfo &dstInfo, const TensorInfo &biasInfo, const TensorInfo &resInfo, const TensorInfo &linearResInfo, float alpha = 1.0f, float beta = 0.0f, float gamma = 1.0f, ResOpsFunction kind = ResOpsFunction::RES_IDENTIAL, int numthreads = 0)->KuDNN::Status
ValidateInput(const TensorInfo &srcInfo, const TensorInfo &weiInfo, const TensorInfo &dstInfo, const TensorInfo &resInfo, const TensorInfo &linearResInfo, float alpha = 1.0f, float beta = 0.0f, float gamma = 1.0f, ResOpsFunction kind = ResOpsFunction::RES_IDENTIAL, int numthreads = 0)->KuDNN::Status

**Table 3** Input parameters of ValidateInput
Parameter	Data Type	Description	Value Range
srcInfo	KuDNN::TensorInfo	Source tensor information.	{shape, type, layout}
weiInfo	KuDNN::TensorInfo	Weight tensor information.	{shape, type, layout}
dstInfo	KuDNN::TensorInfo	Destination tensor information.	{shape, type, layout}
biasInfo	KuDNN::TensorInfo	Bias tensor information.	{shape, type, layout}
resInfo	KuDNN::TensorInfo	Res tensor information.	{shape, type, layout}
linearResInfo	KuDNN::TensorInfo	Linear + Res output tensor information.	{shape, type, layout}
alpha	float	Linear parameter.	Default value: 1.0
beta	float	Linear parameter.	Default value: 0.0
gamma	float	Res operation parameter.	Default value: 1.0 (when kind is set to RES_EXT, and gamma is multiplied by the res matrix. In all other cases, this parameter has no effect.)
kind	enum class	Activation function type.	Default value: KuDNN::ResOpsFunction::RES_IDENTIAL Other values include RES_EXT and RES_MU.
numthreads	int	Number of threads	Default value: 0

Supported Data Types

postops supports the FP16, FP32, and BF16 data types. (The Shape, Type, and Layout parameters must be passed during TensorInfo object initialization. The following lists the data types supported by Type.)

**Table 4** Type supported during TensorInfo object initialization
srcInfo	weiInfo	dstInfo	biasInfo	resInfo	linearResInfo
KuDNN::Element::TypeT::F16(fp16)	KuDNN::Element::TypeT::F16(fp16)	KuDNN::Element::TypeT::F16(fp16)	KuDNN::Element::TypeT::F16(fp16)	KuDNN::Element::TypeT::F16(fp16)	KuDNN::Element::TypeT::F16(fp16)
KuDNN::Element::TypeT::F32(fp32)	KuDNN::Element::TypeT::F32(fp32)	KuDNN::Element::TypeT::F32(fp32)	KuDNN::Element::TypeT::F32(fp32)	KuDNN::Element::TypeT::F32(fp32)	KuDNN::Element::TypeT::F32(fp32)
KuDNN::Element::TypeT::BF16(bf16)	KuDNN::Element::TypeT::BF16(bf16)	KuDNN::Element::TypeT::BF16(bf16)	KuDNN::Element::TypeT::BF16(bf16)	KuDNN::Element::TypeT::BF16(bf16)	KuDNN::Element::TypeT::BF16(bf16)

A maximum of 5 dimensions are supported. The supported sequential data layouts include a, ab, abc, abcd, and abcde.

They correspond to KuDNN::Layout::A, KuDNN::Layout::AB, KuDNN::Layout::ABC, KuDNN::Layout::ABCD, and KuDNN::Layout::ABCDE, respectively.

**Table 5** Layout supported during TensorInfo object initialization
Dimension	srcInfo	weiInfo	dstInfo	biasInfo
2D	ab/ba	ab/ba	ab/ba	ab/ba
3D	abc/acb	abc/acb	abc/acb	abc/acb
4D	abcd/abdc	abcd/abdc	abcd/abdc	abcd/abdc
5D	abcde/abced	abcde/abced	abcde/abced	abcde/abced

Examples

Linear + ops operation with two-dimensional data of the FP32 type and ops of the RES_IDENTIAL type:

// Example
// using SizeType = KuDNN::SizeType;
using Shape = KuDNN::Shape;
using Type KuDNN::Element::TypeT
// Tensor initialization
const KuDNN::TensorInfo srcTensor= {{3, 2}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
const KuDNN::TensorInfo weiTensor= {{2, 2}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
const KuDNN::TensorInfo dstTensor= {{3, 2}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
const KuDNN::TensorInfo biasTensor = {{3, 2}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
const KuDNN::TensorInfo resTensor = {{3, 2}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
const KuDNN::TensorInfo linearResTensor = {{3, 2}, KuDNN::Element::TypeT::F32, KuDNN::Layout::AB};
float alpha = 1.0f;
float beta = 0.0f;
float gamma = 1.0f;KuDNN::ResOpsFunction algKind = KuDNN::ResOpsFunction::RES_IDENTIAL;
int numThreads = 0;
SizeType srcSize = srcATensor.GetTotalTensorSize();
SizeType dstSize = weiBTensor.GetTotalTensorSize();
SizeType weiSize = dstCTensor.GetTotalTensorSize();
SizeType biasSize = biasTensor.GetTotalTensorSize();
SizeType resSize = resTensor.GetTotalTensorSize();
SizeType linearResSize = linearResTensor.GetTotalTensorSize();
// Allocate memory for storing the input parameters and result.
float * src = malloc(srcSize* sizeof(float));
float * wei = malloc(weiSize* sizeof(float));
float * dst = malloc(dstSize* sizeof(float));
float * bias = malloc(biasSize* sizeof(float));
float * res = malloc(resSize* sizeof(float));
float * linearRes = malloc(linearResSize* sizeof(float));
// Construct the operator.
KuDNN::LinearResFWD
linearResFwd(srcTensor, weiTensor, dstTensor,
biasTensor, resTensor, linearResTensor, alpha, beta, gamma, algKind, numThreads);
// Run the operator.
linearResFwd.Run(src, wei, dst, bias, res, linearRes, gamma, numThreads);

Parent topic: Linear + postops (resext, resmul, residential)