Rate This Document
Findability
Accuracy
Completeness
Readability

Interface Description

Table 1 lists the adapted APIs supported by KuDNN.

Table 1 Interfaces

Operator

Function

Data Type Supported by KuDNN

Linear

Linear transformation

input weight bias output (fp16 fp16 fp16 fp16)

input weight bias output (fp16 fp16 fp32 fp32)

input weight bias output (int8 int8 int32 int32)

input weight bias output (fp16 int8 int32 fp32)

input weight bias output (fp32 fp32 fp32 fp32)

SoftMax

Probability distribution normalization

fp32

Conv

Local feature extraction

fp16, fp32

GroupNorm

Channel group normalization

fp16, fp32

LayerNorm

Layer group normalization

fp16, fp32

RMSNorm

Root mean square normalization

fp16

Embedding

Maps integer indices to fixed vectors

fp16

Table 2 Data types

Data Type

Torch Data Type

fp16

torch.float16

fp32

torch.float32

int8

torch.int8

int32

torch.int32