Instructions

Interface Definition

Initialize EmbeddingLayerFWD.

EmbeddingLayerFWD(const int num_embeddings, const int embedding_dim, __fp16 *_weight,const int padding_idx = -1, const float max_norm = 1.0, const float norm_type = 2.0) ->void

**Table 1** Input parameters of EmbeddingLayerFWD
Parameter	Data Type	Description	Value Range
num_embeddings	int	Number of embedding vectors at the embedding layer.	[1, INT_MAX]
embedding_dim	int	Dimension of each embedding vector.	[1, INT_MAX]
_weight	_fp16*	Weight.	Non-empty in the inference phase
padding_idx	int	Marker for padding.	[INT_MIN, num_embeddings)
max_norm	float	Maximum norm of the weight vector.	Not involved in the inference phase
norm_type	float	Norm type of the weight vector.	Not involved in the inference phase

Run the operator.

Run(int *indice, int dims) ->void

**Table 2** Input parameters of Run
Parameter	Data Type	Description	Value Range
indice	int*	Array of vector indexes to be extracted.	-
dims	int	Number of dimensions of the destination vector.	[1, INT_MAX]

ValidateInput is used to validate the EmbeddingLayerFWD input parameters and is automatically triggered during operator construction.

ValidateInput(const int num_embeddings, const int embedding_dim, __fp16 *_weight, const int padding_idx, const float max_norm, const float norm_type) ->KuDNN::Status

**Table 3** Input parameters of ValidateInput
Parameter	Data Type	Description	Value Range
num_embeddings	int	Number of embedding vectors at the embedding layer.	[1, INT_MAX]
embedding_dim	int	Dimension of each embedding vector.	[1, INT_MAX]
_weight	_fp16*	Weight.	Non-empty in the inference phase
padding_idx	int	Marker for padding.	[INT_MIN, num_embeddings)
max_norm	float	Maximum norm of the weight vector.	Not involved in the inference phase
norm_type	float	Norm type of the weight vector.	Not involved in the inference phase

Supported Data Types

Embedding supports the FP16 data type. Specifically, the data type for the g_weight array and _weight supports FP16.

Examples

Extract the vectors from the g_weight array that correspond to each index in the indice array, and embed them into a fixed-size result vector.

__fp16 g_weight[10][3] = {
{0.1, 0.12, 0.15}, {0.21, 0.22, 0.25}, {0.11, 0.21, 0.32}, {0.77, 0.02, 0.05}, {0.54, 0.32, 0.08},
{0.14, 0.42, 0.25}, {0.31, 0.2, 0.07}, {0.22, 0.14, 0.03}, {0.17, 0.28, 0.32}, {0.44, 0.26, 0.23}
}; // Initialize the vector array using the FP16 type.
int num_embeddings = 10;
int embedding_dim = 3;
int num = 4;
// Construct the operator: KuDNN::EmbeddingLayerFWD embedding(num_embeddings, embedding_dim, (__fp16*)g_weight);
int indice[4] = {1, 2, 3, 7}; // Index array
// Run the operator.
__fp16 *out = embedding.Run(indice, num);

Parent topic: Embedding