Function Description

Operator Function

Performs layer normalization.

Formula

The formula of the layer normalization operator in the case of three dimensions is as follows:

$\text{[math]}$

The mean deviation and variance can be calculated at run time or provided by the user. To calculate the two values at run time, use the following formulas:

$\text{[math]}$

Table 1 describes the parameters in the formulas.

**Table 1** Formula parameters
Parameter	Description
$\text{[math]}$	Scale
$\text{[math]}$	Offset
$\text{[math]}$	Mean deviation
$\text{[math]}$	Variance
$\text{[math]}$	Constant used to improve numerical stability

Parent topic: Layer Normalization