Function Description

Function

SoftMax is a common activation function used in multi-classification problems. It converts a set of arbitrary real numbers into a probability distribution whose output values range from 0 to 1, and the sum of all output values is 1.

The main features are as follows:

• Normalized output: The SoftMax function normalizes the input to ensure that the output is a valid probability distribution. Even if the input is any number, the output sum of the SoftMax function is still 1. It is commonly used at the output layer of multi-classification problems.
• Non-linear: The SoftMax function is a non-linear function. It can perform non-linear transformation on the input to increase the representation capability of the model, thereby better fitting complex data patterns.
• Translation invariance: The SoftMax function is commonly used at the output layer to convert the original output of a neural network into a vector representing class probabilities. During training, the difference between the SoftMax output and the actual label can be used as a loss function. Through backward propagation, network parameters are updated to minimize the loss and improve model performance.

Formula

Where,

• is the axis along which the operation is performed.
• represents the outermost index (on the left side of the axis).
• represents the innermost index (on the right side of the axis).
• is used to generate a numerically stable result, which is defined as .