Function Description
Function
Linear+ReLU, Linear+SiLU, Linear+GeLU: Perform ReLU, SiLU, and GeLU operations after the Linear operator computation.
Formula
Computation Type |
forward |
|---|---|
relu |
|
swish |
|
gelu_erf |
|
gelu_tanh |
|
Parent topic: Linear + postops (ReLU, SiLU, GeLU)