tf.nn.silu
Computes the SiLU or Swish activation function: x * sigmoid(beta * x).
tf.nn.silu(
features, beta=1.0
)
beta : Hyperparameter for Swish activation function. Default value 1.0.
The SiLU activation function was introduced in "Gaussian Error Linear Units (GELUs)" Hendrycks et al. 2016 and "Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning" Elfwing et al. 2017 and was independently discovered (and called swish) in "Searching for Activation Functions" Ramachandran et al. 2017
| Args | |
|---|---|
features |
A Tensor representing preactivation values. |
beta |
A 'Tensor' representing value of beta hyperparameter. |
| Returns | |
|---|---|
| The activation value. |
© 2022 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 4.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/versions/r2.9/api_docs/python/tf/nn/silu
