torch.nn.functional.glu

torch.nn.functional.glu(input, dim=-1) → Tensor [source]

The gated linear unit. Computes:

$$\text{GLU}(a, b) = a \otimes \sigma(b)$$

where input is split in half along dim to form a and b, $\sigma$ is the sigmoid function and $\otimes$ is the element-wise product between matrices.

See Language Modeling with Gated Convolutional Networks.

Parameters

• input (Tensor) – input tensor
• dim (int) – dimension on which to split the input. Default: -1

Return type

Tensor