analogvnn.nn.activation.ELU

Module Contents

Classes

SELU	Implements the scaled exponential linear unit (SELU) activation function.
ELU	Implements the exponential linear unit (ELU) activation function.

class analogvnn.nn.activation.ELU.SELU(alpha: float = 1.0507, scale_factor: float = 1.0)

Bases: analogvnn.nn.activation.Activation.Activation

Implements the scaled exponential linear unit (SELU) activation function.

Variables:

• alpha (nn.Parameter) – the alpha parameter.

• scale_factor (nn.Parameter) – the scale factor parameter.

__constants__ = ['alpha', 'scale_factor']

alpha: torch.nn.Parameter

scale_factor: torch.nn.Parameter

forward(x: torch.Tensor) → torch.Tensor

Forward pass of the scaled exponential linear unit (SELU) activation function.

Parameters:

x (Tensor) – the input tensor.

Returns:

the output tensor.

Return type:

Tensor

backward(grad_output: Optional[torch.Tensor]) → Optional[torch.Tensor]

Backward pass of the scaled exponential linear unit (SELU) activation function.

Parameters:

grad_output (Optional[Tensor]) – the gradient of the output tensor.

Returns:

the gradient of the input tensor.

Return type:

Optional[Tensor]

static initialise(tensor: torch.Tensor) → torch.Tensor

Initialisation of tensor using xavier uniform, gain associated with SELU activation function.

Parameters:

tensor (Tensor) – the tensor to be initialized.

Returns:

the initialized tensor.

Return type:

Tensor

static initialise_(tensor: torch.Tensor) → torch.Tensor

In-place initialisation of tensor using xavier uniform, gain associated with SELU activation function.

Parameters:

tensor (Tensor) – the tensor to be initialized.

Returns:

the initialized tensor.

Return type:

Tensor

class analogvnn.nn.activation.ELU.ELU(alpha: float = 1.0507)

Bases: SELU

Implements the exponential linear unit (ELU) activation function.

Variables:

• alpha (nn.Parameter) – 1.0507

• scale_factor (nn.Parameter) –