Norms

Collection of Ivy normalization classes.

class ivy.stateful.norms.LayerNorm(normalized_shape, epsilon=None, elementwise_affine=True, new_std=None, device=None, v=None, dtype=None)[source]

Bases: Module

__init__(normalized_shape, epsilon=None, elementwise_affine=True, new_std=None, device=None, v=None, dtype=None)[source]

Class for applying Layer Normalization over a mini-batch of inputs

Parameters
  • normalized_shape – Trailing shape to applying the normalization to.

  • epsilon – small constant to add to the denominator, use global ivy._MIN_BASE by default.

  • elementwise_affine – Whether to include learnable affine parameters, default is True.

  • new_std – The standard deviation of the new normalized values. Default is 1.

  • device – device on which to create the layer’s variables ‘cuda:0’, ‘cuda:1’, ‘cpu’ etc. (Default value = None)

  • v – the variables for each submodule in the sequence, constructed internally by default.