Norms

Collection of Ivy normalization classes.

class ivy.stateful.norms.LayerNorm(normalized_shape, /, *, epsilon=1e-05, elementwise_affine=True, new_std=1.0, device=None, v=None, dtype=None)[source]

Bases: Module

__init__(normalized_shape, /, *, epsilon=1e-05, elementwise_affine=True, new_std=1.0, device=None, v=None, dtype=None)[source]

Class for applying Layer Normalization over a mini-batch of inputs

Parameters
  • normalized_shape – Trailing shape to applying the normalization to.

  • epsilon (float) – small constant to add to the denominator, (default: 1e-05) use global ivy._MIN_BASE by default.

  • elementwise_affine (bool) – Whether to include learnable affine parameters, default is True. (default: True)

  • new_std (float) – The standard deviation of the new normalized values. Default is 1. (default: 1.0)

  • device – device on which to create the layer’s variables ‘cuda:0’, ‘cuda:1’, ‘cpu’ etc. (Default value = None)

  • v – the variables for each submodule in the sequence, constructed internally by default.

This should have hopefully given you an overview of the norms submodule,If you have any questions, please feel free to reach out on our discord in the norms channel or in the norms forum!