Xavier Initialization - Page 3

Xavier initialization is a method for setting the starting weights of a neural network so signal variance remains reasonably stable across layers. It chooses random values scaled according to the number of inputs and outputs connected to a layer. The method, also called Glorot initialization, is commonly associated with sigmoid or tanh activations, while He initialization is often preferred for ReLU networks. Stable starting values help prevent activations and gradients from shrinking or growing too rapidly before training can correct them. Xavier initialization does not guarantee convergence, and its suitability depends on architecture, activation functions, normalization, optimizer settings, and the depth of the network.