ml
Weight Initialization
Xavier Uniform/Normal, He Uniform/Normal, LeCun Normal, Random, Zeros, Ones — histogram of 20 samples vs theoretical PDF, per-activation recommendations.
Configuration
Formula
N(0, σ²), σ = √(2 / fan_in)
Computed: σ = 0.125
Sample mean
0.026718
theory: 0
Sample variance
0.016273
theory: 0.015625
| w1 | w2 | w3 | w4 | w5 | w6 | w7 | w8 | w9 | w10 |
|---|---|---|---|---|---|---|---|---|---|
| 0.0111 | 0.2047 | -0.0959 | 0.1230 | 0.0446 | -0.1909 | 0.0345 | 0.0760 | -0.0673 | 0.1507 |
| 0.0846 | 0.1943 | -0.2180 | -0.0703 | -0.0131 | 0.1994 | 0.0077 | -0.1532 | 0.1287 | 0.0836 |
Weight Distribution
Sample histogramTheoretical distribution
Method Comparison
| Method | Parameter | Notes |
|---|---|---|
| Xavier Uniform | limit = 0.1768 | — |
| Xavier Normal | σ = 0.1021 | — |
| He Uniform | limit = 0.2165 | ✓ recommended |
| He Normal | σ = 0.125 | ✓ recommended |
| LeCun Normal | σ = 0.08839 | — |
| Random Normal | σ = 0.01 | — |
| Random Uniform | limit = 0.05 | — |
| Zeros | — | ⚠ symmetry |
| Ones | — | ⚠ symmetry |
Tip: Click any row to switch method. Recommended methods are highlighted for the selected activation. He Normal is generally preferred for ReLU networks; Xavier for saturating activations.