ml
Weight Initialization
Xavier Uniform/Normal, He Uniform/Normal, LeCun Normal, Random, Zeros, Ones — histogram of 20 samples vs theoretical PDF, per-activation recommendations.
Configuration
Formula
N(0, σ²), σ = √(2 / fan_in)
Computed: σ = 0.125
Sample mean
-0.014214
theory: 0
Sample variance
0.023007
theory: 0.015625
| w1 | w2 | w3 | w4 | w5 | w6 | w7 | w8 | w9 | w10 |
|---|---|---|---|---|---|---|---|---|---|
| -0.1463 | 0.0454 | -0.1658 | -0.0933 | -0.0096 | 0.1359 | -0.1621 | 0.1195 | 0.2104 | -0.0974 |
| 0.2149 | -0.0797 | 0.2283 | 0.1310 | -0.0916 | -0.1491 | 0.0276 | -0.2642 | 0.0650 | -0.2036 |
Weight Distribution
Sample histogramTheoretical distribution
Method Comparison
| Method | Parameter | Notes |
|---|---|---|
| Xavier Uniform | limit = 0.1768 | — |
| Xavier Normal | σ = 0.1021 | — |
| He Uniform | limit = 0.2165 | ✓ recommended |
| He Normal | σ = 0.125 | ✓ recommended |
| LeCun Normal | σ = 0.08839 | — |
| Random Normal | σ = 0.01 | — |
| Random Uniform | limit = 0.05 | — |
| Zeros | — | ⚠ symmetry |
| Ones | — | ⚠ symmetry |
Tip: Click any row to switch method. Recommended methods are highlighted for the selected activation. He Normal is generally preferred for ReLU networks; Xavier for saturating activations.