ml
Weight Initialization
Xavier Uniform/Normal, He Uniform/Normal, LeCun Normal, Random, Zeros, Ones — histogram of 20 samples vs theoretical PDF, per-activation recommendations.
Configuration
Formula
N(0, σ²), σ = √(2 / fan_in)
Computed: σ = 0.125
Sample mean
-0.044627
theory: 0
Sample variance
0.016898
theory: 0.015625
| w1 | w2 | w3 | w4 | w5 | w6 | w7 | w8 | w9 | w10 |
|---|---|---|---|---|---|---|---|---|---|
| 0.0778 | -0.1917 | 0.0623 | 0.0081 | -0.0057 | -0.1654 | -0.0380 | 0.0939 | 0.0954 | -0.0406 |
| -0.1559 | -0.0808 | -0.1133 | -0.1838 | 0.1350 | -0.0294 | -0.3253 | 0.0156 | 0.1504 | -0.2012 |
Weight Distribution
Sample histogramTheoretical distribution
Method Comparison
| Method | Parameter | Notes |
|---|---|---|
| Xavier Uniform | limit = 0.1768 | — |
| Xavier Normal | σ = 0.1021 | — |
| He Uniform | limit = 0.2165 | ✓ recommended |
| He Normal | σ = 0.125 | ✓ recommended |
| LeCun Normal | σ = 0.08839 | — |
| Random Normal | σ = 0.01 | — |
| Random Uniform | limit = 0.05 | — |
| Zeros | — | ⚠ symmetry |
| Ones | — | ⚠ symmetry |
Tip: Click any row to switch method. Recommended methods are highlighted for the selected activation. He Normal is generally preferred for ReLU networks; Xavier for saturating activations.