ml
Weight Initialization
Xavier Uniform/Normal, He Uniform/Normal, LeCun Normal, Random, Zeros, Ones — histogram of 20 samples vs theoretical PDF, per-activation recommendations.
Configuration
Formula
N(0, σ²), σ = √(2 / fan_in)
Computed: σ = 0.125
Sample mean
0.040579
theory: 0
Sample variance
0.022222
theory: 0.015625
| w1 | w2 | w3 | w4 | w5 | w6 | w7 | w8 | w9 | w10 |
|---|---|---|---|---|---|---|---|---|---|
| 0.0687 | 0.2136 | 0.0667 | 0.0595 | -0.0847 | 0.1115 | 0.0456 | -0.1626 | 0.2906 | 0.2388 |
| -0.0019 | 0.0524 | -0.0968 | 0.1226 | 0.2439 | 0.0992 | 0.0073 | -0.0740 | -0.0726 | -0.3161 |
Weight Distribution
Sample histogramTheoretical distribution
Method Comparison
| Method | Parameter | Notes |
|---|---|---|
| Xavier Uniform | limit = 0.1768 | — |
| Xavier Normal | σ = 0.1021 | — |
| He Uniform | limit = 0.2165 | ✓ recommended |
| He Normal | σ = 0.125 | ✓ recommended |
| LeCun Normal | σ = 0.08839 | — |
| Random Normal | σ = 0.01 | — |
| Random Uniform | limit = 0.05 | — |
| Zeros | — | ⚠ symmetry |
| Ones | — | ⚠ symmetry |
Tip: Click any row to switch method. Recommended methods are highlighted for the selected activation. He Normal is generally preferred for ReLU networks; Xavier for saturating activations.