Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts
Generalization
Stanisaw Jastrzebski 1 2 Devansh Arpit 3 Oliver strand 2 Giancarlo Kerg 4 Huan Wang 3 Caiming Xiong 3
Richard Socher 3 Kyunghyun Cho * 2 5 Krzysztof J. Geras * 1 2
Abstract commonly observed that using a moderately large learning
rate in the early phase of training results in better generaliza-
The early phase of tr ...


雷达卡




京公网安备 11010802022788号







