Michael Biehl et al 1996 J. Phys. A: Math. Gen. 29 4769 doi:10.1088/0305-4470/29/16/005
Michael Biehl, Peter Riegler and Christian Wöhler
Show affiliationsThe dynamics of on-line learning in neural networks with continuous units is dominated by plateaux in the time dependence of the generalization error. Using tools from statistical mechanics, we show for a soft committee machine the existence of several fixed points of the dynamics of learning that give rise to complicated behaviour, such as cascade-like runs through different plateaux with a decreasing value of the corresponding generalization error. We find learning-rate-dependent phenomena, such as splitting and disappearing of fixed points of the equations of motion. The dependence of plateau lengths on the initial conditions is described analytically and simulations confirm the results.
07.05.Mh Neural networks, fuzzy logic, artificial intelligence
02.30.-f Function theory, analysis
05.10.-a Computational methods in statistical physics and nonlinear dynamics
62M45 Neural nets and related approaches
Issue 16 (21 August 1996)
Received 1 March 1996
Michael Biehl et al 1996 J. Phys. A: Math. Gen. 29 4769
Xiao Xue et al 2007 Chinese Phys. 16 3655
J. P. Phillips and L. Cuesta 2000 The Astronomical Journal 119 335
E Harikumar et al 2000 J. Phys. A: Math. Gen. 33 7149
S. G. Djorgovski et al 2003 ApJ 591 L13
Keh Ying Lin and Jing Xian Huang 1995 J. Phys. A: Math. Gen. 28 3641
P Gaspard 1989 J. Phys. A: Math. Gen. 22 4589
P Deuar and P D Drummond 2006 J. Phys. A: Math. Gen. 39 2723
Aranya B Bhattacherjee et al 2004 J. Phys. B: At. Mol. Opt. Phys. 37 2355
Dominic G B Edelen 2003 Class. Quantum Grav. 20 3661