Abstract
The relationship between the geometrical structure of weight space and replica symmetry breaking (RSB) in multilayer neural networks is studied using a toy model. The distribution of sizes of the disconnected domains of solution space is computed analytically and compared to the RSB calculation of the Gardner volume. We are able to show explicity that ergodicity breaking and RSB are not equivalent. Repeating these calculations using the cavity approach allows us to interpret the geometrical meaning of a RSB ansatzd.