Abstract.
We present an efficient learning algorithm for the problem of training neural networks with discrete synapses, a well-known hard (NP-complete) discrete optimization problem. The algorithm is a variant of the so-called Max-Sum (MS) algorithm. In particular, we show how, for bounded integer weights with q distinct states and independent concave a priori distribution (e.g. l1 regularization), the algorithm’s time complexity can be made to scale as 
per node update, thus putting it on par with alternative schemes, such as Belief Propagation (BP), without resorting to approximations. Two special cases are of particular interest: binary synapses 
and ternary synapses 
with l0 regularization. The algorithm we present performs as well as BP on binary perceptron learning problems, and may be better suited to address the problem on fully-connected two-layer networks, since inherent symmetries in two layer networks are naturally broken using the MS approach.
Access this article
The computer you are using is not registered by an institution with a subscription to this article. Please choose one of the options below.
Login
Purchase from
Purchase this article from our trusted document delivery partners.
Make a recommendation
To gain access to this content, please complete the Recommendation Form and we will follow up with your librarian or Institution on your behalf.
For corporate researchers we can also follow up directly with your R&D manager, or the information management contact at your company. Institutional subscribers have access to the current volume, plus a 10-year back file (where available).