On the atypical solutions of the symmetric binary perceptron

We consider the symmetric binary perceptron (SBP), introduced in [4], where we let ${\boldsymbol{G}} = (\boldsymbol{g}_a)_{a = 1}^M$ be a collection of M i.i.d. standard Gaussian random vectors in $\mathbb{R}^N$, with $M = \lfloor\alpha N \rfloor$ for a fixed $\alpha>0$ and for $\kappa>0$, we consider the set of binary solutions ${\boldsymbol{x}} \in \{-1,+1\}^N$ to the system of linear inequalities

$$\begin{align} \big| \langle {\boldsymbol{g}}_a , {\boldsymbol{x}} \rangle \big| \unicode{x2A7D} \kappa \sqrt{N} ~~~~~ \mbox{for all}~ 1 \unicode{x2A7D} a \unicode{x2A7D} M\, . \end{align} \tag{ 1 }$$

We denote the set of solutions by $S({\boldsymbol{G}}, \kappa)$, and its cardinality by

$$\begin{align} Z\left({\boldsymbol{G}},\kappa\right) = \big|S\left({\boldsymbol{G}}, \kappa\right)\big|\, . \end{align} \tag{ 2 }$$

It was shown by Aubin et al [4] that $S({\boldsymbol{G}},\kappa)$ is nonempty with high probability if and only if $\kappa \gt \kappa_{\tiny \mathrm{SAT}}(\alpha)$ where $\kappa_{\tiny \mathrm{SAT}}(\alpha)$ is defined by the equation

$$\begin{align} \operatorname{\mathbb{P}}\left(|Z| \unicode{x2A7D} \kappa\right)^{\alpha} = 1/2\, ,~~~~ Z \sim N\left(0,1\right)\, . \end{align} \tag{ 3 }$$

Moreover, in the limit of small $\alpha$ we have

$$\begin{align} \kappa_{\tiny \mathrm{SAT}}\left(\alpha\right) \underset{\alpha\rightarrow0}{\sim}\sqrt{\frac{\pi}{2}}\, 2^{-1/\alpha}\, . \end{align} \tag{ 4 }$$

Our main interest is in investigating the possibility of finding solutions efficiently when $\kappa \gt \kappa_{\tiny \mathrm{SAT}}(\alpha)$.

Mézard and Krauth [25] showed in their seminal work using the non-rigorous replica method [31] that the solution landscape of the one-sided perceptron (where there is no absolute value in the constraints (1)) is dominated by isolated solutions lying at large mutual Hamming distances, a structure sometimes called 'frozen replica symmetry breaking' [17, 22, 23, 29, 40]. From the mathematics point of view, the frozen replica symmetry breaking prediction was proven true for the SBP in works by Perkins and Xu [34] and Abbé et al [1], who showed that for all $\kappa \gt \kappa_{\tiny \mathrm{SAT}}(\alpha)$, a solution drawn uniformly at random from $S(\boldsymbol{G},\kappa)$ is isolated with high probability, in the sense that it is separated from any other solution by a Hamming distance linear in N.

This type of landscape property has been traditionally associated with algorithmic hardness, with the rationale that an algorithm performing local moves is unlikely to succeed in the face of such extreme clustering, as argued, for instance, by Zdeborová and Mézard [39], or Huang and Kabashima [22]. In some problems, this predicted algorithmic hardness was confirmed empirically, e.g. [39, 40]. In other problems, a prominent example being the binary perceptron (symmetric or not), it is known that certain efficient heuristics are able to find solutions for α small enough as a function of $\kappa$ [6, 7, 16, 18, 19, 23, 24]. Statistical physics studies of the neighborhood of the solutions returned by efficient heuristics have put forward the intriguing observation that in the binary perceptron problem, a dense region of other solutions surrounds the ones which are returned [5, 9, 10]. This means that efficient algorithms may be drawn to rare, well connected subset(s) of $S(\boldsymbol{G},\kappa)$. Moreover, these efficient algorithms fail to return a solution when α becomes large, suggesting the existence of a computational phase transition in the binary perceptron (symmetric or not).

For the symmetric version of the problem, this state of affairs has been partially elucidated in two recent mathematical works: in [2], Abbé et al show the existence of clusters of solutions of linear diameter for all $\kappa \gt \kappa_{\tiny \mathrm{SAT}}(\alpha)$, and maximal diameter for $\alpha$ small enough. In a different direction, Gamarnik et al [21] established an almost sharp result in the regime of small $\alpha$, stating the following: there exists constants $c_0, c_1 \gt0$ such that for α small enough,

• if $\kappa \unicode{x2A7E} c_0 \sqrt{\alpha}$ then a certain online algorithm of Bansal and Spencer [12] finds a solution in $S({\boldsymbol{G}},\kappa)$, and
• if $\kappa \unicode{x2A7D} c_1 \sqrt{-\alpha/\log(\alpha)}$ then $S({\boldsymbol{G}},\kappa)$ exhibits a overlap gap property ruling out a wide class of efficient algorithms.

We mention that the positive result which holds for $\kappa \unicode{x2A7E} c_0 \sqrt{\alpha}$ is established in the case where the constraint matrix G is Rademacher instead of Gaussian; nevertheless, the same result is expected in the Gaussian case.

Baldassi et al [8] suggest that this computational transition can be probed by studying the monotonicity properties of the local entropy of solutions around atypical solutions ${\boldsymbol{x}}_0$ as a function of the distance from this solution. One can interpret the results of [8] as evidence towards a conjecture that finding a solution is computationally easy precisely when there exist some rare solutions around which this local entropy is monotone in the distance and that the problem becomes hard when this local entropy develops a local maximum at some distance r₀ from the reference solution ${\boldsymbol{x}}_0$. If such a conjecture is correct, then it must agree with the above-mentioned finding of Gamarnik et al [21] in the regime of small $\alpha$. This question motivated the present work.

Another gap in the physics literature we elucidate in this work relates to the fact that the replica method on the one-step replica symmetry breaking level so far has not managed to find clusters of solutions in the binary perceptron. Indeed, the method can count rare clusters as long as they correspond to fixed points of a corresponding message-passing algorithm, see e.g. [38]. Parallels between the 1RSB calculation and the analysis of solutions with a monotonic local entropy have been put forward in [5, 9, 10], but not in the form where one writes the standard 1RSB equations and shows that they have a solution corresponding to rare subdominant clusters. We show that the standard 1RSB framework actually does present such solutions which describe subdominant clusters of extensive entropy, and we give likely reasons why these solutions were missed in past investigations.

1.2.1. Local entropy around high margin solutions.

We define and study a notion of local entropy around solutions which are typical at some margin $\kappa_0 \lt \kappa$. While typical solutions at $\kappa_0$ are isolated from each other, it was shown in [2] that they belong to connected components of solutions at margin $\kappa$ having a linear diameter in N. Here, we show that these solutions are surrounded by exponentially many solutions at margin $\kappa$.

Consistently with the statistical physics literature, we say that there is a cluster of extensive entropy around a reference solution ${\boldsymbol{x}}_0$ when the local entropy as a function of the distance achieves a local maximum at some distance from ${\boldsymbol{x}}_0$. We show that for a certain range of $\kappa$ typical solutions at margin $\kappa_0$ have extensive entropy clusters around them. We define the entropy of these clusters as the value of the entropy at a local maximum. An analogous investigation of local entropy around large margin solutions was performed in [11] for the one-sided binary perceptron using the replica method.

In our case, the symmetry of the constraints (1) allows us to derive simpler formulas for the local entropy in the regime of small $\alpha$, essentially via a first moment method. This is due to the present model being contiguous to a corresponding simpler planted model in which the first and second moment computations can be conducted. We show that under a certain assumption on the concentration of the entropy of the SBP, while for any constant value of $\alpha$ the second moment is exponentially larger than the square of the first moment, the exponent of the ratio of these quantities, when normalized by N, tends to zero in the limit of small $\alpha$.

The resulting entropy of these clusters is plotted in figure 1 for various values of $\kappa$ and $\kappa_0$ in the $\alpha$ → 0 limit. We observe that at a certain margin $\kappa_\textrm{entr}(\kappa_0)$ the entropy curve stops because the local entropy curve becomes monotone in the distance for $\kappa \gt \kappa_\textrm{entr}(\kappa_0)$. As discussed above, the existence of reference solutions such that the local entropy curve is monotone was speculated to provoke the onset of a region of parameters where finding solutions is algorithmically easy. In this paper, we show the existence of solutions–those typical at $\kappa_0$–for which the local entropy is monotone, and hence we do not expect the problem to be computationally hard for $\kappa \gt \kappa_\textrm{entr}(\kappa_0)$. In figure 1 we see that the smallest $\kappa$ where this happens is $\kappa_\textrm{entr} \equiv \textrm{min}_{\kappa_0} \kappa_\textrm{entr}(\kappa_0) = \kappa_\textrm{entr}(\kappa_0 = \kappa_{\tiny \mathrm{SAT}})$. For this reason, a large part of this investigation is devoted to the case $\kappa_0 = \kappa_{\tiny \mathrm{SAT}}(\alpha)$.

Zoom In Zoom Out Reset image size

Motivated by these findings, we then study the local entropy of solutions that are at a Hamming distance Nr from the solution planted at $\kappa_0 = \kappa_{\tiny \mathrm{SAT}}(\alpha)$. This is akin to the Franz–Parisi potential as studied in the physics of spin glasses [20]. Here, we compute this potential around a typical solution at $\kappa_0$. Our findings, again in the regime of small $\alpha$, are summarized in figure 2(left), where it is apparent that the local entropy as a function of the distance r from a reference solution is monotone when $\kappa \unicode{x2A7E} \tilde{\kappa}_\textrm{entr} \sqrt{-\alpha/\log(\alpha)}$ and has a local maximum at an intermediate distance r₀ when $\kappa \lt \tilde{\kappa}_\textrm{entr} \sqrt{-\alpha/\log(\alpha)}$, with $\tilde{\kappa}_\textrm{entr} \approx 1.429$ given by implicit equations (92) and (93). We also show that no solutions can be found in an interval of distances from the reference solution when $\kappa \lt \tilde{\kappa}_\textrm{ener} \sqrt{-\alpha/\log(\alpha)}$ with $\tilde{\kappa}_\textrm{ener} \approx 1.239$ given by the implicit equations (87) and (88).

Zoom In Zoom Out Reset image size

From these results, we note the existence of a logarithmic gap in $1/\alpha$ in the value of κ where the local entropy curve becomes monotone and the value where the Bansal–Spencer algorithm is proved to succeed, in the regime of small α. It is an interesting open problem to close this gap, either by showing that efficient algorithms can find solutions for all $\kappa \unicode{x2A7E} \tilde{\kappa}_\textrm{entr} \sqrt{-\alpha/\log(\alpha)}$ or by showing the local entropy approach is not indicative of algorithmic hardness.

1.2.2. The 1RSB computation of the complexity curve.

We note that in the statistical physics literature, clusters as defined above are also associated with a fixed point of the approximate message passing (AMP) algorithm or equivalently the Thouless–Anderson–Palmer (TAP) equations. The cluster entropy can be thus computed as the Bethe entropy corresponding to the AMP/TAP fixed point that is reached by AMP run at κ and initialized in one of the typical solutions at margin κ₀. For $\kappa \gt \kappa_\textrm{entr}(\kappa_0)$ the AMP/TAP converges to the same fixed point as would be reached from a random initialization, corresponding to an entropy covering the whole space of solutions. Using this relation, the onset of a region where algorithms may be able to find these solutions is then related to the existence of solutions such that a AMP/TAP iteration initialized at these points converges to the same fixed point as if the iteration was initialized uniformly at random from $S({\boldsymbol{G}},\kappa)$. Indeed, it was observed empirically that solutions found by efficient algorithms always have such a property of AMP/TAP or the belief propagation algorithm converging to the same fixed point as from a random initialization [15, 28].

In the existing statistical physics literature, using the replica method on the one-step replica symmetry breaking level, researchers so far have not found clusters of solutions of extensive entropy in the binary perceptron. This is a point of concern as this method is supposed to count all clusters of solutions corresponding to the TAP/AMP fixed points, including the rare non-equilibrium ones [17, 26, 30, 32, 38]. This a priori casts doubt on the efficacy of the replica method and the validity of its predictions for the number of clusters of a given size, since the method misses a large part of the phase space (unless some explicit conditioning is done as in [9, 10]).

We propose, based on the replica method, that the answer to this question lies in the properties of the complexity (the logarithm of the number of clusters) versus entropy (the logarithm of the number of solutions in the clusters) $\Sigma(s)$. We observe that the numerical value of the complexity is rather large compared to the entropy. The slope of $\Sigma(s)$ gives the value of the so-called Parisi parameter x that is therefore rather large: $x \gg 1$. Since the value of x describing the equilibrium properties of the system is always between 0 and 1 it is not that surprising that the literature has not investigated solutions of the replica equations corresponding to $x \gg 1$. When we consider a large range of values of x in the standard 1RSB equations for SBP [4], we obtain the $\Sigma(s)$ depicted in figure 2 right. We then provide an argument that leads us to conjecture that in the small α limit, the curve $\Sigma(s)$ corresponds to the one we obtain via the approach of planting at κ₀. Thus even though, in general, by planting we construct only some of the rare clusters, it seems that in fact we construct the most frequent ones in the limit of small α.

Another property that we unveil is related to the fact that the curve $\Sigma(s)$ is usually expected to be concave. The non-concave parts were so far considered 'unphysical' in the literature (e.g. figure 8 in [14] or figure 5 in [38]). We show in our present work that the so-called 'unphysical branch' of the replica/cavity prediction is actually not 'unphysical' in the SBP and that it reproduces the curve $\Sigma(s)$ obtained from the local entropy calculation at small α and small internal cluster entropy. Moreover, we show that some of the relevant parts of the curve $\Sigma(s)$ cannot be obtained in the usually iterative way of solving the 1RSB equations at a fixed value of the Parisi parameter x. To access this part of the curve we need to adjust the value of x adaptively in every step when solving the 1RSB fixed point equations iteratively.

The rest of the paper is organized as follows: section 2 defines the local entropy and states the main theorem 1 in the small α limit. Section 3 introduces the planted model and its contiguity to the original model; a key element of the proof. Section 4 contains the moment computations in the planted model, ending with the proof of theorem 1. In section 5 we use the result of theorem 1 and study the properties of the asymptotic formula of the local entropy in the small α limit. In section 6 we study the one-step-replica-symmetry breaking solution of the SBP and its relation to the local entropy. This section investigates general values of α, not only the small α limit. Finally, we conclude in section 7.

Sections 2–4 are fully mathematically rigorous. In section 5 we analyze the resulting local entropy formula heuristically, solving the corresponding fixed point equations numerically, and deriving the numerical values for the energetic and the entropic thresholds. In section 6 we rely on the replica method which is well-accepted and widely used in theoretical statistical physics but not rigorously justified from the mathematical standpoint.

The energetic threshold occurs when in a range of intermediate distances the local entropy $\phi_1(r)$ is negative. This means that we want to find the exact point where the minimum of the entropy (excluding m = 1) is zero. We start by setting $\tilde{\kappa}_0 = 0$ in equation (81) to obtain the simplified form of the local entropy

$$\begin{align} \phi_1\left(r = \frac{-\alpha \tilde{r}}{4\log\left(\alpha\right)}\right) = \frac{\alpha \tilde{r}}{4}+\alpha\log\left[\textrm{erf}\left(\frac{\tilde{\kappa}_\mathrm{ener}}{\sqrt{2 \tilde{r}}}\right)\right]+o\left(\alpha\right)\, . \end{align} \tag{ 86 }$$

The potential is then null when

$$\begin{align} 1 = -4\frac{\log\left[\textrm{erf}\left(\frac{\tilde{\kappa}_\mathrm{ener}}{\sqrt{2 \tilde{r}}}\right)\right]}{\tilde{r}}+o\left(1\right) \end{align} \tag{ 87 }$$

and the rhs of the upper equation has a maximum for

$$\begin{align} \frac{\log\left[\textrm{erf}\left(\frac{\tilde{\kappa}_\mathrm{ener}}{\sqrt{2 \tilde{r}}}\right)\right]}{\tilde{r}^2}+\sqrt{\frac{2}{\pi}}\frac{\tilde{\kappa}_\mathrm{ener} e^{\frac{-\tilde{\kappa}_\mathrm{ener}^2}{2\tilde{r}}}}{\textrm{erf}\left(\frac{\tilde{\kappa}_\mathrm{ener}}{\sqrt{2 \tilde{r}}}\right) \tilde{r}^{5/2}} = o\left(1\right)\, . \end{align} \tag{ 88 }$$

Finally, if we solve the two previous equations, we obtain the set of values $\{\tilde{\kappa}_\textrm{ener},\tilde{r}\}$ for which the potential stops being negative for any value of the magnetization m. Numerically we obtain

$$\begin{align} \kappa_\mathrm{ener}& = \tilde{\kappa}_\mathrm{ener} \sqrt{-\alpha /\log\left(\alpha\right)}+o\left(\frac{\alpha}{\log\left(\alpha\right)}\right)\approx1.238\,518 \sqrt{-\alpha /\log\left(\alpha\right)}\, , \end{align} \tag{ 89 }$$

$$\begin{align} 1-m^2& = -\alpha\tilde{r} /\log\left(\alpha\right)+o\left(\frac{\alpha}{\log\left(\alpha\right)}\right)\approx-1.351\,180\,\alpha /\log\left(\alpha\right)\, . \end{align} \tag{ 90 }$$

The entropic threshold occurs when the local maximum other than m ≠ 0 of the free entropy cease to exist. We recall that the local entropy for $\tilde{\kappa}_0 = 0$ reads

$$\begin{align} \phi\left(r = \frac{-\alpha \tilde{r}}{4\log\left(\alpha\right)}\right) = \frac{\alpha \tilde{r}}{4}+\alpha\log\left[\textrm{erf}\left(\frac{\tilde{\kappa}_\mathrm{ener}}{\sqrt{2 \tilde{r}}}\right)\right]+o\left(\alpha\right) \end{align} \tag{ 91 }$$

with a non-trivial local maximum obtained by solving the fixed point equation

$$\begin{align} \frac{\alpha}{4} = \alpha\frac{\tilde{\kappa}_\mathrm{ener} e^{-\frac{\tilde{\kappa}_\mathrm{ener}^2}{2\tilde{r}}}}{\sqrt{2\pi} \tilde{r}^{3/2} \textrm{erf}\left(\frac{\tilde{\kappa}_\mathrm{ener}}{\sqrt{2 \tilde{r}}}\right)}+o\left(\alpha\right)\,. \end{align} \tag{ 92 }$$

Again, the rhs of the previous has a maximum for

$$\begin{align} \frac{\tilde{\kappa}_\mathrm{ener}^2}{2\tilde{r}^2}-\frac{3}{2\tilde{r}}+\frac{\tilde{\kappa}_\mathrm{ener} e^{-\frac{\tilde{\kappa}_\mathrm{ener}^2}{2\tilde{r}}}}{\sqrt{2\pi}\tilde{r}^{3/2} \textrm{erf}\left(\frac{\tilde{\kappa}_\mathrm{ener}}{\sqrt{2 \tilde{r}}}\right)} = o\left(1\right)\, . \end{align} \tag{ 93 }$$

Finally, we can solve numerically the two previous equations and we obtain

$$\begin{align} \kappa_\mathrm{entr}& = \tilde{\kappa}_\mathrm{entr} \sqrt{-\alpha /\log\left(\alpha\right)}+o\left(\frac{\alpha}{\log\left(\alpha\right)}\right)\approx1.428\,754 \sqrt{-\alpha /\log\left(\alpha\right)}\, , \end{align} \tag{ 94 }$$

$$\begin{align} 1-m^2& = -\alpha\tilde{r} /\log\left(\alpha\right)+o\left(\frac{\alpha}{\log\left(\alpha\right)}\right)\approx-0.782\,487\,\alpha /\log\left(\alpha\right)\, . \end{align} \tag{ 95 }$$

In this section, we focus on the relation between the complexity of the clusters around the high-margin solutions and their local entropy. We define the complexity as the logarithm of the number of clusters around solutions at margin κ₀, normalized by N, and we recall that the local entropy of a cluster is the value of the local entropy $\phi_1(r = \frac{1-m}{2})$ at the nearest local maximum to the reference solution. By contiguity to the planted model, the clusters of solutions with margin $\kappa\gt\kappa_0$ living around two different planted configurations are distant, since the reference configurations are nearly orthogonal with high probability. Thus, heuristically, counting their exponential number (or complexity) simply consists of enumerating the number of typical solutions at κ₀ we can plant.

Taking these previous considerations into account the obtained clusters have a complexity that depends solely on κ₀ while their local entropy is a function of κ and κ₀. Fixing κ while tuning κ₀ enables us to scan across sets of clusters with different complexities and local entropies, all containing atypical solutions of the SBP with margin κ. More specifically, the complexity is

$$\begin{align} \Sigma\left[\kappa_0\right] = \log 2+\alpha\log \operatorname{\mathbb{P}}\left(|Z| \unicode{x2A7D} \kappa_0\right) = \log 2 +\alpha\log\left\{{\mathrm{erf}\left[\frac{\kappa_0}{\sqrt{2}}\right]}\right\}\, , \end{align} \tag{ 96 }$$

and the entropy of a cluster is

$$\begin{align} s\left[\kappa_0,\kappa\right]& = -\frac{1-m}{2}\log{\left(\frac{1-m}{2}\right)}+\frac{\alpha}{\mathcal{N}_{\kappa_0}}\int\! \mathcal{D}Z_0\, \mathbf{1}\left\{\vert Z_0\vert \unicode{x2A7D} \kappa_0\right\}\,\nonumber\\& \quad \times \log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+Z_0}{\sqrt{2\left(1-m^2\right)}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-Z_0}{\sqrt{2\left(1-m^2\right)}}\right]}\right\}\,\nonumber\\ &\quad +o\left(\frac{1-m}{2}\log{\left(\frac{1-m}{2}\right)}\right) \, ,\nonumber \end{align} \tag{ 97 }$$

in which m is evaluated with the fixed-point equation

$$\begin{align} -\log{\left(\frac{1-m}{2}\right)}+o\left(\log{\left[\frac{1-m}{2}\right]}\right)& = \frac{4\alpha m}{{\mathcal{N}}_{\kappa_0}} \int \mathcal{D}B\frac{\mathbf{1}\left\{\vert B\vert \unicode{x2A7D} \kappa_0\right\}}{{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2\left(1-m^2\right)}}\right]}+{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2\left(1-m^2\right)}}\right]}} \nonumber\\& \quad\times \left\{\frac{\left(\kappa+B\right)e^{\frac{-\left(\kappa+B\right)^2}{2\left(1-m^2\right)}}}{\sqrt{2\pi}\left(1-m^2\right)^{3/2}}+\frac{\left(\kappa-B\right)e^{\frac{-\left(\kappa-B\right)^2}{2\left(1-m^2\right)}}}{\sqrt{2\pi}\left(1-m^2\right)^{3/2}}\right\} \, . \end{align} \tag{ 98 }$$

Using the rescaling from the previous section we can finally write for these two functions in the leading order in α → 0:

$$\begin{align} s\left[\tilde{\kappa}_0,\tilde{\kappa}\right]& = \frac{\alpha\tilde{r}}{4}+\frac{\alpha }{{\mathcal{N}}_{\tilde{\kappa}_0}}\int\! \mathcal{D}B\, \mathbf{1}\left\{\vert B\vert \unicode{x2A7D} \kappa_0\right\}\,\log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\tilde{\kappa}+B}{\sqrt{2\tilde{r}}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\tilde{\kappa}-B}{\sqrt{2\tilde{r}}}\right]}\right\}+o\left({\alpha}\right)\, , \end{align} \tag{ 99 }$$

$$\begin{align} \Sigma\left[\tilde{\kappa}_0\right]& = \log\left(2\right)+\frac{\alpha}{2}\log\left( \frac{\alpha}{\log\alpha}\right)+\alpha\log\left({\tilde{\kappa}_0}\right) = \Sigma_o +\alpha\log\left(\sqrt{\frac{2}{\pi}}{\tilde{\kappa}_0}\right)\, , \end{align} \tag{ 100 }$$

where we recall that $\tilde{r}$ is evaluated with

$$\begin{align} 1& = \frac{4}{{\mathcal{N}}_{\tilde{\kappa}_0}}\int \mathcal{D}B\frac{\mathbf{1}\left\{\vert B\vert \unicode{x2A7D} \kappa_0\right\}}{{\mathrm{erf}\left[\frac{\tilde{\kappa}+B}{\sqrt{2\tilde{r}}}\right]}+{\mathrm{erf}\left[\frac{\tilde{\kappa}-B}{\sqrt{2\tilde{r}}}\right]}} \left\{\frac{\left(\tilde{\kappa}+B\right)e^{\frac{-\left(\tilde{\kappa}+B\right)^2}{2\tilde{r}}}}{\sqrt{2\pi}\tilde{r}^{3/2}}+\frac{\left(\tilde{\kappa}-B\right)e^{\frac{-\left(\tilde{\kappa}-B\right)^2}{2\tilde{r}}}}{\sqrt{2\pi}\tilde{r}^{3/2}}\right\}+o\left({\alpha}\right)\, \end{align} \tag{ 101 }$$

and

$$\begin{align} 1-{m}^2& = -\alpha \tilde{r} /\log\left(\alpha\right)\, ,\; \kappa_0 = {\tilde{\kappa}_0} \sqrt{-\alpha /\log\left(\alpha\right)}\, ,\; \kappa = {\tilde{\kappa}} \sqrt{-\alpha /\log\left(\alpha\right)}\, , \nonumber\\ \Sigma_o& = \log\left(2\right)+\alpha\log\left( \frac{-\alpha}{\log\alpha}\right)\, . \end{align} \tag{ 102 }$$

In figure 2 the right-hand side displays several curves of complexity $\Sigma[\tilde{\kappa}_0]$ as a function of the local entropy $s[\tilde{\kappa}_0,\tilde{\kappa}]$ for fixed values of $\tilde{\kappa}$. Three regimes can be outlined for $\kappa \lt \kappa_\textrm{entr}$. First, for $s[\tilde{\kappa}_0,\tilde{\kappa}]\approx 0$, we have locally convex curves (and $\tilde{\kappa}_0-\tilde{\kappa} = o(1)$). This result appears quite surprising as usually these $\Sigma(s)$ curves are fully concave [14, 38]. Then, the curve becomes concave while having $s[\tilde{\kappa}_0,\tilde{\kappa}] = \mathcal{O}(\alpha)$ and $\tilde{\kappa}_0-\tilde{\kappa} = \mathcal{O}(1)$. In this regime, the complexity continues to scale as $\Sigma[\tilde{\kappa}_0]-\Sigma_o = \mathcal{O}(\alpha)$ and the local entropy is upper bounded by $s[0,\tilde{\kappa}]$. Finally, if we set $\tilde{\kappa}_0\ll \tilde{\kappa}$ (i.e. $\kappa_0 = o\left(\sqrt{-\alpha/\log(\alpha)}\right)\,$) the complexity jumps from $\Sigma[\tilde{\kappa}_0]\approx\Sigma_o$ to $\Sigma[\tilde{\kappa}_0] = 0$. In this case, the entropy remains fixed (in first order) at $s[\tilde{\kappa}_0,\tilde{\kappa}] = s[0,\tilde{\kappa}]$. We sketched these three regimes for the complexity versus entropy curves in figure 3. For $\tilde{\kappa} \gt \tilde{\kappa}_\textrm{entr}$ only the first regime exists since for small enough κ₀ the local maximum of the potential disappears.

Zoom In Zoom Out Reset image size

In this section, we show how the clustered structures we obtained with the planting approach can also be observed via the ordinary 1-RSB computation [32]. For this we will consider the set of solutions $S({\boldsymbol{G}}, \kappa)$ of the unbiased SBP. In particular, we will consider its cardinality

$$\begin{align} Z\left(\kappa\right) : = \big| S\left({\boldsymbol{G}}, \kappa\right)\big| \end{align} \tag{ 103 }$$

and its total entropy function as the logarithm of Z averaged over the disorder G

$$\begin{align} \phi_{N} : = \frac{1}{N} \operatorname{\mathbb{E}}_{{\boldsymbol{G}}}\left[ \log Z\left( \kappa \right) \right]\, . \end{align} \tag{ 104 }$$

So as to perform the average over the disorder we will use the replica trick [32]. This trick takes the form of

$$\begin{align} \operatorname{\mathbb{E}}_{{\boldsymbol{G}}}\left[ \log Z\left( \kappa \right) \right]\underset{n\rightarrow 0}{ = }\frac{\operatorname{\mathbb{E}}_{{\boldsymbol{G}}}\left[ Z^n\left( \kappa \right)\right]-1}{n} \end{align} \tag{ 105 }$$

where each of the n introduced copies of the system is called a replica. This technique enables to shift from a computation where interactions are random and the replica decoupled to a computation where the replica interacts with deterministic couplings. With this approach, the rest of the computation mainly consists in evaluating the quantity $\operatorname{\mathbb{E}}_{{\boldsymbol{G}}}\left[ Z^n( \kappa )\right]$ at a fixed-point of the overlap matrix $Q\in \mathrm{I\!R^{n\times n}}$, where

$$\begin{align} Q^{a,b} = \operatorname{\mathbb{E}}_{\boldsymbol{G}}\left[{\boldsymbol{y}}^a\cdot {\boldsymbol{y}}^{b} \,\,\Big\vert\,\, {\boldsymbol{y}}^a,{\boldsymbol{y}}^b\in S\left({\boldsymbol{G}}, \kappa\right) \right]\quad\text{and} \quad a,b\in\left[\!\left[1,n\right]\!\right]\, . \end{align} \tag{ 106 }$$

Moreover, as the constraints on the overlaps are introduced in the following fashion

$$\begin{align} \delta\left({\boldsymbol{y}}^a\cdot {\boldsymbol{y}}^{b}-Q^{a,b}\right) = \int \mathrm{d}\widehat{Q}^{a,b} e^{i \widehat{Q}^{a,b} \left({\boldsymbol{y}}^a\cdot {\boldsymbol{y}}^{b}-Q^{a,b}\right) }\underset{N\rightarrow +\infty}{ = } \int \mathrm{d}\widehat{Q}^{a,b} e^{\widehat{Q}^{a,b} \left({\boldsymbol{y}}^a\cdot {\boldsymbol{y}}^{b}-Q^{a,b}\right) , } \end{align} \tag{ 107 }$$

we will have also to evaluated $\operatorname{\mathbb{E}}_{{\boldsymbol{G}}}\left[ Z^n( \kappa )\right]$ at a fixed-point of the matrix $\widehat{Q}$. In more detail, the computation consists of evaluating

$$\begin{align} \operatorname{\mathbb{E}}_{\boldsymbol{G}}\left[ Z^n\left( \kappa \right)\right] & = \prod_{a = 1}^n\left(\sum_{\boldsymbol{y}^a\in \left\{-1,1\right\}^N}\int\prod_{a < b}\mathrm{d}Q^{a,b} \mathrm{d}\widehat Q^{a,b} e^{\widehat{Q}^{a,b} \left({\boldsymbol{y}}^a\cdot {\boldsymbol{y}}^{b}-Q^{a,b}\right) }\right. \nonumber\\ & \left. \quad \times \int \prod_{a = 1}^n \prod_{j = 1}^M \left[\mathrm{d} v^{a,j} \Theta\left(\kappa-v^{a,j}\right)\right] e^{-\frac{1}{2}\sum_{a,b}v^{a,j} \Sigma^{a,b} v^{b,j}}\right) . \end{align} \tag{ 108 }$$

with

$$\begin{align} v^{a,j} = {\boldsymbol{g}}_j\cdot {\boldsymbol{y}}^a\quad\text{and}\quad \Sigma^{a,b} = \operatorname{\mathbb{E}}_{\boldsymbol{G}}\left[\left({\boldsymbol{g}}_j\cdot {\boldsymbol{y}}^a\right)\left({\boldsymbol{g}}_j\cdot {\boldsymbol{y}}^b\right)\right] = \operatorname{\mathbb{E}}_{\boldsymbol{G}}\left[{\boldsymbol{y}}^a\cdot {\boldsymbol{y}}^b\right] = Q^{a,b}. \end{align} \tag{ 109 }$$

The computation of $\operatorname{\mathbb{E}}_{{\boldsymbol{G}}}\left[ Z^n( \kappa )\right]$ with the 1-step replica symmetric (1-RSB) ansatz implies the following form for the matrices Q and $\widehat Q$

$$\begin{align} Q^{a,b} = \left\{ \begin{array}{ll} 1 & \text{if}\; a = b \\ q_1 & \lfloor\frac{a}{x}\rfloor = \lfloor\frac{b}{x}\rfloor \\[4pt] q_0 & \text{otherwise}\\ \end{array} \right.\! \quad\text{and}\quad \widehat Q^{a,b} = \left\{ \begin{array}{ll} \widehat{Q} & \text{if}\; a = b \\ \widehat q_1 & \lfloor\frac{a}{x}\rfloor = \lfloor\frac{b}{x}\rfloor \\[4pt] \widehat q_0 & \text{otherwise}\\ \end{array} \right.\! . \end{align} \tag{ 110 }$$

With this ansatz equation (108) boils down to

$$\begin{align} \phi^\textrm{1-RSB}& = \lim_{n \to 0}\, \lim_{N \to \infty} \frac{\operatorname{\mathbb{E}}_{{\boldsymbol{G}}}\left[ Z^n\left( \kappa \right)\right]-1}{nN} = -\frac{x\widehat{q}_1}{2}+\frac{x\left(1-x\right)q_1\widehat{q}_1}{2}+\frac{x^2 q_0\widehat{q}_0}{2}\nonumber\\ &\quad +\alpha\int \mathcal{D}t \, \log\left\{\int \mathcal{D}z \, e^{x\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1-q_0}z+\sqrt{q_0}t,1-q_1\right]}\right\}\nonumber\\ &\quad +\int \mathcal{D}t \, \log\left\{\int \mathcal{D}z \, e^{x\phi_\mathrm{in}\left[\sqrt{\widehat{q}_1-\widehat{q}_0}z+\sqrt{\widehat{q}_0}t\right]}\right\} \end{align} \tag{ 111 }$$

with

$$\begin{align} \phi^{\kappa}_\mathrm{out}\left[\omega,V\right]& = \log\left[\int_{\frac{-\omega-\kappa}{\sqrt{V}}}^{\frac{-\omega+\kappa}{\sqrt{V}}} \mathcal{D}u\right] = \log\left[\frac{1}{2}\textrm{erf}\left(\frac{\kappa-\omega}{\sqrt{2V}}\right)+\frac{1}{2}\textrm{erf}\left(\frac{\kappa+\omega}{\sqrt{2V}}\right)\right]\, , \end{align} \tag{ 112 }$$

$$\begin{align} \phi_\mathrm{in}\left[B\right]& = \log\left[\sum_{x = \pm 1}e^{Bx}\right] = \log\left[2\cosh{\left(B\right)}\right]\, . \end{align} \tag{ 113 }$$

For more details on the calculation steps to derive $\phi^\textrm{1-RSB}$ we redirect the interested readers to the first appendix of [4]. Before moving on with the analysis of the 1-RSB potential, a first simplification consists in taking into account a symmetry in the in/out channels: $\phi^{\kappa}_\textrm{out}[\omega,V] = \phi^{\kappa}_\textrm{out}[-\omega,V]$ and $\phi_\textrm{in}[B] = \phi_\textrm{in}[-B]$. Indeed, this symmetry implies that optimizing the potential yields the solution $q_0 = \widehat{q}_0 = 0$. Thus, in the following, we will always take this solution. Then, the remaining equations we have to verify for the fixed point are

$$\begin{align} \widehat{q}_1& = f\left(q_1\right) = -\frac{2\alpha}{\left(1-x\right)}\times \dfrac{\int \mathcal{D}z \,\partial_{q_1}\phi^{\mathrm{out}}\left[\sqrt{q_1}z,1-q_1\right] e^{x\phi^\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right]}}{\int \mathcal{D}z \, e^{x\phi^\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right]}}\, , \end{align} \tag{ 114 }$$

$$\begin{align} q_1& = g\left(\widehat q_1\right) = \frac{2}{1-x}\left[\frac{1}{2}-\dfrac{\int \mathcal{D}z \,\partial_{\widehat{q}_1}\phi^\mathrm{in}\left[\sqrt{\widehat{q}_1}z\right] e^{x\phi^\mathrm{in}\left[\sqrt{\widehat{q}_1}z\right]}}{\int \mathcal{D}z \, e^{x\phi^\mathrm{in}\left[\sqrt{\widehat{q}_1}z\right]}}\right]\, . \end{align} \tag{ 115 }$$

With these definitions, the entropy and complexity of the clusters can be determined at the fixed point as

$$\begin{align} s = \partial_x \phi^\textrm{1-RSB}\, , \quad \Sigma = \phi^\textrm{1-RSB}-xs\quad \text{and}\quad \frac{\partial \Sigma}{\partial s} = -x\, . \end{align} \tag{ 116 }$$

When it comes to solving the 1RSB equations, we focus in this subsection on α = 0.5 as a representative value not close to zero, the corresponding satisfiability threshold is $\kappa_{\tiny \mathrm{SAT}}(\alpha = 0.5) = 0.319$. We obtained four branches of solutions when solving the fixed-point equations (114) and (115) with respect to q₁ and $\widehat{q}_1$ for the 1-RSB potential (and browsing through values for the Parisi parameter x). Two of these solutions are unstable under the iteration scheme

$$\begin{align} \widehat{q}_1^{t+1} = f\left(\widehat{q}_1^t\right)\, ,\quad {q}_1^{t+1} = g\left(q_1^t\right)\, , \end{align} \tag{ 117 }$$

while the remaining two are stable. When browsing different values of x, we also observe a threshold value for κ for which the overall behavior of these fixed points changes. We will call this value $\kappa_\textrm{break}(\alpha = 0.5)\approx 0.455$. In figure 4(left panel) we plot the complexity Σ as a function of their entropy s for the four branches. When tuning x each solution describes a trajectory that we highlighted with either a dashed (unstable fixed point) or a full line (stable fixed point). One key question arising from these results is how we should select the fixed-point branch that corresponds to the actual clusters of solutions in the problem. First, we clearly need to restrict to non-negative Σ and non-negative s. Moreover, we know that the correct equilibrium state is given by the solution where the total entropy

$$\begin{align} s_\mathrm{total} = \Sigma+s ~\big|~ \Sigma \unicode{x2A7E} 0\, ,~ s\unicode{x2A7E} 0 \end{align} \tag{ 118 }$$

is maximized. For the present model, this happens for s = 0 when the (negative) slope of the $\Sigma(s)$ curve is infinite. This can be seen by realizing that the slope of the curve $\Sigma(s)$ is much smaller than −1. We recall that this slope is equal to −x, where x is the Parisi parameter, as explained in equation (116). We highlighted this equilibrium point with a colored dot in the left panel of figure 4. In particular, this point $\Sigma(0)$ corresponds to the equilibrium frozen 1RSB solution of the SBP problem with a value corresponding to one computed in [4]. We note that this criterion for equilibrium is rather unusual among other models where the 1RSB solution was evaluated. Usually, either both $\Sigma\gt0$ and s > 0 at the point where the negative slope x = 1 corresponding to the so-called dynamical-1RSB phase, or the maximum is achieved when $\Sigma = 0$ at a (negative) slope strictly between $0\lt x \lt1$ corresponding to the so-called static-1RSB phase. Here, we observe the equilibrium being achieved for $x\rightarrow +\infty$ corresponding to frozen-1RSB at equilibrium. Finally, we observe that for $\kappa\gt\kappa_\textrm{break} \sim 0.455$ (still considering α = 0.5) the curve $\Sigma(s)$ for positive values of both s and Σ breaks into two branches. Consequently, there is a finite range of values for the entropy s where we do not obtain any fixed point. The meaning of such a gap is unclear, but it appears in other problems and their 1-RSB solution [39].

Zoom In Zoom Out Reset image size

In figure 4 right panel, we plot the complexity Σ as a function of the entropy s selecting the branch that is an analytic continuation of the equilibrium $\Sigma(0)$ point and compare it to the one obtained via the planting approach. For this comparison, we need the local entropy in the planted model at finite values of α that is derived in appendix . We note that the two complexities exactly agree at s = 0 as is expected because at s = 0 both the complexities correspond to the total number of solutions at that κ. For s > 0, the two complexities have a similar shape, being clearly convex for small values of s. We note again that the overall values of Σ are larger than the values of s meaning that the slope actually takes a rather large value in the whole range of those curves. We recall that in the context of the 1-RSB computation this slope is equal to −x, where x is the Parisi parameter. Taking its value much larger than one is not common in other models for which 1RSB was studied. This is likely the reason why these extensive size clusters were not described earlier in the literature for the binary perceptron. We further see that the 1RSB complexity, when it exists, is strictly larger than the one obtained via planting as again expected since via planting we obtain only some of the clusters of solution whereas the 1RSB computation should be able to count all of them. Then, when $\kappa \gt \kappa_\textrm{break}$, the planted model predicts the existence of clusters with an internal entropy that lies inside the fixed-point gap of the 1-RSB approach. This indicates that the 1RSB solution does not fully describe the space of solutions in this case. This may have many causes. For example, we may have missed a branch of fixed points in our analysis of the 1-RSB potential. Or, this region may involve a replica ansatz with further symmetry breaking. Or perhaps these rare clusters simply cannot be obtained with a replica computation. Finally, when $\kappa\gt\kappa_\textrm{ener}(\alpha = 0.5)\approx 0.499$, the curves $\Sigma(s)$ obtained with planting stop at some positive values of Σ and s and thus look again qualitatively similar to the portion of the curve $\Sigma(s)$ that is obtained from 1RSB by analytically continuing from the equilibrium $\Sigma(s = 0)$ point.

Overall, the 1RSB approach evaluated at sufficiently larger values of the Parisi parameter x identified clusters of extensive size in parts of the solution corresponding to a convex curve that is unstable under the iterations of the 1RSB fixed point equations. These curves are partly compatible with the complexity obtained from planting. Yet there are still regions of $\kappa, s$ for which we obtain extensive clusters of solution from the planting procedure but not from the 1RSB. The reason behind this paradox is left for future work.

For small α, the situation becomes actually clearer. In the next section, we will discuss this case.

We now focus, as in the first part of the paper, on the regime of small α. Using our results from the planting computation, and anticipating a similarity of behaviour in the 1RSB, we can deduce the behavior of the Parisi parameter x in the low α limit. Indeed, alike the 1-RSB computation, we saw that the planting approach probes clustered solutions. It also allows for computing their complexity Σ and local entropy s, see equations (99) and (100). As mentioned above, in the context of a 1-RSB computation we have $\partial\Sigma/\partial s = -x$. Thus, if we plug-in the entropy and complexity from equations (99) and (100) we can compute $\partial\Sigma/\partial s$ and estimate the Parisi parameter.

By doing so we obtain two regimes for which the slope $\partial\Sigma/\partial s$ becomes infinite in the low α limit. First, when $\tilde{\kappa}_0\ll\tilde{\kappa}$ the entropy remains constant at first order in α while the complexity roughly jumps from $\Sigma_o$ to zero, see the left panel in figure 2. It indicates that to recover these states with the 1-RSB computation we should set $x\gg1$. The second regime for which we observe an infinite slope corresponds to $\tilde{\kappa}_0\approx\tilde{\kappa}$. Indeed, we have $\vert\partial \Sigma/ \partial s \vert\sim (\tilde{\kappa}-\tilde{\kappa}_0)^{-1}$ close to $\tilde{\kappa}_0 = \tilde{\kappa}$. Consequently, if we want to probe the clusters with almost zero local entropy we should also set $x\gg1$ to obtain this regime.

A last piece of information given by the planted model is that these clusters (in the two regimes mentioned above) correspond to a limit where $q = m^2\approx 1$. Thus, if we impose the same condition in the 1-RSB fixed-point equations, we have that setting $q_1\approx1$ in equation (115) implies $\widehat q_1\gg 1$. Therefore, in order to find these clusters we will not only set $x\gg 1$ but we will also take $q_1\approx 1$ and $\widehat q_1\gg 1$.

The first regime we mentioned will be referred as the maximum entropy regime, while the second one will be referred as the minimum entropy regime. First, we see that the entropic contribution can be simplified identically in both regimes. Indeed, when setting $\widehat q_1\gg1$ we obtain

$$\begin{align} \phi_\mathrm{in}\left[\sqrt{\widehat{q}_1}z\right]&{\approx}\sqrt{\widehat{q}_1}\vert z\vert+\log\left[1+e^{-2\sqrt{\widehat{q}_1}\vert z\vert}\right]\nonumber\\ &{\approx}\sqrt{\widehat{q}_1}\vert z\vert+e^{-2\sqrt{\widehat{q}_1}\vert z\vert}\nonumber \end{align} \tag{ 119 }$$

which then yields

$$\begin{align} \log\left\{\int \mathcal{D}z \, e^{x\phi_\mathrm{in}\left[\sqrt{\widehat{q}_1}z\right]}\right\}&\approx \log\left\{2\int_0^{+\infty} \mathcal{D}z \,e^{x\sqrt{\widehat{q}_1}z+xe^{-2\sqrt{\widehat{q}_1}z}} \right\}\nonumber\\ &\approx \log\left\{2\int_0^{+\infty} \mathcal{D}z \,e^{x\sqrt{\widehat{q}_1}z}\left(1+xe^{-2\sqrt{\widehat{q}_1}z}\right) \right\}\nonumber\\ &\approx \log\left\{2 \left(e^{\frac{x^2 \widehat{q}_1}{2}}+xe^{\frac{\left(x-2\right)^2 \widehat{q}_1}{2}}\right) \right\}\nonumber\\ &\approx \log 2+\frac{x^2 \widehat{q}_1}{2}+x e^{-2\left(x-1\right) \widehat{q}_1}\, .\nonumber \end{align} \tag{ 120 }$$

As we will see in the following subsections, the simplification for the energetic term will be regime-dependent.

6.3.1. Maximum entropy regime.

For the maximum entropy regime, we can again go back to the results from the planting model to help us make the correct approximation. We know, for example, that we should have $1-q_1\sim \kappa^2$ in the low α limit (as both quantities have the same scaling in α). This implies that we should have

$$\begin{eqnarray} &&\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right] = \mathcal{O}\left(1\right) \quad \text{with}\quad z\in\left[-\kappa,\kappa\right]. \end{eqnarray} \tag{ 121 }$$

Therefore, with $x\gg1$, we will approximate the energetic term with a saddle-point method. In other words, we will compute

$$\begin{align} \int \mathcal{D}z \, e^{x\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right]}\approx e^{x\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}Z_0,1-q_1\right]} \int \mathcal{D}z \, e^{x\frac{q_1 \left(z-Z_0\right)^2}{2}\partial^2_\omega\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}Z_0,1-q_1\right]} \end{align} \tag{ 122 }$$

where Z₀ corresponds to the maxima of $\phi^{\kappa}_\textrm{out}[\sqrt{q_1}z,1-q_1]$. In particular, with this function we have Z₀ = 0. By doing the saddle-point approximation in Z₀ = 0 we obtain

$$\begin{align} \alpha\log\left\{\int \mathcal{D}z \, e^{x\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right]}\right\} \approx \alpha x \log\left[\textrm{erf}\left(\frac{\kappa}{1-q_1}\right)\right]-\frac{\alpha}{2}\log\left[1+\Delta x\right] \end{align} \tag{ 123 }$$

with

$$\begin{align} \Delta = \left\vert q_1 \partial_\omega^2 \phi^{\kappa}_\mathrm{out}\left(0,1-q_1\right) \right\vert \, . \end{align} \tag{ 124 }$$

Finally, if we combine the simplification of both the entropic and energetic contributions the total entropy becomes

$$\begin{align} \phi^{\textrm{1-RSB}}\approx&-\frac{x\widehat{q}_1}{2}+\frac{x\left(1-x\right)q_1\widehat{q}_1}{2}+\alpha x \log\left[\textrm{erf}\left(\frac{\kappa}{\sqrt{2\left(1-q_1\right)}}\right)\right]\nonumber\\ &-\frac{\alpha}{2}\log\left(1+\Delta x\right)+\log\left(2\right)+\frac{x^2\widehat{q}_1}{2}+xe^{-2\left(x-1\right)\widehat{q}_1}\, \nonumber \end{align} \tag{ 125 }$$

and its fixed point equations are, at first order in x,

$$\begin{align} \frac{x\left(x-1\right)\widehat{q}_1}{2}& = \frac{\alpha x \kappa e^{\frac{-\kappa^2}{2\left(1-q1 \right)}}}{\sqrt{2\pi}\left(1-q_1\right)^{3/2} \textrm{erf}\left(\frac{\kappa}{\sqrt{2\left(1-q_1\right)}}\right)}\, , \end{align} \tag{ 126 }$$

$$\begin{align} q_1& = 1-4e^{-2\left(x-1\right)\widehat{q}_1}\, . \end{align} \tag{ 127 }$$

Now, we are able to draw a direct parallel with the planted system at low α and $\kappa_0\ll \kappa$. Indeed, if we use the correspondence $q_1\equiv m^2$ and $(x-1)\widehat{q}_1\equiv \widehat{m}$, equations (126) and (127) are nothing but the fixed-point equations for the planted model (see equations (A23) and (A24)). This indicates a posteriori that the 1-RSB calculation enables us to recover the same clusters as in the planted model where we have set $\kappa_0\ll \kappa$. To make the identification between the two approaches even more direct we can focus on the entropy and the complexity of this 1-RSB fixed-point. We obtain at first order in x that

$$\begin{align} s& = \partial_x \phi^{\textrm{1-RSB}} = \frac{\left(1-q_1\right)x\widehat{q}_1}{2}+\alpha\log\left[\textrm{erf}\left(\frac{\kappa}{\sqrt{2\left(1-q_1\right)}}\right)\right]\nonumber\\ & = \left. \phi\left(r = \frac{1-m}{2}\right)\right\vert_{\alpha\ll 1,\,\kappa_0\ll \kappa,\, m\approx1} \end{align} \tag{ 128 }$$

and

$$\begin{align} \Sigma = \phi^{\textrm{1-RSB}}-xs = 0\, . \end{align} \tag{ 129 }$$

Thus, at first order in x these clustered states have exactly the same entropy and complexity as the ones from the planted system with $\kappa_0\ll\kappa$ (and $\alpha\ll 1$).

6.3.2. Minimum entropy regime.

In this section, we want to probe clusters with a very small entropy. Now, keeping κ fixed, this means that we will have to set q₁ extremely close to one and eventually have $1-q_1\ll\kappa$ in order to go up to zero local entropy. This scaling between q₁ and κ is incompatible with the saddle-point approximation we performed for the maximum entropy regime, as equation (121) is not verified anymore. In fact, in this case, we have to introduce an asymptotic expansion

$$\begin{align} \phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right]&\approx\log\left\{\Theta\left(\kappa/\sqrt{q_1}-\vert z\vert \right)-\sqrt{\frac{1-q_1}{2\pi}}\left[\frac{e^{\frac{-\left(\kappa+\sqrt{q_1}z\right)^2}{2\left(1-q_1\right)}}}{\kappa+\sqrt{q_1}z}+\frac{e^{\frac{-\left(\kappa-\sqrt{q_1}z\right)^2}{2\left(1-q_1\right)}}}{\kappa-\sqrt{q_1}z}\right]\right\} \end{align} \tag{ 130 }$$

where we used the identity

$$\begin{align} \textrm{erf}\left(x\right)\underset{x\rightarrow+\infty}{\approx}1-\frac{e^{-x^2}}{x\sqrt{\pi}}\, . \end{align} \tag{ 131 }$$

We then estimate the interval of value of z ($z\in[-Z_0,Z_0]$) for which $e^{x\phi^{\kappa}_\textrm{out}[\sqrt{q_1}z,1-q_1]}$ remains finite. In other words, we compute the value of z for which the function $e^{x\phi^{\kappa}_\textrm{out}[\sqrt{q_1}z,1-q_1]}$ is equal to an arbitrary value $1/C$,

$$\begin{align} & e^{x\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right]} = \frac{1}{C}\nonumber\\ &\quad\implies x\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right] = -\log C \nonumber\\ &\quad\implies \sqrt{\frac{1-q_1}{2\pi}}\times\frac{e^{\frac{-\left(\kappa-\sqrt{q_1}z\right)^2}{2\left(1-q_1\right)}}}{\kappa-\sqrt{q_1}z}\underset{\frac{\kappa}{\sqrt{q_1}}>z>0}{ = }\frac{\log C}{x}\nonumber\\ &\quad\implies \frac{\kappa-\sqrt{q_1}z}{\sqrt{2\left(1-q_1\right)}}\underset{\frac{\kappa}{\sqrt{q_1}}>z>0}{ = }\sqrt{\frac{W_0\left(\frac{2}{a^2}\right)}{2}}\quad \text{with}\quad a = \frac{2 \log C}{x\sqrt{\pi}}\nonumber\\ & \quad\implies \frac{\kappa-\sqrt{q_1}z}{\sqrt{2\left(1-q_1\right)}}\underset{\underset{\frac{\kappa}{\sqrt{q_1}}>z>0}{x\rightarrow +\infty}}{ = }\sqrt{\frac{\log{x^2}}{2}}\nonumber\\ & \quad\implies z \underset{\underset{\frac{\kappa}{\sqrt{q_1}}>z>0}{x\rightarrow +\infty}}{ = }\frac{\kappa-\sqrt{2\left(1-q_1\right)\log x}}{\sqrt{q_1}}\, \nonumber \end{align} \tag{ 132 }$$

where $W_0(.)$ is the Lambert function with branch index k = 0. This computation thus shows that $e^{x\phi^{\kappa}_\textrm{out}[\sqrt{q_1}z,1-q_1]}$ jumps from 1 to any arbitrary fraction $1/C$ exactly at

$$\begin{align} Z_0 &\underset{\underset{\frac{\kappa}{\sqrt{q_1}}>z>0}{x\rightarrow +\infty}}{ = }\frac{\kappa-\sqrt{2\left(1-q_1\right)\log x}}{\sqrt{q_1}}\, . \end{align} \tag{ 133 }$$

In this limit, we thus have for the energetic contribution

$$\begin{align} \alpha \log\left\{\int \mathcal{D}z \, e^{x\phi^{\kappa}_\mathrm{out}\left[\sqrt{q_1}z,1-q_1\right]}\right\}& = {\alpha}\log\left\{\int_{-Z_0}^{Z_0}{D}z\right\} = {\alpha}\log\left\{\textrm{erf}\left(\frac{\kappa-\sqrt{2\left(1-q_1\right)\log x}}{\sqrt{2 q_1}}\right)\right\}\, . \end{align} \tag{ 134 }$$

And finally, if we put together the simplified entropic and energetic contribution to the 1-RSB potential we obtain

$$\begin{align} \phi^{\textrm{1-RSB}} \approx&-\frac{x\widehat{q}_1}{2}+\frac{x\left(1-x\right)q_1\widehat{q}_1}{2}+{\alpha}\log\left\{\textrm{erf}\left(\frac{\kappa-\sqrt{2\left(1-q_1\right)\log x}}{\sqrt{2 q_1}}\right)\right\}\nonumber\\ &+\log\left(2\right)+\frac{x^2\widehat{q}_1}{2}+xe^{-2\left(x-1\right)\widehat{q}_1}\, \nonumber \end{align} \tag{ 135 }$$

and the corresponding fixed point equations are (at first order in x)

$$\begin{align} \frac{x^2\widehat{q}_1}{2}& = \alpha\frac{e^{-{\kappa}^2/2}}{\textrm{erf}\left(\frac{\kappa}{\sqrt{2}}\right)}\sqrt{\frac{\log x}{\pi \left(1-q_1\right)}}\, , \end{align} \tag{ 136 }$$

$$\begin{align} q_1& = 1-4e^{-2\left(x-1\right)\widehat{q}_1}\, . \end{align} \tag{ 137 }$$

The combination of these two fixed-point equations implies $\kappa\gg\sqrt{2(1-q_1)\log x}$. Consequently, the term $x^2(1-q_1)\widehat{q}_1$ can be neglected and the 1-RSB free energy boils down to

$$\begin{align} \phi^{\textrm{1-RSB}}\approx \alpha\log\left\{\textrm{erf}\left(\frac{\kappa}{\sqrt{2 }}\right)\right\} +\log\left(2\right)\, . \end{align} \tag{ 138 }$$

We thus recover the case of the planted system at $\kappa_0 = \kappa$ as we have

$$\begin{align} s& = \partial_x \phi^{\textrm{1-RSB}} = 0\, , \end{align} \tag{ 139 }$$

$$\begin{align} \Sigma& = \phi^{\textrm{1-RSB}}\approx\alpha\log\left\{\textrm{erf}\left(\frac{\kappa}{\sqrt{2 }}\right)\right\} +\log\left(2\right)\,. \end{align} \tag{ 140 }$$

In [4] the authors showed in a more standard computation that these equilibrium configurations (verifying a frozen 1-RSB structure) can also be obtained by imposing q₀ = 0, q₁ = 1 and x = 1.

In figure 5 we display for several values of $\tilde{\kappa} = \kappa\sqrt{-\log(\alpha)/\alpha}$ the complexity as a function of the entropy. The light-colored full lines correspond to the results obtained with the planted model. The dashed and dotted lines correspond respectively to the maximum and minimum entropy fixed-point branches of the 1-RSB free energy. To obtain them we solved the fixed-point equations in each regime for large but finite Parisi parameter x. As shown by the previous computations each end of the curve sees a close match between the planting approach and one of the $x\rightarrow+\infty$ regimes. This leads us to conjecture that in the limit of small α, the planted and 1-RSB $\Sigma(s)$ curves match exactly. In other words, the planting approach actually captures the dominant clusters for each size s.

Zoom In Zoom Out Reset image size

While in the main text we derive the local entropy rigorously at small α, in this appendix we give the expressions for a generic value of α. This we have not established rigorously and instead resorted to the replica method applied to the contiguous planted system. Following [4] the replica symmetric (RS) free energy for the planted binary perceptron is

$$\begin{align} \phi^{\kappa_0,\kappa}_\mathrm{planted}\left[q,\widehat{q},m,\widehat{m}\right]& = \frac{1}{N}\operatorname{\mathbb{E}}_{{\boldsymbol{G}},{\boldsymbol{x}}_0} \log \left[Z\left({\boldsymbol{x}}_0 , \kappa , r = \frac{1-m}{2}\right) \right]\nonumber\\& = -\frac{1-q}{2}\widehat{q}-m\widehat{m}+\alpha\int \mathcal{D}z\, \mathrm{d}w P_{\kappa_0}\left[w\right]\,\phi^{\kappa}_\mathrm{out}\left[w,z,q,m\right]\nonumber\\ &\quad +\int\mathcal{D}z \,\left\{\phi_\mathrm{in}\left[z,\widehat{q},\widehat{m}\right]\right\} \, ,\nonumber \end{align} \tag{ A1 }$$

with

$$\begin{align} \phi^{\kappa}_\mathrm{out}\left[w,z,q,m\right]& = \log\left(\Phi^{\kappa}_\mathrm{out}\left[\omega = mw+\sqrt{q-m^2}z,V = 1-q\right]\right) \end{align} \tag{ A2 }$$

$$\begin{align} & = \log\left[\int_{\frac{-mw-\sqrt{q-m^2}z-\kappa}{\sqrt{1-q}}}^{\frac{-mw-\sqrt{q-m^2}z+\kappa}{\sqrt{1-q}}} \mathcal{D}u\right]\nonumber\\ & = \log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-mw-\sqrt{q-m^2}z}{\sqrt{2\left(1-q\right)}}\right]}\right.\nonumber\\ &\left.\qquad +\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+mw+\sqrt{q-m^2}z}{\sqrt{2\left(1-q\right)}}\right]}\right\}\, ,\nonumber\\ \phi_\mathrm{in}\left[z,\widehat{q},\widehat{m}\right] & = \log\left(\Phi_\mathrm{in}\left[B = \widehat{m}x_o+\sqrt{\widehat{q}}z\right]\right)\nonumber\\ & = \log\left[\sum_{x = \pm 1}e^{\left(\widehat{m}x_o+\sqrt{\widehat{q}}z\right)x}\right]\nonumber\\ & = \log\left[2\cosh\left(\widehat{m}x_o+\sqrt{\widehat{q}}z\right)\right] \end{align} \tag{ A3 }$$

and where $\mathcal{D}z$ (as well as $\mathcal{D}x$ later) represents an integration with a scalar normal-distributed variable. Moreover, in the following we will take $P_{\kappa_0}[w] = e^{-w^2/2}\mathbf{1}\{\vert w \vert \unicode{x2A7D} \kappa_0\}/\mathcal{N}_{\kappa_0}$ where $\mathcal{N}_{\kappa_0}$ is the prefactor normalizing the distribution. This distribution is given by the typical clusters dominating the probability measure $P^{{\boldsymbol{g}}}_{\kappa_0}[{\boldsymbol{x}}]$. The variable x_o corresponds to the planted configuration and can take the values ±1 indifferently.

This free energy has saddle points for a set of parameters $\{q,\widehat{q},m,\widehat{m}\}$ verifying

$$\begin{eqnarray} \widehat{m}& = &\alpha\int \mathcal{D}z\, \mathrm{d}w P_{\kappa_0}\left[w\right] \partial_m \phi^{\kappa}_\mathrm{out}\left[w,z,q,m\right]\, , \end{eqnarray} \tag{ A4 }$$

$$\begin{eqnarray} \widehat{q}& = &-2\alpha\int \mathcal{D}z\, \mathrm{d}w P_{\kappa_0}\left[w\right] \partial_q \phi^{\kappa}_\mathrm{out}\left[w,z,q,m\right]\, , \end{eqnarray} \tag{ A5 }$$

$$\begin{eqnarray} m& = &\int \mathcal{D}z\,x_o\, \textrm{tanh}\left(\widehat{m}x_o+\sqrt{\widehat{q}}z\right) = \int \mathcal{D}z\, \textrm{tanh}\left(\widehat{m}+\sqrt{\widehat{q}}z\right) \, , \end{eqnarray} \tag{ A6 }$$

$$\begin{eqnarray} q& = &1-\frac{1}{\sqrt{\widehat{q}}}\int \mathcal{D}z\, z\, \textrm{tanh}\left(\widehat{m}x_o+\sqrt{\widehat{q}}z\right) = \int \mathcal{D}z\, \textrm{tanh}^2\left(\widehat{m}+\sqrt{\widehat{q}}z\right)\, . \end{eqnarray} \tag{ A7 }$$

In figure 6 we plot the complexity, see equation (96), as a function of the local entropy of the planted model. As explained in section 5.3 the entropy is obtained by evaluating $\phi^{\kappa_0,\kappa}_\textrm{planted}$ with its only non-trivial saddle-point over $\{q,m,\widehat{q},\widehat{m}\}$. The complexity is the exponential number of possible typical solutions for the binary perceptron at α and κ₀, see equation (96), which corresponds to the number of possibilities of planting the configuration. The results we display in this figure are obtained for $\alpha = 10^{-2}$.

Zoom In Zoom Out Reset image size

In this section, we focus on the planted model in the limit $\alpha\ll 1$. Indeed, it can be shown in this case that the non-trivial saddle point of the planted free energy verifies $\widehat{q}\ll\widehat{m}$, $q-m^2\ll 1-m$ and $1-m = o(1)$. We start first by rewriting the energetic contribution as

$$\begin{align} \alpha\int \mathcal{D}z\, \mathrm{d}w P_{\kappa_0}\left[w\right]\,\phi^{\kappa}_\mathrm{out}\left[w,z,q,m\right]& = \alpha\int \mathcal{D}B\, \frac{1}{2\,\mathcal{N}_{\kappa_0}}\left\{{\mathrm{erf}\left[\frac{\sqrt{q}\kappa_0+mB}{\sqrt{q-m^2}}\right]}+{\mathrm{erf}\left[\frac{\sqrt{q}\kappa_0-mB}{\sqrt{q-m^2}}\right]}\right\} \nonumber\\ &\quad \times \log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{1-q}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{1-q}}\right]}\right\} \end{align} \tag{ A8 }$$

with the change of variable

$$\begin{align} B = \frac{mw+\sqrt{q-m^2}z}{\sqrt{q}}\, , \quad B^\top = \frac{-mz+\sqrt{q-m^2}w}{\sqrt{q}} \end{align} \tag{ A9 }$$

and integrating over $B^\top$. Recalling the fact that we focus on probing a saddle-point with $q-m^2\ll1-m$ and $1-m = o(1)$ (i.e. $1-q = o(1)$), we have for the energetic contribution

$$\begin{align} \alpha\int \mathcal{D}z\, \mathrm{d}w P_{\kappa_0}\left[w\right]\,\phi^{\kappa}_\mathrm{out}\left[w,z,q,m\right]& = \alpha\int \mathcal{D}B\, \frac{1}{2\,\mathcal{N}_{\kappa_0}}\left\{{\mathrm{erf}\left[\frac{\kappa_0+B}{\sqrt{2\left(q-m^2\right)}}\right]}+{\mathrm{erf}\left[\frac{\kappa_0-B}{\sqrt{2\left(q-m^2\right)}}\right]}\right\}\nonumber\\ &\quad \times \log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2\left(1-q\right)}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2\left(1-q\right)}}\right]}\right\}+\mathcal{O}\left(1-q\right). \end{align} \tag{ A10 }$$

and the saddle point equations over q and m become

$$\begin{align} \widehat{m}& = \frac{2\alpha m}{\mathcal{N}_{\kappa_0}} \int \mathcal{D}B\left\{\frac{\left(\kappa_0+B\right)e^{\frac{-\left(\kappa_0+B\right)^2}{2\left(q-m^2\right)}}}{\sqrt{2\pi}\left(q-m^2\right)^{3/2}}+\frac{\left(\kappa_0-B\right)e^{\frac{-\left(\kappa_0-B\right)^2}{2\left(q-m^2\right)}}}{\sqrt{2\pi}\left(q-m^2\right)^{3/2}}\right\}\nonumber\\ &\quad \times\log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2\left(1-q\right)}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2\left(1-q\right)}}\right]}\right\}\, +\mathcal{O}\left(\frac{1}{\sqrt{1-q}}\right), \end{align} \tag{ A11 }$$

$$\begin{align} \widehat{q}& = -\frac{\alpha}{\mathcal{N}_{\kappa_0}} \int \mathcal{D}B\left\{\frac{\left(\kappa_0+B\right)e^{\frac{-\left(\kappa_0+B\right)^2}{2\left(q-m^2\right)}}}{\sqrt{2\pi}\left(q-m^2\right)^{3/2}}+\frac{\left(\kappa_0-B\right)e^{\frac{-\left(\kappa_0-B\right)^2}{2\left(q-m^2\right)}}}{\sqrt{2\pi}\left(q-m^2\right)^{3/2}}\right\}\nonumber\\ & \quad \times \log \left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2\left(1-q\right)}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2\left(1-q\right)}}\right]}\right\}\nonumber\\ & \quad +\frac{\alpha}{\mathcal{N}_{\kappa_0}}\int \mathcal{D}B\, \frac{{\mathrm{erf}\left[\frac{\kappa_0+B}{\sqrt{2\left(q-m^2\right)}}\right]}+{\mathrm{erf}\left[\frac{\kappa_0-B}{\sqrt{2\left(q-m^2\right)}}\right]}}{{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2\left(1-q\right)}}\right]}+{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2\left(1-q\right)}}\right]}}\left\{\frac{\left(\kappa+B\right)e^{\frac{-\left(\kappa+B\right)^2}{2\left(1-q\right)}}}{\sqrt{2\pi}\left(1-q\right)^{3/2}}+\frac{\left(\kappa-B\right)e^{\frac{-\left(\kappa-B\right)^2}{2\left(1-q\right)}}}{\sqrt{2\pi}\left(1-q\right)^{3/2}}\right\}\nonumber\\ & \quad +\mathcal{O}\left(\frac{1}{\sqrt{1-q}}\right)\nonumber\\ & = -\frac{\widehat{m}}{2m}+\frac{\alpha}{\mathcal{N}_{\kappa_0}}\int \mathcal{D}B\, \frac{{\mathrm{erf}\left[\frac{\kappa_0+B}{\sqrt{2\left(q-m^2\right)}}\right]}+{\mathrm{erf}\left[\frac{\kappa_0-B}{\sqrt{2(q-m^2)}}\right]}}{{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2(1-q)}}\right]}+{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2(1-q)}}\right]}}\left\{\frac{(\kappa+B)e^{\frac{-(\kappa+B)^2}{2(1-q)}}}{\sqrt{2\pi}(1-q)^{3/2}}+\frac{(\kappa-B)e^{\frac{-(\kappa-B)^2}{2(1-q)}}}{\sqrt{2\pi}(1-q)^{3/2}}\right\}\nonumber\\ & \quad +\mathcal{O}\left(\frac{1}{\sqrt{1-q}}\right). \end{align} \tag{ A12 }$$

Now with $q-m^2 = o(1)$ the energetic contribution becomes

$$\begin{align} \frac{\alpha}{\mathcal{N}_{\kappa_0}}\int \mathcal{D}z\, \mathrm{d}w P_{\kappa_0}\left[w\right]\,\phi^{\kappa}_\mathrm{out}\left[w,z,q,m\right]& = \frac{\alpha}{\mathcal{N}_{\kappa_0}}\int \mathcal{D}B\, \Theta\left(\kappa_0-\vert B\vert\right)\,\log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2\left(1-m^2\right)}}\right]}\right. \nonumber\\& \left. \quad +\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2\left(1-m^2\right)}}\right]}\right\} +\mathcal{O}\left({q-m^2}\right)\nonumber \end{align} \tag{ A13 }$$

and we obtain using the saddle point with respect to m

$$\begin{align} \widehat{m}& = \frac{2\alpha m}{\mathcal{N}_{\kappa_0}}\int \mathcal{D}B\frac{\Theta\left(\kappa_0-\vert B\vert\right)}{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2\left(1-m^2\right)}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2\left(1-m^2\right)}}\right]}} \nonumber\\ &\quad\times \left\{\frac{\left(\kappa+B\right)e^{\frac{-\left(\kappa+B\right)^2}{2\left(1-m^2\right)}}}{\sqrt{2\pi}\left(1-m^2\right)^{3/2}}+\frac{\left(\kappa-B\right)e^{\frac{-\left(\kappa-B\right)^2}{2\left(1-m^2\right)}}}{\sqrt{2\pi}\left(1-m^2\right)^{3/2}}\right\} +\mathcal{O}\left(\frac{q-m^2}{\left(1-m^2\right)^{3/2}}\right)\, . \end{align} \tag{ A14 }$$

If we inject this solution in equation (A12) we obtain $\widehat{q} = o(1/(1-m)^{3/2})$, while equation (A14) implies $\widehat{m} = \mathcal{O}(1/(1-m)^{3/2})$. Finally, setting $\widehat{q}\ll\widehat{m}$ in equations (A4) and (A5) we can derive

$$\begin{align} q& = \int \mathcal{D}z\, \textrm{tanh}^2\left(\widehat{m}+\sqrt{\widehat{q}}z\right) \hspace{-0.1cm}\underset{\widehat{q}\ll\widehat{m}}{ = }\textrm{tanh}^2\left(\widehat{m}\right) \, , \end{align} \tag{ A15 }$$

$$\begin{align} m& = \int \mathcal{D}z\, \textrm{tanh}\left(\widehat{m}+\sqrt{\widehat{q}}z\right) \underset{\widehat{q}\ll\widehat{m}}{ = }\textrm{tanh}\left(\widehat{m}\right)\, . \end{align} \tag{ A16 }$$

In a nutshell, by setting $q-m^2\ll1-m$ and $1-m = o(1)$ we obtain $\widehat{m} = \mathcal{O}(1/(1-m)^{3/2})$ and $\widehat{q} = o(1/(1-m)^{3/2})$. Then, we showed that $\widehat{q}\ll \widehat{m}$ implies $q-m^2\ll1-m$. At this stage we still need to check if closing equations (A14) and (A16) will indeed provide $1-m = o(1)$

In fact, using $q-m^2\ll 1-m$, $1-m = o(1)$, $\widehat{q}\ll\widehat{m}$ along with equation (A16) we have

$$\begin{align} \phi^{\kappa_0,\kappa}_\mathrm{planted}\left[m\right]& = -\frac{1-m}{2}\log{\left[\frac{1-m}{2}\right]}+\frac{\alpha }{\mathcal{N}_{\kappa_0}}\int\! \mathcal{D}B\, \Theta\left(\kappa_0-\vert B\vert\right)\,\nonumber\\ &\quad \times \log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa+B}{\sqrt{2\left(1-m^2\right)}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\kappa-B}{\sqrt{2\left(1-m^2\right)}}\right]}\right\}\nonumber\\ & \quad +o\left(\frac{1-m}{2}\log{\left[\frac{1-m}{2}\right]}\right)\,. \end{align} \tag{ A17 }$$

Then, the planted free energy is a non-trivial function for only a restricted range of parameters κ and m. It happens when the entropic and energetic contributions compete with each other. This leads us to introduce a rescaling of the form $1-{m}^2 = -\alpha \tilde{r} /\log(\alpha)$, $\kappa_0 = {\tilde{\kappa}_0} \sqrt{-\alpha /\log(\alpha)}$ and $\kappa = {\tilde{\kappa}} \sqrt{-\alpha /\log(\alpha)}$. For $\alpha\ll 1$, this rescaling enables to check directly that $1-m = o(1)$ as we have $\alpha/\log(\alpha)\ll1$. In other words, if there exist a solution for the saddle-point equations (A14) and (A16) it will be for $1-m = o(1)$ in the low α limit. If we now rewrite the above planted free energy with this rescaling we obtain

$$\begin{align} \phi^{\tilde{\kappa}_0,\tilde{\kappa}}_\mathrm{planted}\left[\tilde{r}\right]& = \frac{\alpha\tilde{r}}{4}+\frac{\alpha }{\tilde{\mathcal{N}}_{\kappa_0}}\int\! \mathcal{D}B\, \Theta\left(\tilde{\kappa}_0-\vert B\vert\right)\,\log\left\{\frac{1}{2}{\mathrm{erf}\left[\frac{\tilde{\kappa}+B}{\sqrt{2\tilde{r}}}\right]}+\frac{1}{2}{\mathrm{erf}\left[\frac{\tilde{\kappa}-B}{\sqrt{2\tilde{r}}}\right]}\right\}+o\left(\alpha\right)\, , \end{align} \tag{ A18 }$$

$$\begin{align} 1& = \frac{4}{\tilde{\mathcal{N}}_{\kappa_0}}\int \mathcal{D}B\frac{\Theta\left(\tilde{\kappa}_0-\vert B\vert\right)}{{\mathrm{erf}\left[\frac{\tilde{\kappa}+B}{\sqrt{2\tilde{r}}}\right]}+{\mathrm{erf}\left[\frac{\tilde{\kappa}-B}{\sqrt{2\tilde{r}}}\right]}} \left\{\frac{\left(\tilde{\kappa}+B\right)e^{\frac{-\left(\tilde{\kappa}+B\right)^2}{2\tilde{r}}}}{\sqrt{2\pi}\tilde{r}^{3/2}}+\frac{\left(\tilde{\kappa}-B\right)e^{\frac{-\left(\tilde{\kappa}-B\right)^2}{2\tilde{r}}}}{\sqrt{2\pi}\tilde{r}^{3/2}}\right\}+o\left(1\right)\, \end{align} \tag{ A19 }$$

with

$$\begin{align} \tilde{\mathcal{N}}_{\kappa_0} = \int \mathcal{D}B\, \Theta\left(\tilde{\kappa}_0-\vert B\vert\right)\,. \end{align} \tag{ A20 }$$

A last simplification can be made if we set $\tilde{\kappa}_0 = 0$, for example this condition is verified when planting at $\kappa_{\tiny \mathrm{SAT}}$ as $\kappa_{\tiny \mathrm{SAT}}\sqrt{-\log(\alpha)/\alpha}\rightarrow 0$ when α is sent to zero. In this case the integration over B can be dropped and we obtain for the local entropy and its saddle-point equation

$$\begin{align} \phi^{\tilde{\kappa}_0,\tilde{\kappa}}_\mathrm{planted}\left[\tilde{r}\right] = \frac{\tilde{r}}{4}+\alpha\log\left\{ {\mathrm{erf}\left[\frac{\tilde{\kappa}}{\sqrt{2\tilde{r}}}\right]} \right\}+o\left(\alpha\right)\, , \end{align} \tag{ A21 }$$

$$\begin{align} 1 = \frac{4\tilde{\kappa} e^{\frac{-\tilde{\kappa}^2}{2\tilde{r}}}}{\sqrt{2\pi}\tilde{r}^{3/2}{\mathrm{erf}\left[\frac{\tilde{\kappa}}{\sqrt{2\tilde{r}}}\right]}}+o\left(1\right) \, . \end{align} \tag{ A22 }$$

As a final note, and in order to draw a parallel with the 1-RSB computation, we outline here that the saddle-point equations (A14) and (A16) become after setting $m\rightarrow 1$ ($\widehat m\gg1$) and $\kappa_0 = \kappa_{\tiny \mathrm{SAT}}\rightarrow0$

$$\begin{align} m& = 1-2e^{-2\widehat{m}}\, , \end{align} \tag{ A23 }$$

$$\begin{align} \widehat{m}& = \frac{2\alpha\kappa m}{\sqrt{2\pi\left(1-m^2\right)}\left(1-m^2\right){\mathrm{erf}\left[\frac{\kappa}{\sqrt{2\left(1-m^2\right)}}\right]}}e^{\frac{-\kappa^2}{2\left(1-m^2\right)}}\,. \end{align} \tag{ A24 }$$