Phase diagram of interacting Fermi gas in spin–orbit coupled square lattices

The spin–orbit (SO) coupled optical lattices have attracted considerable interest. In this paper, we investigate the phase diagram of the interacting Fermi gas with Rashba-type spin–orbit coupling (SOC) on a square optical lattice. The phase diagram is investigated in a wide range of atomic interactions and SOC strength within the framework of the cluster dynamical mean-field theory (CDMFT). We show that the interplay between the atomic interactions and SOC results in a rich phase diagram. In the deep Mott insulator regime, the SOC can induce diverse spin ordered phases. Whereas near the metal–insulator transition (MIT), the SOC tends to destroy the conventional antiferromagnetic fluctuations, giving rise to distinctive features of the MIT. Furthermore, the strong fluctuations arising from SOC may destroy the magnetic orders and trigger an order to disorder transition in close proximity of the MIT.

When competing with strong atomic interactions, SOC introduces additional degrees of quantum fluctuation, giving rise to remarkable many-body ground states. For example, the study of the superfluid to Mott insulator transition in the Bose-Hubbard model with synthetic SOC has demonstrated that, Rashba-type SOC can induce intriguing magnetism in the deep Mott regime [29][30][31][32][33][34][35][36]39], as well as an exotic superfluid phase with magnetic textures near the Mott transition [30,36]. Despite this, the essential properties of the metal-insulator transition (MIT) of interacting fermion systems have been demonstrated less often.
The MIT lies at the heart of many-body physics, which has achieved great advances in optical lattices [40,41]. In this paper, we investigate the phase diagram of the interacting Fermi gas with Rashba-type SOC on a square optical lattice. Such a system can be described by the spin-orbit (SO) coupled fermionic Hubbard model, see equation (1). We show that the SOC has important implications on the properties of MIT and phase diagram of this system. In the deep Mott insulator regime, the SOC can induce diverse spin ordered phases. Whereas near the MIT, the SOC tends to destroy the conventional antiferromagnetic fluctuations, resulting in the distinctive features of the MIT. Further we found that, though the spin configurations in the deep Mott regime can be captured by an effective spin model, it fails near the MIT. The strong fluctuations arising from SOC may destroy the magnetic orders and trigger an order to disorder transition. These issues are investigated within the unified theoretical framework of cluster dynamical mean-field theory (CDMFT) [42][43][44].
The paper is organized as follows. In the following section we introduce the definition of the SO coupled fermionic Hubbard model. Subsequently, in section 3, we present the methodology of the CDMFT in the presence of the SOC. In section 4, we analyze in detail the MIT and spin configurations in the entire phase diagram. Finally in section 5, we discuss some experimental related issues and present conclusions.

The model
We consider a system of two-component Fermi gas moving in an optical square lattice. In the tight binding approximation, the Hamiltonian reads where t is the overall tunneling matrix element and c iσ (c i † σ ) denotes fermionic annihilation (creation) operator for a fermion of spin , σ = ↑ ↓ on the lattice site i. The first term describes the nearest-neighboring hoppings with the hopping matrices given by denotes a non-Abelian gauge field which can be generated by the laser-induced spin-flipped tunneling [21,22]. In this paper we set β α = − , which implies that the SOC is of Rashba type [29][30][31][32][33][34][35][36][37][38][39]. In this case, the spin-conserved hopping term is proportional to t cos α, and the spin-flipped term is in proportion to t sin α. The value U is the on-site atomic repulsion and μ is the chemical potential. The particle number operator is n n nˆî

Calculation method
We study the physical properties of Hamiltonian equation (1) with the CDMFT, using the Hirsch-Fye Quantum Monte Carlo algorithm as the impurity solver [45,46]. The CDMFT incorporates spatial correlations and has been shown to be successful in the study of MIT and magnetic orders. Below we describe briefly how to determine the energy gap and magnetization in the framework of CDMFT.
In the presence of SOC, we can map the square lattice onto two sets of sublattices for spin up (down) respectively, as shown in figure 1(a). The 2 × 2 clusters are embedded in a self-consistent medium with the Weiss function of the cluster represented by g  term in equation (1), g ↑↓ and g ↓↑ are generally nonzero. Here, the Weiss function is determined by the cluster self-energy (i ) Σ ω via the coarse-grained Dyson equation [42,43] where t k ( )is the Fourier-transformed hopping matrix with wave vector k in the cluster reduced Brillouin zone of the superlattice, and (i ) is the self-energy of the cluster. Then, we introduce two- , and define the cluster Green's function as ω is determined, the impurity solver can be used to compute the cluster Green's function G (i ) ω . Eventually, by using the Dyson equation The energy gap Δ can be derived from the local density of states (LDOS). By implementing the analytic extension of the imaginary time cluster Green's function G (i ) ω via the maximum entropy method [47], we have Then in the spectrum of LDOS, we can obtain the energy gap Δ by the energy width of zero density of states. The spin phases in the Mott insulating regime can be characterized by the spin structure factor |with q the 2D wave vector. Here, S Ŝ i i = 〈 〉 denotes local magnetic order parameter on site i of the cluster, with three components given by To define the magnetization of spin ordered phases, we can rotate the local magnetic order parameter S i on each cluster site to a global coordinate system: where 1 i ϵ = ± is for sites belonging to sublattice A(B) respectively. This general definition of magnetization is also applied to other spin phases throughout this paper. In what follows, we shall investigate the phase diagram on the half-filled square lattice for a wide range of atomic interactions and SOC strength.

Results
Our main results are summarized in figure 2. First, we examine the case with 0 α = in Hamiltonian equation (1), which recovers the Hubbard model on a conventional square lattice. In figure 3(a), we plot the scaling analysis of the critical interaction strength U t c for the MIT. We show that, for larger clusters, the interaction strength U t c at zero temperature would approach to much smaller values. This is because half-filled square lattice has a perfect nesting Fermi surface, which strongly enhances the antiferromagnetic (AF) fluctuations and essentially drives system into an insulator with infinitesimal atomic interaction [48]. On the other hand, we can restrict the CDMFT calculations to be in the paramagnetic phase, a much larger U t 6.05 c ≃ will then be obtained, agreeing well with the result from G. Kotliar'(s) group [49]. Below we can see, once the SOC is away from 0 α = , the perfect nesting Fermi surface is destroyed and the critical value of U t c for the MIT becomes finite. Now, we turn to the effects of the SOC on the MIT. We concentrate on the region of [0, 2] α π ∈ , and the relevant physical results are not affected in other regions. First, the single-particle spectrum is split into two bands (see figure 1(b)), with the zero energy Fermi surface possessing a particle and hole Fermi-pocket around the center and corner of the Brillouin zone. The corresponding density of states (DOS) for non-interacting fermions is shown in figure 1(c), where the zero energy DOS is suppressed and the bandwidth shrinks gradually with increasing α. The suppressed zero energy DOS reduces the correlation effects on Fermi surface and hence enhances U t c of the MIT, whereas the shrinking bandwidth tends to stabilize the Mott insulator at a smaller U t c . The two effects compete with each other, leading to the drastic changes of the MIT boundary in the phase diagram. In figure 2 we show that, away from 0 α = , the value of U t c rapidly increases due to the suppression of the conventional AF fluctuations on the square lattices. Subsequently, the MIT exhibits a nonmonotonic behavior as a function of α. Specially at 2 α π = , where relativistic Dirac fermions emerge in the metallic phase [28], the MIT occurs at a finite atomic interaction with U t 4.1 c = . In order to feature the MIT in the presence of SOC, in figure 3(b) we plot the evolution of DOS at different atomic interactions for 1.0 α = . We show that, compared to the 0 α = case, the zero-energy spectral peak in the metallic phase (red line) is largely suppressed by the SOC. Simultaneously, two satellite peaks appear corresponding to the Van Hove singularity shown in figure 1(c). Then, the zero energy peaks are gradually reduced and a gap opens with the increase of atomic interactions.
In figure 4(a), we plot the corresponding single-particle gap Δ and magnetization m as functions of U t for 1.0 α = . The insulating phase characterized by a non-zero Δ is accompanied by a finite m simultaneously, indicating that a magnetic order arises. In figure 2, we determine the specific magnetic phases by identifying the spin configurations on 2 × 2 cluster. We show that as α increases the system transits from the xy-antiferromagnet (xy-AFM) to the spiral, the stripe, and the spin vortex (SV) phases. The structure factor of the xy-AFM has a peak at q ( , ) π π = , the stripe phase at q (0, ) π = , and the SV phase at q ( , 0) π = and q (0, ) π = . Between the xy-AFM and the stripe phases, spiral phases where the spins spiral in the z-q plane with q q ( , ) π = the in-plane wave vector may appear. However, the spiral phase is hard to be explicitly identified on 2 × 2 cluster. To  overcome this difficulty, we explore on a larger 4 × 4 cluster, and a spiral-4 phase with spatial period of 4 × 2 lattice sites is clearly identified in figure 5. Qualitatively, the magnetic phase transitions can be understood from an effective spin model. For U t 1 ≫ , we can apply the second order perturbation theory to the system and obtain ( ) Here, the first term is the conventional Heisenberg coupling and the second term denotes the so-called Dzyaloshinskii-Moriya (DM)-type super-exchange [50,51]. The induced DM-type term favors spiral type order and competes with the Heisenberg coupling, tending to form diverse spin phases.
Note that, the above effective spin model (8) works only for the deep Mott regime with the atomic kinetic energies being treated perturbatively. In close proximity to the more interested Mott transition, such a perturbative description breaks down and the strong fluctuations arising from the SOC may destroy the magnetic orders and trigger an order to disorder transition. To address this issue, one needs to implement a nonperturbative method such as the CDMFT to explore in detail the phase diagram as in figure 2. We find that, despite the robustness of the diverse spin phases in the Mott insulating regime with up to modest values of the SOC, a quantum nonmagnetic insulating (NMI) phase can emerge in the vicinity of the MIT for The NMI phase is characterized in figure 4(b), where the single-particle gap Δ and magnetization m occur for different atomic interactions U c 1 and U c 2 . Specifically in the intermediate region U U U c c 1 2 ⩽ ⩽ , the system enters into an insulating state but with no long-range magnetic order within the current CDMFT methodology. This suggests a tricritical point, where by increasing α depending on U t , one can enter either SV phase in Mott insulating regime or NMI phase from the metal phase. We note that for α being close to 2 π , the SOC can induce relativistic Dirac cones in the metallic phase with the DOS being almost suppressed at zero energy, which may destroy the spin ordered phases near the boundary of MIT. Whereas for c α α < , the metal phase transits directly into the SV phase.  The emergence of the NMI phase is further confirmed on a 4 × 4 cluster. The larger size of cluster incorporate more spin correlations and thus, a better description of the atomic correlations and SOC induced fluctuations can be expected. Figure 6 plots the phase diagram for α being close to 2 π . We found that, in the 4 × 4 cluster, the regime of the NMI phase is slightly expanded, which seems to indicate that the NMI is robust in this system. We further show, in the inset of figure 6, the temperature dependence of the NMI phase. The interval between the metallic and SV phases enlarge with decreasing temperature. This demonstrates that the NMI phase is more stable at low temperatures by the suppression of thermal fluctuations.
The NMI phase breaks neither spin nor lattice symmetry, suggesting a potential spin-liquid (SL) ground state. Such a fundamental state was first proposed by Anderson [52] and has long been sought in the frustrated spin systems [53]. Recently, interacting fermion models have attracted wide attentions [54][55][56][57], and it was reported that a SL state can be identified on honeycomb lattice between the semimetal and the AF insulator with t U t 3.5 4.3 ⩽ ⩽ [54]. Despite this, its presence has been challenged since the interval of the SL phase is small, and may vanish under the finite-size scaling [58][59][60]. The latest results using large-scale quantum Monte Carlo (QMC) showed that, if the SL state exists, the possible regime reduces substantially to a small interval t U t 3. 8 3.9 ⩽ ⩽ [58]. Similar situations have been encountered for the staggered-flux model on a square lattice [61,62]. Here, the essential feature characterizing the present system is the considerably large space of parameters, where the NMI phase emerges. This is in sharp contrast to the limited phase space ( t U t 3. 4 3.9 ⩽ ⩽ ) obtained in the interacting fermions on honeycomb lattices [63]. In particular, the predicted NMI phase occurs until c α α > , showing that it is a strong field effect of the SOC. The absence of magnetic orders in the NMI phase implies strong short-range spin correlation. However, it may decay as a power-law or exponentially. To explore this issue, we calculate the staggered spin-spin correlation function x x y y z z r r r r 0 0 0 as shown in figure 7, where the spin correlation functions are fitted to a power-law as C r r ( ) 1 ∼ γ . In the NMI phase, we find that the exponent α is less than 2 with 1.6 α ∼ in our simulations. Whereas in the deep Mott insulating regime where spin is ordered, α becomes much smaller. Therefore, the NMI phase seems to suggest a candidate of algebraic SL.
Here we should mention that, although we present strong evidence that a NMI phase occurs in a considerably large regime of parameters, a further systematic investigation of the charge gap and the magnetization as a function of the cluster size is certainly needed before a definite conclusion can be drawn. A large-scale QMC calculation with the SOC would be implemented in future studies.

Discussion and conclusions
Finally, we discuss the experiment related issues. The above phenomena of the intriguing MIT and matter states can be investigated in experiments. In optical lattices, the Mott insulating phase can be detected by site-resolved imaging of single atoms [64][65][66][67][68], and the spin textures occurring in the Mott phase can be observed via in situ microscopy [69] or through spin-resolved time-of-flight measurements [70]. On the other hand, the spin correlation can be measured by the spin structure factors in optical Bragg scattering [71,72], which may present the signatures of the spin ordered phases and the power-law scaling of the NMI phase. In addition, an extremely low temperature has been recently realized to approach the superexchange energy scales [73,74].
In summary, we have investigated the half-filled Fermi gas with Rashba-type SOC on a square lattice. We show that this system displays a rich phase diagram. The interplay between the atomic interactions and the SOC results in distinctive features of the MIT. In the deep Mott regime, the SOC can induce diverse spin ordered phases. Near the Mott transition, we find the strong field effect of the SOC can drive an order to disorder transition, and a quantum NMI phase emerges. These properties can be explored in experiments.