The $U(1)_{L_\mu-L_\tau}$ breaking phase transition, muon $g-2$, dark matter, collider and gravitational wave

Combining the dark matter and muon $g-2$ anomaly, we study the $U(1)_{L_\mu-L_\tau}$ breaking phase transition, gravitational wave spectra, and the direct detection at the LHC in an extra $U(1)_{L_\mu-L_\tau}$ gauge symmetry extension of the standard model. The new fields includes vector-like leptons ($E_1,~ E_2,~ N$), $U(1)_{L_\mu-L_\tau}$ breaking scalar $S$ and gauge boson $Z'$, as well as the dark matter candidate $X_I$ and its heavy partner $X_R$. A joint explanation of the dark matter relic density and muon $g-2$ anomaly excludes the region where both $min(m_{E_1},m_{E_2},m_N,m_{X_R})$ and $min(m_{Z'},m_S)$ are much larger than $m_{X_I}$. In the parameter space accommodating the DM relic density and muon $g-2$ anomaly, the model can achieve a first order $U(1)_{L_\mu-L_\tau}$ breaking phase transition, whose strength is sensitive to the parameters of Higgs potential. The corresponding gravitational wave spectra can reach the sensitivity of U-DECIGO. In addition, the direct searches at the LHC impose stringent bound on the mass spectra of the vector-like leptons and dark matter.

In this paper, we will combine the muon g −2 anomaly and DM observables, and examine the U (1) Lµ−Lτ breaking phase transition (PT), gravitational wave (GW) signatures, and the exclusion of the LHC direct searches in an extra U (1) Lµ−Lτ gauge symmetry extension of SM.The model was proposed by one of our authors in [10], in which the new particles include vector-like leptons (E 1 , E 2 , N ), U (1) Lµ−Lτ breaking scalar S and gauge boson Z ′ , as well as the dark matter candidate X I and its heavy partner X R .When the PT is first-order, GW could be generated and detected in current and future GW experiments, such as LISA [31], Taiji [32], TianQin [33], Big Bang Observer (BBO) [34], DECi-hertz Interferometer GW Observatory (DECIGO) [34] and Ultimate-DECIGO (U-DECIGO) [35].In addition, the null results of the LHC direct searches could exclude some parameter space achieving a first-order PT (FOPT).

II. THE MODEL
Under the local U (1) Lµ−Lτ , the second (third) generation left-handed lepton doublet and right-handed singlet, L µ , µ R , (L τ , τ R ), are charged with charge 1 (-1).To obtain the mass of U (1) Lµ−Lτ gauge boson Z ′ , a complex singlet scalar S is required to break the the U (1) Lµ−Lτ symmetry.Another complex singlet scalar X with U (1) Lµ−Lτ charge is introduced whose lighter component may be as a candidate of DM.Also we add vector-like lepton doublet fields (E " L,R ) and singlet fields (E ′ L,R ) which mediate the X interactions to the muon lepton, and contribute to the muon g − 2. The quantum numbers of these field under the gauge group SU (3) C × SU (2) L × U (1) Y × U (1) Lµ−Lτ are shown in Table I.
The Lagrangian is written as Where µ is the field strength tensor, D µ is the covariant derivative, and g Z ′ is the gauge coupling constant of the U (1) Lµ−Lτ .V and L Y denote the scalar potential and Yukawa interactions.
The scalar potential V containing the SM Higgs parts can be given by with Where v h = 246 GeV and v S are respectively vacuum expectation values (VeVs) of H and S, and the X field has no VeV.One can determine the mass parameters µ 2 h and µ 2 S of Eq. ( 2) by the potential minimization conditions, After S acquires the VeV, the µ term makes the complex scalar X split into two real scalar fields (X R , X I ), and their masses are After the U (1) Lµ−Lτ is broken, there is still remnant Z 2 symmetry, which guarantee the lightest component X I to be as a candidate of DM.
The λ HS and λ HX terms will lead to the couplings of 125 GeV Higgs (h) and DM.
To suppress the stringent constraints from the DM direct detection and indirect detection experiments, we simply assume that the hX I X I coupling is absent, namely taking λ HX = 0 and λ HS = 0. Thus, the 125 GeV Higgs h is purely from h 1 and extra CP-even Higgs S is purely from h 2 .Their masses are given by After S acquires VeV, the U (1) Lµ−Lτ gauge boson Z ′ obtains a mass, The Yukawa interactions L Y can be written as After the Electroweak symmetry breaking, the vector-like lepton masses are given by By making a bi-unitary transformation with the rotation matrices for the right-handed fields and the left-handed fields, where c 2 L,R + s 2 L,R = 1, we can diagonalize the mass matrix for the vector-like lepton, The E 1 and E 2 are the mass eigenstates of charged vector-like leptons, and the mass of neutral vector-like lepton N is The E 1 and E 2 can mediate X R and X I interactions to muon lepton, and have the couplings to the 125 GeV Higgs, III. DARK MATTER AND MUON g − 2 We fix on m h = 125 GeV, v h = 246 GeV, q x = −1, λ HS = 0, and λ HX = 0, and take , and κ 2 as the free parameters.To maintain the perturbativity, we conservatively choose and take the mixing parameters s L and s R as The mass parameters are scanned over in the following ranges: The mass of neutral vector-like lepton N is a function of m E 1 , m E 2 , s L and s R , and m X I < m N is imposed.We require 0 < g Z ′ /m Z ′ ≤ (550 GeV) −1 to be consistent with the bound of the neutrino trident process [36].
The potential stability in Eq. ( 2) requires the following condition, The one-loop diagrams with the vector-like lepton can give additional corrections to the oblique parameters (S, T, U ), which can be calculated as in Refs.[38][39][40].Taking the recent fit results of Ref. [37], we use the following values of S, T , and U , S = −0.01 ± 0.10, T = 0.03 ± 0.12, U = 0.02 ± 0.11, with the correlation coefficients Also the one-loop diagrams of the charged vector-like leptons E 1 and E 2 can contribute to the h → γγ decay, and the bound of the diphoton signal strength of the 125 GeV Higgs is imposed [37], In the model, the dominant corrections to the muon g − 2 are from the one-loop diagrams with the vector-like leptons (E 1 , E 2 ) and scalar fields (X R and X I ), which are approximately calculated as in Refs.[41][42][43] where the function . The combined average for the muon g − 2 with Fermilab E989 [44] and Brookhaven E821 [45], the difference from the SM prediction becomes which leads to a 5.1σ discrepancy.Whereas the recent lattice calculation [47] and the experiment determination [48] of the hadron vacuum polarization contribution to the muon g − 2 point the value closer to the SM prediction, and hence the tension relaxes to a few sigma level.
If kinematically allowed, the DM pair-annihilation processes includes X I X I → In addition, a small mass splitting between the DM and the other new particles (E 1 , E 2 , N , X R ) can lead to coannihilation.The Planck collaboration reported the relic density of cold DM in the universe, Ω c h 2 = 0.1198 ± 0.0015 [49], and the theoretical prediction of the model is calculated by micrOMEGAs-5.2.13 [50].
After imposing the constraints of theory, oblique parameters, and the diphoton signal data of the 125 GeV Higgs, we project the samples accommodating the DM relic density and muon g − 2 anomaly within 2σ ranges in Fig. 1.From Fig. 1 we find that the correct DM relic density can be obtained for most of the parameter region of The DM relic density is mainly produced via the DM pair-annihilation processes However, once the explanation of muon g − 2 anomaly is simultaneously required, most of region of The PT between the unbroken and the broken phase can proceed in basically two different ways.In a FOPT, at the critical temperature T C , the two degenerate minima will be at different points in field space, typically with a potential barrier in between.For a secondorder (cross-over) transition, the broken and symmetric minimum are not degenerate until they are at the same point in field space.In this paper we focus on a first-order U (1) Lµ−Lτ breaking PT.
A. The thermal effective potential In order to examine PT, we first take h 1 , h 2 , and X r as the field configurations, and obtain the field dependent masses of the scalars (h, S, X R , X I ), the Goldstone boson (G, ω, G ± ), the gauge boson, and fermions.The field dependent masses of scalars are given m2 h,S,X R = eigenvalues( M 2 P ) , m2 with The field dependent masses of gauge boson are The field dependent masses of vector-like lepton are with For the quarks of SM, we only consider the top quark, with y t = m t /v h .
In order to examine the U (1) Lµ−Lτ breaking PT, we need to study the thermal effective potential V ef f in terms of the classical fields (h 1 , h 2 , X r ), which is composed of four parts: V 0 is the tree-level potential, V CW is the Coleman-Weinberg (CW) potential [51], V CT is the counter term, V T is the thermal correction [52], and V ring is the resummed daisy corrections [53,54].In this paper, we calculate V ef f in the Landau gauge.
The tree-level potential V 0 in terms of their classical fields (h 1 , h 2 , X r ) from the Eq. ( 2), The CW potential in the MS scheme at 1-loop level has the form [51]: where i = h, S, X R , X I , G, ω, G ± , W ± , Z, Z ′ , t, E 1 , E 2 , µ, and s i is the spin of particle i. Q is a renormalization scale, and we take Q = m S .The constants C i = 3 2 for scalars or fermions and C i = 5  6 for gauge bosons.n i is the number of degree of freedom, With V CW being included in the potential, the minimization conditions of scalar potential and the CP-even mass matrix will be shifted slightly.To maintain these relations, the counter terms V ct should be added, The relevant coefficients are determined by which are evaluated at the EW minimum of {h 1 = v h , h 2 = v S , X r = 0} on both sides.As a result, the VeVs of h 1 , h 2 , X r and the CP-even mass matrix will not be shifted.We check that the following relations hold true For the left seven equations, there are eight parameters to be fixed, so that one renormalization constant is left for determination.We choose to use δm 2 1 , δm 2 2 , δm 2 X , δλ H , δλ S , δλ X , δλ SX , and set δµ = 0.It is a well-known problem that the second derivative of the CW potential in the vacuum suffers from logarithmic divergences originating from the vanishing Goldstone masses.In order to avoid the problems with infrared divergent Goldstone contributions, we simply remove the Goldstone corrections in the renormalization conditions following the approaches of [55,56].This is simply a change of renormalization conditions and the shift it causes in the potential shape is negligible.
The thermal contributions V T to the potential can be written as [52] V , and the functions J B,F are Finally, the thermal corrections with resumed ring diagrams are given [53,54].
) for the CP-even and CP-odd scalar fields are the eigenvalues of the full mass matrix, where Π X (X = P, A) are given by The thermal Debye mass of The Debye masses of longitudinal gauge bosons are In Fig. 2, we display the scatter plots achieving FOFP and accommodating the DM relic density, muon g − 2 anomaly, and various constraints mentioned previously.We observe that the strength of FOPT is sensitive to the parameter λ S .As λ S decreases, the the critical temperature T C tends to decrease, and the strength of FOPT tends to increase.We pick out two benchmark points (BPs) to show how the PT happens.The input parameters of BP1 and BP2 are displayed in Table II, and their phase histories are exhibited in Fig. 3 on field configurations versus temperature plane.For the BP1 and BP2, the potential minima at any temperatures always locate at ⟨X r ⟩=0.As the universe cools, a FOPT takes place during which the h 2 acquires a nonzero VeV and the other two fields remain zero.As the temperature continues to decrease, the h 1 starts to develop a nonzero VeV during the second PT which is second-order.Finally, the observed vacuum is obtained at the present temperature.which is implemented in the MadAnalysis5 [58][59][60] with assuming 95% confidence level for the exclusion limit.The simulations for the samples are performed by MG5 aMC-3.3.2 [61] with PYTHIA8 [62] and Delphes-3.2.0 [63].
In Fig. 4, we employ the ATLAS analysis of 2ℓ + E miss T at the LHC to restrict the parameter space which has been satisfied by the DM relic density, muon g − 2, FOPT, and various constraints discussed above.Fig. 4 indicates that the DM mass is allowed to be as low as 100 GeV when the value of min(m E 1 , m E 2 ) − m X I is small.This is because the muon from the vector-like lepton decay has soft energy, and its detection efficiency is decreased at the LHC.As the DM mass increase, the value of min(m E 1 , m E 2 ) − m X I increases, and the lightest charged vector-like lepton is allowed to have a more large mass.

B. Gravitational wave signature
Stochastic GWs are produced during a FOPT via bubble collision, sound waves in the plasma and the magneto-hydrodynamics turbulence.Since most of the PT energy is pumped into the surrounding fluid shells, making sound waves the dominant contribution to GWs, we will focus on the GW spectrum from the sound waves in the plasma.
The sound wave spectra can be expressed as functions of two FOPT parameters β and α, Where H n is the Hubble parameter at the nucleation temperature T n , and g * is the effective number of relativistic degrees of freedom.β characterizes roughly the inverse time duration of the strong first-order PT, and α is defined as the vacuum energy released from the PT normalized by the total radiation energy density ρ R at T n .
The GW spectrum from the sound waves can be expressed by [64] Ω where v w is the wall velocity, and we take v w = c s = 1/3 with c s being the sound velocity.
f sw is the present peak frequency of the spectrum, Hz . ( The κ v is the fraction of latent heat transformed into the kinetic energy of the fluid [65], κ v ≃ α 2/5 0.017 + (0.997 + α) 2/5 .
(53) The suppression factor of Eq. ( 51) [66] Υ(τ sw ) = 1 appears due to the finite lifetime τ sw of the sound waves [67,68], We calculate GW spectra for thousands of parameter points accommodating the muon g − 2, DM relic density, and the exclusion limits of the LHC direct searches, and find that all the peak strengths are below the sensitivity curve of BBO.About 10 percent of the survived points can generate U-DECIGO sensitive gravitational wave, including BP1 and BP2.These points favor a small λ S for which the strength of FOPT tends to have a large value.The GW spectra of BP1 and BP2 are shown along with expected sensitivities of various future interferometer experiments in Fig. 5.The lowest peak frequency is 0.003 HZ from BP1, and the highest peak frequency is 0.2 HZ from BP2.

VI. CONCLUSION
We study an extra U (1) Lµ−Lτ gauge symmetry extension of the standard model by considering the dark matter, muon g −2 anomaly, the U (1) Lµ−Lτ breaking PT, GW spectra, and the bound from the direct detection at the LHC.We obtained the following observations: (i) A joint explanation of the dark matter relic density and muon g − 2 anomaly rules out the region where both min(m E 1 , m E 2 , m N , m X R ) and min(m Z ′ , m S ) are much larger than m X I .
(ii) A first-order U (1) Lµ−Lτ breaking PT can be achieved in the parameter space explaining the DM relic density and muon g − 2 anomaly simultaneously, and the corresponding gravitational wave spectra can reach the sensitivity of U-DECIGO.(iii) The mass spectra of the vector-like leptons and dark matter are stringently restricted by the direct searches at the LHC.

FIG. 1 :
FIG. 1: The surviving samples explaining the DM relic density and muon g − 2 anomaly while satisfying the constraints from theory, oblique parameter, and 125 GeV Higgs diphoton signal.The circles and squares are excluded and allowed by the DM relic data.
ruled out.This is because the muon g − 2 anomaly favors small interactions between the vector-like leptons and muon mediated by X I , which leads to X I X I → µ + µ − process to fail to produce the correct DM relic density.The X I does not couple to the SM quark, and its couplings to the muon lepton and vector-like leptons are restricted by the muon g − 2 anomaly.Therefore, the model can accommodate the bound from the DM direct detection naturally.IV.U (1) Lµ−Lτ BREAKING PHASE TRANSITION At high temperatures, the global minimum of the finite-temperature effective potential is at the origin, i.e.SU (2) L × U (1) Y × U (1) Lµ−Lτ is unbroken.When the temperature drops, the potential changes and at some point develops a minimum at non-vanishing field values.

FIG. 4 :
FIG. 4: All the samples achieve FOFP and accommodate the DM relic density, muon g−2 anomaly, and various constraints mentioned previously.The circles and squares are excluded and allowed by the direct searches for 2ℓ + E miss T

TABLE I :
The U (1) Lµ−Lτ quantum numbers of the new fields.

TABLE II :
Input parameters for the BP1 and BP2.