Dephasing processes for low-lying phonons of Ag⁺ and Tl⁺ β-alumina were observed in the time domain using the femtosecond pulse laser in the temperature range of 15–350 K. The dephasing dynamics associated with the time evolution of the vibrational coherence state can be directly identified with the phonon decay in the femtosecond transient. In this study, the temperature dependence of the dephasing property was decomposed into three terms in order to understand the correlation between vibration and diffusion: (i) a static structural disorder, (ii) an anharmonic coupling and (iii) an ionic diffusion. The dephasing property of the low-lying phonon in Ag β-alumina as a superionic conductor was compared with that of the isomorphous Tl β-alumina, whose ionic conductivity was about 10³ times less than Ag β-alumina. The magnitude of static disorder shows a large value ( meV) in Ag β-alumina, which is twice that of Tl β-alumina ( meV) with the same structure and the same number of excess cations by nonstoichiometry. The coefficient of phonon lifetimes originating from the cubic anharmonicity of a potential well is about five times larger in Ag β-alumina than Tl β-alumina. A strongly temperature dependent decay component is only seen in the superionic conductor Ag β-alumina above 200 K, and is not observed in Tl β-alumina. This is attributed to the phonon dephasing caused by the jump motion of the Ag⁺ ions. The estimated activation energy and pre-exponential factor are E_a = 71 meV and  meV in Ag β-alumina, respectively, for the correlation time τ_c = τ₀exp(E_a/k_BT). The value of  meV, which can be regarded as an attempt frequency for the jump, coincides with the low-lying phonon frequency  meV, but the E_a = 71 meV is different from that of the dc conductivity (E_a = 173 meV). These results, which are strongly coupled with the elementary excitation from oscillation to ionic diffusion in the picosecond time domain, would be of prime importance for the superionic conduction mechanism.