Merging compact binaries are the one source of gravitational radiation so far identified. Because short-period systems that will merge in less than a Hubble time have already been observed as binary pulsars, they are important both as gravitational wave sources for observatories such as LIGO, but also as progenitors for short gamma-ray bursts (SGRBs). The fact that these systems must have large systemic velocities implies that by the time they merge, they will be far from their formation site. The locations of merging sites depend sensitively on the gravitational potential of the galaxy host, which until now has been assumed to be static. Here we refine such calculations to incorporate the temporal evolution of the host's gravitational potential as well as that of its nearby neighbors using cosmological simulations of structure formation. This results in merger site distributions that are more diffusively distributed with respect to their putative hosts, with locations extending out to distances of a few Mpc for lighter halos. The degree of mixing between neighboring compact binary populations computed in this way is severely enhanced in environments with a high number density of galaxies. We find that SGRB redshift estimates based solely on the nearest galaxy in projection can be very inaccurate, if progenitor systems inhere large systematic kicks at birth.