We present three-band Hubble Space Telescope imaging of three nearby (z ≤ 0.1) compact symmetric objects: 4C 31.04, 1946+708, and 1146+596. These objects were chosen on the basis of proximity to Earth as well as H I 21 cm line absorption. The inner H-band isophotes of these galaxies are well fitted by Nuker models, typical of nearby elliptical galaxies. Each shows a significant flattening in the isophotal profile at radii ~05, as well as significant variations in ellipticity and P.A. However, as previous authors have noted, neither is uncommon for elliptical galaxies. All three objects show modest departures from Nuker law models at radii of 1–5 h kpc. Each galaxy shows large well-distributed dust features, which are somewhat concentrated in the nuclear regions in features that resemble disks or tori. We find that the amount of dust in these galaxies is about 10 times higher than normal for elliptical galaxies and radio galaxy hosts. The major axes of the nuclear dust disks tend to be oriented roughly perpendicular to the radio axis. One galaxy, 4C 31.04, exhibits bright nuclear regions well aligned with the radio axis, while another, 1146+596, shows a significant near-IR excess resembling a stellar bar along its dust disk. The combination of outwardly normal isophotal profiles with significant variations in P.A. and ellipticity is consistent with the host galaxies being relatively recent merger remnants and the mergers having occurred 10⁸ yr ago. Such a merger could have "triggered" the onset of the current active phase seen in these objects, but our data require a significant time delay between the merger event and the onset of nuclear activity. However, these data are also consistent with the hypothesis that the onset of nuclear activity in radio galaxies is due to relatively minor "feeding" events and/or the formation of "bars within bars," events which would disturb the internal kinematics only slightly.