We present deep U-, V-, and I-band images of the "ultracompact" blue dwarf galaxy POX 186 obtained with the Planetary Camera 2 of the Hubble Space Telescope. We have also obtained a near-ultraviolet spectrum of the object with the Space Telescope Imaging Spectrograph and combine this with a new ground-based optical spectrum. The images confirm the galaxy to be extremely small, with a maximum extent of only 300 pc, a luminosity of ~10^-4L*, and an estimated mass of ~10⁷ M_☉. Its morphology is highly asymmetric, with a tail of material on its western side that may be tidal in origin. The U-band image shows this tail to be part of a stream of material in which stars have recently formed. Most of the star formation in the galaxy is, however, concentrated in a central, compact (d ~ 10-15 pc) star cluster. We estimate this cluster to have a total mass of ~10⁵ M_☉, to be forming stars at a rate of less than 0.05 yr^-1, and to have a maximum age of a few million years. The outer regions of the galaxy are significantly redder than the cluster, with V-I colors consistent with a population dominated by K and M stars. From our analysis of the optical spectrum we find the galaxy to have a metallicity Z 0.06 Z_☉ and to contain a significant amount of internal dust [E(B-V) 0.28]; both values agree with previous estimates. While these results rule out earlier speculation that POX 186 is a protogalaxy, its morphology, mass, and active star formation suggest that it represents a recent (within ~10⁸ yr) collision between two clumps of stars of subgalactic size (~100 pc). POX 186 may thus be a very small dwarf galaxy that, dynamically speaking, is still in the process of formation. This interpretation is supported by the fact that it resides in a void, so its morphology cannot be explained as the result of an encounter with a more massive galaxy. Clumps of stars this small may represent the building blocks required by hierarchical models of galaxy formation, and these results also support the recent "downsizing" picture of galaxy formation in which the least massive objects are the last to form.