We present new imaging of the surface of Pluto and Charon obtained during 2002-2003 with the Hubble Space Telescope (HST) Advanced Camera for Surveys (ACS) instrument. Using these data, we construct two-color albedo maps for the surfaces of both Pluto and Charon. Similar mapping techniques are used to re-process HST/Faint Object Camera (FOC) images taken in 1994. The FOC data provide information in the ultraviolet and blue wavelengths that show a marked trend of UV-bright material toward the sunlit pole. The ACS data are taken at two optical wavelengths and show widespread albedo and color variegation on the surface of Pluto and hint at a latitudinal albedo trend on Charon. The ACS data also provide evidence for a decreasing albedo for Pluto at blue (435 nm) wavelengths, while the green (555 nm) data are consistent with a static surface over the one-year period of data collection. We use the two maps to synthesize a true visual color map of Pluto's surface and investigate trends in color. The mid- to high-latitude region on the sunlit pole is, on average, more neutral in color and generally higher albedo than the rest of the surface. Brighter surfaces also tend to be more neutral in color and show minimal color variations. The darker regions show considerable color diversity arguing that there must be a range of compositional units in the dark regions. Color variations are weak when sorted by longitude. These data are also used to constrain astrometric corrections that enable more accurate orbit fitting, both for the heliocentric orbit of the barycenter and the orbit of Pluto and Charon about their barycenter.