The dust extinction of gamma-ray burst (GRB) host galaxies, which contains important clues to the nature of GRB progenitors and is crucial for dereddening, is still poorly known. Here we propose a straightforward method for determining the extinction of GRB host galaxies by comparing the observed optical spectra to the intrinsic ones extrapolated from the X-ray spectra. The rationale for this method comes from the standard fireball model: if the optical flux decay index equals that of the X-ray flux, then there is no break frequency between the optical and X-ray bands, and therefore we can derive the intrinsic optical flux from the X-ray spectra. We apply this method to three GRBs for which the optical and X-ray fluxes have the same decay indices and to another GRB with an inferred cooling break frequency, and we obtain the rest-frame extinction curves of their host galaxies. The derived extinction curves are "gray" and do not resemble any extinction curves of local galaxies (e.g., the Milky Way, the Small or Large Magellanic Clouds, or nearby starburst galaxies). The amount of extinction is rather large (with a visual extinction of A_V ~ 1.6-3.4 mag). We model the derived extinction curves in terms of the silicate-graphite interstellar grain model. As is expected from the gray nature of the derived extinction curve, the dust size distribution is skewed to large grains. We determine, for the first time, the local dust-to-gas ratios of GRB host galaxies by using the model-derived dust parameters and the hydrogen column densities determined from X-ray absorptions.