We describe a moderate-resolution Far Ultraviolet Spectroscopic Explorer (FUSE) survey of H₂ along 70 sight lines to the Small and Large Magellanic Clouds, using hot stars as background sources. FUSE spectra of 67% of observed Magellanic Cloud sources (52% of LMC and 92% of SMC) exhibit absorption lines from the H₂ Lyman and Werner bands between 912 and 1120 Å. Our survey is sensitive to N(H₂) ≥ 10¹⁴ cm^-2; the highest column densities are log N(H₂) = 19.9 in the LMC and 20.6 in the SMC. We find reduced H₂ abundances in the Magellanic Clouds relative to the Milky Way, with average molecular fractions = 0.010 for the SMC and = 0.012 for the LMC, compared with = 0.095 for the Galactic disk over a similar range of reddening. The dominant uncertainty in this measurement results from the systematic differences between 21 cm radio emission and Lyα in pencil beam sight lines as measures of N(H I). These results imply that the diffuse H₂ masses of the LMC and SMC are 8 × 10⁶ and 2 × 10⁶ M, respectively, 2% and 0.5% of the H I masses derived from 21 cm emission measurements. The LMC and SMC abundance patterns can be reproduced in ensembles of model clouds with a reduced H₂ formation rate coefficient, R ~ 3 × 10^-18 cm³ s^-1, and incident radiation fields ranging from 10-100 times the Galactic mean value. We find that these high-radiation, low formation rate models can also explain the enhanced N(4)/N(2) and N(5)/N(3) rotational excitation ratios in the Clouds. We use H₂ column densities in low rotational states (J = 0 and 1) to derive kinetic and/or rotational temperatures of diffuse interstellar gas, and we find that the distribution of rotational temperatures is similar to Galactic gas, with T₀₁ = 82 ± 21 K for clouds with N(H₂) ≥ 10^16.5 cm^-2. There is only a weak correlation between detected H₂ and far-infrared fluxes as determined by IRAS, perhaps as a result of differences in the survey techniques. We find that the surface density of H₂ probed by our pencil beam sight lines is far lower than that predicted from the surface brightness of dust in IRAS maps. We discuss the implications of this work for theories of star formation in low-metallicity environments.