The detection of planets around very low-mass stars with the radial velocity (RV) method is hampered by the fact that these stars are very faint at optical wavelengths where the most high-precision spectrometers operate. We investigate the precision that can be achieved in RV measurements of low mass stars in the near-infrared (NIR) Y-, J-, and H-bands, and we compare it to the precision achievable in the optical assuming comparable telescope and instrument efficiencies. For early-M stars, RV measurements in the NIR offer no or only marginal advantage in comparison with optical measurements. Although they emit more flux in the NIR, the richness of spectral features in the optical outweighs the flux difference. We find that NIR measurement can be as precise as optical measurements in stars of spectral type ~M4, and from there the NIR gains in precision toward cooler objects. We studied potential calibration strategies in the NIR finding that a stable spectrograph with a ThAr calibration can offer enough wavelength stability for m s^–1 precision. Furthermore, we simulate the wavelength-dependent influence of activity (cool spots) on RV measurements from optical to NIR wavelengths. Our spot simulations reveal that the RV jitter does not decrease as dramatically toward longer wavelengths as often thought. The jitter strongly depends on the details of the spots, i.e., on spot temperature and the spectral appearance of the spot. At low temperature contrast (~200 K), the jitter shows a decrease toward the NIR up to a factor of 10, but it decreases substantially less for larger temperature contrasts. Forthcoming NIR spectrographs will allow the search for planets with a particular advantage in mid- and late-M stars. Activity will remain an issue, but simultaneous observations at optical and NIR wavelengths can provide strong constraints on spot properties in active stars.