Recent Spitzer observations have revealed a substantial population of z ~ 2 ultraluminous infrared galaxies (ULIRGs) with deep silicate absorption (τ_9.7 > 1). This paper reports a 20 cm radio study of such a sample to elucidate their physical nature. We discover that a substantial fraction (40%) of deep silicate absorption ULIRGs at z ~ 2 are moderately radio-loud, with L_1.4GHz = 10²⁵-10²⁶ W Hz^-1. This is in strong contrast with z 1 radio galaxies and radio-loud quasars where none of the sources with available IRS spectra have τ_9.7 > 1. In addition, we observe radio jets in two of our sources, with one having a double lobe structure ~200 kpc in extent and the other showing a one-sided jet extending ~90 kpc from the nucleus. The likely high inclination of the latter, coupled with its deep silicate absorption, implies the mid-IR obscuration does not share an axis with the radio jets. These sources are highly obscured quasars, observed in the transition stage after the birth of the radio source, but before feedback effects dispel the interstellar medium and halt the black hole accretion and starburst activity.