We have explored the redshift evolution of the luminosity function of red and blue galaxies up to z = 3.5. This was possible by joining a deep I-band composite galaxy sample, which includes the spectroscopic K20 sample and the Hubble Deep Field samples, with the deep H_AB = 26 and K_AB = 25 samples derived from the deep NIR images of the Hubble Deep Field-North and Hubble Deep Field-South, respectively. About 30% of the sample has spectroscopic redshifts and the remaining fraction well-calibrated photometric redshifts. This allowed us to select and measure galaxies in the rest-frame blue magnitude up to z ~ 3 and derive the redshift evolution of the B-band luminosity function of galaxies separated by their rest-frame U - V color or specific (i.e., per unit mass) star formation rate. The class separation was derived from passive evolutionary tracks or from their observed bimodal distributions. Both distributions appear bimodal at least up to z ~ 2, and the locus of red/early galaxies is clearly identified up to these high redshifts. Both luminosity and density evolutions are needed to describe the cosmological behavior of the red/early and blue/late populations. The density evolution is greater for the early population with a decrease by 1 order of magnitude at z ~ 2-3 with respect to the value at z ~ 0.4. The luminosity densities of the early- and late-type galaxies with M_B < -20.6 appear to have a bifurcation at z > 1. Indeed, while star-forming galaxies slightly increase or keep constant their luminosity density, "early" galaxies decrease in their luminosity density by a factor of ~5-6 from z ~ 0.4 to z ~ 2.5-3. A comparison of one of the latest versions of the hierarchical cold dark matter models shows a broad agreement with the observed number and luminosity density evolutions of both populations.