Individual choices are either based on personal experience or on information provided by peers. The latter case causes individuals to conform to the majority in their neighbourhood. Such herding behaviour may be very efficient in aggregating disperse private information, thereby revealing the optimal choice. However if the majority relies on herding, this mechanism may dramatically fail to aggregate the information correctly, causing the majority to adopt the wrong choice. We address these issues in a simple model of interacting agents who aim at giving a correct forecast of a public variable, either seeking private information or resorting to herding. As the fraction of herders increases, the model features a phase transition beyond which a state where most agents make the correct forecast coexists with one where most of them are wrong. Simple strategic considerations suggest that indeed such a system of agents self-organizes deep in the coexistence region. There, agents tend to agree much more among themselves than with what they aim at forecasting, as found in recent empirical studies.