Basic physical concepts and theoretical ideas concerning dielectric heterostructures are reviewed from a historical perspective. This background for today's theory of dielectric heterostructures is discussed in some detail because the guiding principles for our understanding can be traced to the earliest developments in electromagnetism. To give an impression of the accelerating progress, I shall distinguish five stages in the development of our understanding of the dielectric properties of heterostructures. Historical remarks are included and technical concepts are introduced informally. For each stage, I call attention to synthetic works or compendia created during the interval. The first stage was reached towards the second half of the 19th century with the work of James Clerk Maxwell. Next the second stage was initiated by Bruggeman through the concept of an effective medium. Bounding methods form the third stage, with many investigators involved, beginning with the work of Wiener; the importance and ingenuity of these methods cannot be overstated. The fourth stage was the introduction of the crucial concept of percolation through an infinite cluster of connected particles and the modern approach to criticality which began in the mid-20th century with the work of Broadbent and Hammersley. Finally, the fifth stage we have experienced in the last decades involves the rapidly developing subject of computational electromagnetics: computers have moved the emphasis away from the general theory of macroscopic electromagnetism towards a better look at the detailed features of the randomness and connectedness of heterostructures. It is concluded that computational techniques provide a versatile tool for studying the dielectric properties of complex composite materials and that considerable progress can be achieved by comparing numerical results against analytical predictions for the properties of these models. As the capabilities for performing realistic simulations increase, it might become possible to routinely design on a computer, at least in part, a combination of materials chosen specifically to achieve a desired response to an incident electromagnetic wave for a variety of technological and industrial processes ranging from electromagnetic shielding and capacitive video disk units to mammalian tissue simulants.