In the canonical approach to Lorentzian quantum general relativity in four spacetime dimensions an important step forward has been made by Ashtekar, Isham and Lewandowski some eight years ago through the introduction of a Hilbert space structure, which was later proved to be a faithful representation of the canonical commutation and adjointness relations of the quantum field algebra of diffeomorphism invariant gauge field theories by Ashtekar, Lewandowski, Marolf, Mourão and Thiemann. This Hilbert space, together with its generalization due to Baez and Sawin, is appropriate for semi-classical quantum general relativity if the spacetime is spatially compact. In the spatially non-compact case, however, an extension of the Hilbert space is needed in order to approximate metrics that are macroscopically nowhere degenerate. For this purpose, in this paper we apply the theory of the infinite tensor product (ITP) of Hilbert Spaces, developed by von Neumann more than sixty years ago, to quantum general relativity. The cardinality of the number of tensor product factors can take the value of any possible Cantor aleph, making this mathematical theory well suited to our problem in which a Hilbert space is attached to each edge of an arbitrarily complicated, generally infinite graph. The new framework opens access to a new arsenal of techniques, appropriate to describe fascinating physics such as quantum topology change, semi-classical quantum gravity, effective low-energy physics etc from the universal point of view of the ITP. In particular, the study of photons and gravitons propagating on fluctuating quantum spacetimes should now be in reach.