Einstein offered the principle of general covariance as the fundamental physical principle of his general theory of relativity and as responsible for extending the principle of relativity to accelerated motion. This view was disputed almost immediately with the counter-claim that the principle was no relativity principle and was physically vacuous. The disagreement persists today. This article reviews the development of Einstein's thought on general covariance, its relation to the foundations of general relativity and the evolution of the continuing debate over his viewpoint.