Parameter identification problems for partial differential equations (PDEs) often lead to large-scale inverse problems. For their numerical solution it is necessary to repeatedly solve the forward and even the inverse problem, as it is required for determining the regularization parameter, e.g., according to the discrepancy principle in Tikhonov regularization. To reduce the computational effort, we use adaptive finite-element discretizations based on goal-oriented error estimators. This concept provides an estimate of the error in a so-called quantity of interest, which is a functional of the searched for parameter q and the PDE solution u. Based on this error estimate, the discretizations of q and u are locally refined. The crucial question for parameter identification problems is the choice of an appropriate quantity of interest. A convergence analysis of the Tikhonov regularization with the discrepancy principle on discretized spaces for q and u provides a possible answer: it shows, that in order to determine the correct regularization parameter, one has to guarantee sufficiently high accuracy in the squared residual norm—which is therefore our quantity of interest—whereas q and u themselves need not be computed precisely everywhere. This fact allows for relatively low dimensional adaptive meshes and hence for a considerable reduction of the computational effort. In this paper, we study an efficient inexact Newton algorithm for determining an optimal regularization parameter in Tikhonov regularization according to the discrepancy principle. With the help of error estimators we guide this algorithm and control the accuracy requirements for its convergence. This leads to a highly efficient method for determining the regularization parameter.