In this paper we describe a technique that may be used to model the geometric uncertainties that accrue during the radiotherapy process. Using data from in-treatment cone beam CT scans, we simultaneously analyse non-uniform observer delineation variability and organ motion together with patient set-up errors via the creation of a point distribution model (PDM). We introduce a novel method of generating a coverage probability matrix, that may be used to determine treatment margins and calculate uncertainties in dose, from this statistical shape model. The technique does not assume rigid body motion and can extrapolate shape variability in a statistically meaningful manner. In order to construct the PDM, we generate corresponding surface points over a set of delineations. Correspondences are established at a set of points in parameter space on spherically parameterized and canonical aligned outlines. The method is demonstrated using rectal delineations from serially acquired in-treatment cone beam CT image volumes of a prostate patient (44 image volumes total), each delineated by a minimum of two observers (maximum six). Two PDMs are constructed, one with set-up errors included and one without. We test the normality assumptions of the PDMs and find the distributions to be Gaussian in nature. The rectal PDM variability is in general agreement with data in the literature. The two resultant coverage probability matrices show differences as expected.