Dynamical systems theory has proven to be a successful approach to understanding mixing, with stretching and folding being the hallmark of chaotic mixing. Here we consider the mixing of a granular material in the context of a different mixing mechanism —cutting and shuffling— as a complementary viewpoint to that of traditional chaotic dynamics. Cutting and shuffling has a theoretical foundation in a relatively new area of mathematics called piecewise isometries (PWIs) with properties that are fundamentally different from the stretching and folding mechanism of chaotic advection. To demonstrate the effect of the cutting and shuffling combined with stretching and folding, we consider the mixing of granular materials of two different colors in a half-filled spherical tumbler that is rotated alternately about orthogonal axes. Mixing experiments using 1 mm particles in a 14 cm diameter tumbler are compared to PWI maps. The experiments are readily related to the PWI theory using continuum model simulations. By comparing experimental, simulation, and theoretical results, we demonstrate that mixing in a three-dimensional granular system can be viewed as mixing by traditional chaotic dynamics (stretching and folding) built on an underlying framework, or skeleton, of mixing due to cutting and shuffling. We further demonstrate that pure cutting and shuffling can generate a well-mixed system, depending on the angles through which the tumbler is rotated. We also explore the generation of interfacial area between the two colors of material resulting from both stretching in the flowing layer and cutting due to switching the axis of rotation.