G54.1+0.3 is a young pulsar wind nebula (PWN), closely resembling the Crab, for which no thermal shell emission has been detected in X-rays. Recent Spitzer observations revealed an infrared (IR) shell containing a dozen point sources arranged in a ring-like structure, previously proposed to be young stellar objects. An extended knot of emission located in the NW part of the shell appears to be aligned with the pulsar's X-ray jet, suggesting a possible interaction with the shell material. Surprisingly, the IR spectrum of the knot resembles the spectrum of freshly formed dust in Cas A, and is dominated by an unidentified dust emission feature at 21 μm. The spectra of the shell also contain various emission lines and show that some are significantly broadened, suggesting that they originate in rapidly expanding supernova (SN) ejecta. We present the first evidence that the PWN is driving shocks into expanding SN ejecta and we propose an alternative explanation for the origin of the IR emission in which the shell is composed entirely of SN ejecta. In this scenario, the freshly formed SN dust is being heated by early-type stars belonging to a cluster in which the SN exploded. Simple dust models show that this interpretation can give rise to the observed shell emission and the IR point sources.