We reported recently some rapid changes of sunspot structure in white-light (WL) associated with major flares. We extend the study to smaller events and present here results of a statistical study of this phenomenon. In total, we investigate 403 events from 1998 May 9 to 2004 July 17, including 40 X-class, 174 M-class, and 189 C-class flares. By monitoring the structure of the flaring active regions using the WL observations from the Transition Region and Coronal Explorer (TRACE), we find that segments in the outer sunspot structure decayed rapidly right after many flares; and that, on the other hand, the central part of sunspots near the flare-associated magnetic neutral line became darkened. These rapid and permanent changes are evidenced in the time profiles of WL mean intensity and are not likely resulted from the flare emissions. Our study further shows that the outer sunspot structure decay as well as the central structure darkening are more likely to be detected in larger solar flares. For X-class flares, over 40% events show distinct sunspot structure change. For M- and C-class flares, this percentage drops to 17% and 10%, respectively. The results of this statistical study support our previously proposed reconnection picture, i.e., the flare-related magnetic fields evolve from a highly inclined to a more vertical configuration.