Dwarf galaxies play an important role in our understanding of galaxy formation and evolution, and starbursts are believed to affect the structure and evolution of dwarf galaxies strongly. We have therefore embarked on a systematic study of 12 of the nearest dwarf galaxies thought to be undergoing bursts of star formation. These were selected primarily by their morphological type (blue "amorphous" galaxies). We show that these blue amorphous galaxies are not physically distinguishable from dwarfs selected as starbursting by other methods, such as blue compact dwarfs (BCDs) and H II galaxies. All these classes exhibit surface brightness profiles that are exponential in the outer regions (r 1.5r_e) but often have a predominantly central blue excess, suggesting a young burst in an older, redder galaxy. Typically, the starbursting "cores" are young (~10⁷-10⁸ yr) events compared to the older (~10⁹-10¹⁰ yr) underlying galaxy (the "envelope"). The ratio of the core to envelope in blue light ranges from essentially zero to about 2. These starbursts are therefore modest events involving only a few percent of the stellar mass. The envelopes have surface brightnesses that are much higher than typical dwarf irregular (dI) galaxies, so it is unlikely that there is a straightforward evolutionary relation between typical dIs and dwarf starburst galaxies. Instead we suggest that amorphous galaxies may repeatedly cycle through starburst and quiescent phases, corresponding to the galaxies with strong and weak/absent cores, respectively. Once amorphous galaxies use up the available gas (either through star formation or galactic winds) so that star formation is shut off, the faded remnants would strongly resemble dwarf elliptical galaxies. However, in the current cosmological epoch, this is evidently a slow process that is the aftermath of a series of many weak, recurring bursts. Present-day dE's must have experienced more rapid and intense evolution than this in the distant past.