With the fall of the western empire, Rome's population collapsed from over a million to only tens of thousands. It wouldn't cross 100,000 again until the 18th century. It just didn't have the wealth or population necessary to hold such shows.
Others (and the Circus Maximus before it was taken over by the Frangipani) were simply saved from getting used as a quarry by getting incorporated in-place as structural elements for new, well, shacks, as documented by the many, many square holes hewn into the formerly flat stone surfaces. Square holes for holding wooden beams I suppose. The (still in use!) appartment on top of the Theatre of Marcellus give an idea of how that went. In this case, they are the outcome of a slow fortress->palace->middle class residential transition.