Rage's temple (and all the gods' temples) actually do not have enough windows to incite anger! The colosseum would be fine, although given its size and complexity it would not be possible to find enough of any destructive material to destroy the whole thing. With planning, though, you could certainly destroy a section of the walls. Unrestored spires would be destructible with planning, too!