A small societal elite of that small island got very rich off imperialism, much like in Spain or Portugal.
The broader enrichment of the British people was kicked off by something different, namely the industrial revolution. Spain and Portugal didn't have those and the average person there remained bitterly poor until deep into the 20th century.