They also loved Chinese silk.
I'm kind of surprised you hadn't run into this yet. It seems like every Roman author is bemoaning the popular love of food drenched in pepper and transparent silk clothes. Try reading Juvenal, it's quite funny to see what he decries and what he considers normal.
Europeans never stopped importing spice, from ancient times to the present day. To my knowledge there was never a complete interruption.