Businesses could choose to register in each state or a single one using the VoES (VAT on e-services) system. These rules have been in place since 2003.
However I imagine that a lot of non-EU companies didn't bother with this either out of ignorance or simply because they didn't care or thought it could be enforced.
Under these new rules, MOSS replaces VoES but it's practically the same thing. This mostly affects EU businesses who now have to charge the rate where the customer belongs, not where the supplier belongs.