The idea is that the EU is a lot less sensitive to losses in the trade volume.
The EU has vital interests in controlling the consumer goods available in its market. If we don't have a say in, for example, how they control their food production, we can't just take their word on it.
If you now think, let's just assume the British government does its job correctly in that regard, and we'll be fine, you're wrong. Companies often know which of their products are going to end up in exports, so they know where they don't need to follow the rules.
The exact figure is disputed, leave were saying £450 million per week, remain were saying it's £150million per week.
https://fullfact.org/europe/our-eu-membership-fee-55-million...
*not including cheaper labor into account.