Worst case, we go to WTO tariffs. The effect of that in theory is that EU imports to the UK will reduce, as they become more expensive relative to the same goods from non-EU countries (note that our tariffs to non-EU countries will reduce post Brexit).
UK exports to the EU will also reduce for the same reason. However, given that we're a net importer it should be clear who is in the worst position.
If the UK can sign a couple of decent trade deals with large economies like India, USA, China then we won't be worried about the EU trade at all.