If you took falling lessons as a kid, and you knew how to properly respond- following the push to a controlled fall, roll, or other redirection- you'd be fine. Sometimes the overreaction is the problem.
Lots of close contact will mean you're far less likely to panic in a fight situation. Most people have never been in a fight let alone hit.
Plus the fitness of course. Judo helps enormously with core strength.
BJJ borrows a lot from Judo, particularly the grappling.