Worse - you get hurt irreversibly, sometimes mortally, very slowly over time. A broken leg is often the least of your problems compared to multiple concussions or the cumulative effects of CTE.

Removing helmets in football without removing almost all forms of contact will likely lead to deaths, which would hard to describe as "much less serious" than brain damage.