You have to manage memory for things like textures, models, and maps - each of which are generally going to be way larger individually than 80KB.

Here's an example of a pathological - but entirely reasonable to encounter - case:

You allocate several textures of various sizes. Your larger 3MB texture blocks might end up seperated by smaller allocations for, say, icons. Or other 3MB textures that can be reused on the next level (to reduce load times). Long story short, unless you do work to ensure textures are correctly grouped by lifetime (which in many cases will depend on and diverge based on user input) you're going to end up with lots of 3MB holes.

You switch levels, and release many/most of your 3MB environment textures. But the icons remain loaded for your UI, so you don't free those. So now you have 3MB holes everywhere.

You start to load the next level. It's using a lot more alpha textures in it's environment - lots of water and glass. The alpha channels adds another 33% to the size - these new textures are mostly 4MB. None of these new textures fit into the 3MB holes. You've already potentially just wasted about half your available memory (or address space), with a single texture size in two formats of note. Within a single system ("render"). Oops.