Because probably only larger companies (very difficult for startups to innovate here!) can actually afford individual offices, and it's not exactly trivial to re-arrange, I think it just takes a long time for the conventional wisdom to change.
I agree, I just don't think it's as much about cost as you might think (or as I used to think). Profitable companies can afford office space.
I feel like at some point (around the turn of the century ~2000) it was cool to be open office, if you weren't you were uncool, and the latest hipster generation didn't want to work for you.
I think things are changing though, and people are seeing past the latest fad of what people today call "open office." It just takes a while for the general-pop wisdom to change. Nobody's ever been fired for building open office, during this latest phase.
We were already open-floor, and they modified the plan to pack us tighter. Nobody ever thought that was "cool", in any way.