I think the culture of the office matters, too. If there's a culture, of, "Hey people are working here let's not be loud." and "She has her headphones in so let's not bother her." then it won't be too distracting or have too much noise.

Conversely, if an office with a closed office plan has a culture where closed doors are common and it's rude to knock, then collaboration will definitely be hindered.

We need to collaborate and to work in peace and quiet. This is probably why it oscillates between open and closed office plans. Neither solution is perfect. The best solution is probably default closed offices with a culture of having your door open and lots of collaborating spaces that are actually used.