Andrew Kelley (author of zig) has a nice talk about programming without pointers allowing ultra fast serialize/deserialization. [0]
And then you have things like cap'n'proto if you want to control your memory layout. [1]
But for "productivity" files, you are essentially right. Portability and simplicity of the format is probably what matters.
[0]: https://www.hytradboi.com/2025/05c72e39-c07e-41bc-ac40-85e83...