For nested data, it doesn't matter to me if it's JSON, YAML, TOML or whatever. Just agree on ONE format.
TOML also is a good example of "creating problems instead of solutions": They deliberately (!) broke compatibility to the INI format due to "I can't stand unquoted strings". Yeah, emotional feelings about CONFIG FILE FORMATS.
And again: If your config files are so complex, how about just creating a TUI/GUI so the user can configure your program in an accessible fashion?
Anyway, yes, there is something that can be improved: Create a standard instead of adding another layer of complexity for something that should be very simple in the first place.
Yes, binary size may feel arbitrary, but it often gives a hint about the hammer's size.