The data for CAD exchange files is usually plain text, the problem is that the exporter doesn't try to preserve the same order of stuff as previous times that it was saved.
That seems almost asinine. It should be trivial to have a standard sorting algorithm so any implementation produces the same result.
I'm almost of the opinion that it should be thrown back on the software to convert their save files to text where possible. Binary files just aren't great for a wide range of issues further than just git.