An additional problem is that Excel, the tool many people use to open CSV files, uses the computers regional settings for decimal separator and thousands separator. CSV files are used as an interchange format, so the format of the numbers in it does not depend on the region of that specific computer.

You can tell Excel to always use '.' as a decimal separator, but then it also uses it for presenting numbers to the user. It boggles my mind that software like Excel doesn't understand the difference between reading/formatting numbers for human use and reading/formatting numbers primarily meant for talking to other software.