Dutch does that too, and I've tried out what happens if you say it correctly ("negentig 'n zes", ninety and six, instead of "zes 'n negentig")
It takes a second to process and then they'll ask "do you mean [reverse order variant]?" so they do kinda get it and I think transitioning to the sane version could be possible without much trouble, but people would have to want to