> would you guess the unit instead of specifying what you expect?
It depends on the circumstances. It might be the least bad thing to do. Or not.
But that wasn't my point. I replied to this:
> I can assume the length of a meter or the weight of a gram
Sure, the length of a meter and the "weight" of a gram are both standardized. (To be very picky, "gram" is a mass, not a weight. The actual weight depends on the "g" constant, which on average is 9.81 m/s^2 on earth, but can vary about 0.5%.)
So if you know the input is in meters, you don't need to do any further processing.
But dealing with input text files with an unknown encoding is like dealing with input lengths with an unknown unit.
So while UTF-8 itself might be standardized, it is not the same as all input text files always being in UTF-8.
You can choose to say that all input text files must be in valid UTF-8, or the program refuses to load them. Or you can use silent heuristics. Or something inbetween.