Its not so much about suddenly starting to decode something. It’s more when your stream for some reason gets misaligned or corrupted and whatever is still streamed is happily consumed with no possible way to detect the error.
Common reason being someone forgot to check how many bytes read() actually returned and just assume it filled the buffer.
Robust data formats have double protection against this class of bugs with unambiguous SOF and EOF boundaries that the consumer can assert. I guess a sanity-byte once every X Megabytes wouldn’t hurt either.