I'm sure you do see them, but having encodings consistent between sending client and receiving client isn't part of IRC; RFC 2812 - Internet Relay Chat: Client Protocol[1] says:

    > 2.2 Character codes
    
    >   No specific character set is specified. The protocol is based on a
    >   set of codes which are composed of eight (8) bits, making up an
    >   octet.  Each message may be composed of any number of these octets;
    >   however, some octet values are used for control codes, which act as
    >   message delimiters.
    
    >   Regardless of being an 8-bit protocol, the delimiters and keywords
    >   are such that protocol is mostly usable from US-ASCII terminal and a
    >   telnet connection.

Mojibake from disparate clients is part of the design; people have had to go outside the standard to make undocumented informally specified variation(s) such as "use utf8". Saying IRC is encoding-agnostic passing 8-bit bytes around, is like saying "I see emoji using TCP/IP every day".

(Contrast with HTTP where a client can declare what it accepts in the request headers on a per-request basis, and servers can declare encodings they are sending in the response headers on a per-message basis. Contrast with proprietary Slack where one company codes every client and can make them all behave the same).

[1] https://www.rfc-editor.org/rfc/rfc2812#section-2.2