> The important part when communicating directly with a device by IP address, is that the payload is encrypted end to end.
Which you can't be certain of unless you can verify where the "end" actually is. The IP address is no help here; the ARP cache can't be trusted. Packets can be redirected to any host on the local network regardless of the official IP address assignments. If you aren't authenticating the other end of the link then you can't be sure no one is snooping. Active MitM attacks on local networks are trivial. You don't even need to compromise the router—any peer on the network will do.
> The router could forge a self signed certificate even more easily than it could forge a certificate signed by a trusted CA.
Sure. As a form of TLS without authentication, one-time-use self-signed certificates would be worse than useless, offering only a false sense of security. The options here are "trust on first use" (betting against a malicious actor being present during initial setup) or offering some way to manually verify the fingerprint of the device's private key. Both approaches require authentication, not just encryption. For the second, more secure option, the cheapest way would be to permanently provision each device with a unique private key and print the key's fingerprint on the device's exterior. For more flexibility, store the key in a small removable HSM (similar to a SIM card) and print the fingerprint on the HSM itself and/or the accompanying documentation.