If you're using CJK fonts and you don't already have a CDN for your stuff, no. Google Fonts' CDN is going to beat your own server from any other continent.
For example, here’s Lato: https://www.jsdelivr.com/package/npm/lato-font
E.g. my site's About page is tiny, but pulls in a CJK font (with delayed load) for one line of text. It would make no sense to self-host that: https://marcan.st/about/
I'm not sure if Google font service actually does facilitate cross-site caching though - there might be tailored URLs or account?
If it truly is a must make it a small (self-hosted) png.
Fwiw I don't really think a one-time large font download is much of a problem - considering that improved typography is a very good way to improve how a site looks. And unlike your png - the text will display without the custom font. Fall-back fonts are only really a problem with icon font sets.
One time. Per site. On slow and/or expensive mobile connection?
The png will also display without the custom font...
It's a shame there are no "bandwidth" queries in css, like there are @media queries - the client should be able to say - just content and essential downloads, please.
You can already do this with JavaScript now: https://developer.mozilla.org/en-US/docs/Web/API/NetworkInfo... though as you can see from the bottom of that link, Browser support is limited at the moment (and experimental so may even be removed/changed!).
Self-hosting a Japanese font for one line of text is completely doable. I self-host all fonts for my site, which has a handful of CJK characters (across 4 fonts: Korean, Japanese, Traditional Chinese, and Simplified Chinese).