I haven't examined your site's code, but it sounds like you should be using the CSS Font Loading API (https://drafts.csswg.org/css-font-loading/) to ensure the font(s) you want are loaded and ready, before attempting to draw text in canvas.
Thank you for the tip. I think I did briefly look at that, but the "experimental" banner on the web api documentation page scared me off. Looking at the browser compatibility chart, it does seem to be pretty widely implemented at least.
FontFaceObserver linked in the blog post is what you want. Also for general usability reasons. The standard API is quite difficult to use.