Yes, and an efficient tokenizer designed only for that language. As the ratio of synthetic data to human data grows this will become more plausible.

LLM as a frontend to LLVM IR maybe.