thank you for the detailed answer, this is all very interesting to me as I'm just starting a Masters in AI. I always had the wrong idea that LLMs couldnt be made deterministic. Now that I know they could be this opens up more possibilities for me.
There are quite a few resources on the web that can help with this. One good one is Andrej Karpathy's "Let's build GPT: from scratch, in code, spelled out.": https://www.youtube.com/watch?v=kCc8FmEb1nY
A somewhat more intensive one is "Build a Large Language Model (From Scratch)" by Sebastian Raschka: https://www.sebastianraschka.com/llms-from-scratch/
And if you really want to go crazy, Stanford's "CS336: Language Modeling from Scratch" is online: https://cs336.stanford.edu/
(The last one is a full graduate-level course, so don't let it detract from your actual master's program!)