leiframming

No, I think the author gets this mixed up.

Zero copy does not play a role here - the problem is how the model is being loaded in the steps that are shown in the article and they have nothing to do with Safe tensors' zero copy ability. Check out the article by Hugging face to see how to do it with only 360GB instead of 720:

https://huggingface.co/docs/accelerate/usage_guides/big_mode...

Interesting article, but the premise is wrong. You don't need 720GB ram to load the model. The author claims that you need to first load up the model (360GB) and then the load the weights (another 360 GB) and the replace the first with the latter. HF allows initializing empty models that do not take up any space using accelerate, especially so that you don't run into that problem.

Still, 180B is a best to load up.