Does it? I loaded Guanaco-65B on a 48GB graphics card using the new flag and it loaded just fine. I don't think the unquantized model fits on a 48GB card. So I'm not quite sure if this is true.
So you still need enough system RAM (or RAM+Swap) to load the unquantized model.
You could absolutely do something streaming, or mmap the weights instead of loading them into system RAM. Just the default interfaces don't.