This arch uses engrams aka per layer embeddings. Not all of those GBs need to go in vram.