Every token generates a KV cache entry based on this token and all previous KV cache entries. This happens in every layer.
KV cache is 100k-1MB per token so quite a bit.
Edit: also you can forward generate 5 or 10 dots in batch without much overhead compared to a single dot since the main cost is pulling KV cache from VRAM so you have free tensor units.