> - Dedup keeps blocks with the same data as a reference. This costs a lot of memory

To get around this you can add an SSD mirror as "Special" vdev, the dedup table will then be saved there, freeing up your memory.