You can get fast and good latency 32 GB DIMMs now. G.Skill has 4x32GB 3600 18-22-22-42, timings not the best, but tighter timings always get harder at higher speed and chips. The 3200 kits are 16-18-18-38.

Edit: Corsair also offers 3200 kits I'm pretty sure, too lazy to look up their exact specs