For me too it was swapping my HDDs for SSDs. Also, getting 64GB RAM (I do lots of video and stuff).
Everything else, moving from CPU generations to newer generations, _felt_ like it barely made a dent.
GPU rendering via CUDA also meant a massive jump.