Some matrix math libraries like Eigen[1] support vectorization via SSE, AVX, NEON, etc. and also use cache friendly algorithms for larger matrices. Highly recommended when you don't need to go quite as low level as individual instructions.
For custom work, https://github.com/VcDevel/std-simd is working it's way into the C++ standard.
I agree, but that feature allows to apply optimizations by specializing these templates.
Works for both micro-optimizations (in their pbroadcast4<__m256d> they do 4 loads, on many CPUs AVX2 can do better with a single load + shuffles) and replacing large parts of Eigen (I was able to improve performance of conjugate gradient solver by moving the sparse matrix into a SIMD-optimized structure).