I wonder what kind of hardware OP runs on, and which compiler is used.

Here on my laptop (Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz), on linux, compiling an eigen example takes 0.7 seconds

    clang++ -fuse-ld=lld eigen.cpp -I/usr/include/eigen3/  0,66s user 0,07s system 100% cpu 0,728 total

if I put the eigen header in a PCH this drops to 0.1 seconds

    clang++ -fuse-ld=lld eigen.cpp -I/usr/include/eigen3/ -include-pch   0,08s user 0,02s system 104% cpu 0,099 total

I have a hard time seeing how 300 lines of code (vs the 10 lines of the example) would multiply compile times by 300 - here's the example I used

    #include <iostream>
    #include <Eigen/Dense>
 
    using Eigen::MatrixXd;
 
    int main()
    {
      MatrixXd m(2,2);
      m(0,0) = 3;
      m(1,0) = 2.5;
      m(0,1) = -1;
      m(1,1) = m(1,0) + m(0,1);
      std::cout << m << std::endl;
    }

I develop a GUI software with boost, Qt, and a fair amount of TMP and my average turn-around time from a change in the average file to the program running is between in and 2 seconds. Maybe it could get down to 0.1~ seconds if using C, but there would be so many less checks and guards !