Pass the release flag when you build loc; `cargo build --release`.

I see ~5 seconds for loc and ~34 seconds for loccount to count a freshly cloned linux repository.

edit: woops, that was user time. These both spin all the cores. loc takes ~0.7 secs and loccount ~6.6 secs real time.