You're right! I was executing time go run main.go

I now built the exe and ran:

    real    0m0.199s
    user    0m0.000s
    sys     0m0.014s

So 0.2s for 10k files on the worst possible hardware/software scenario I could find nearby. Edited my original comment. Thanks!