I do this too, but I use cmake to switch the backend between pandoc and lowdown [0]. Lowdown is not a drop-in replacement for pandoc flavored markdown, but it's between 10-100X faster.
It's very handy to switch when tinkering with template changes because I can convert ~ 1000 markdown files instantaneously on my mid 2010s desktop.