"Hire Fast, Fire Faster" is total nonsense. It's "Hire Slow, Fire Fast".
You hire slow because the risk/cost associated with a bad hire is infinitely higher than the downside of not making a hire. Moreover you should always take as much care as possible when you are potentially fucking with someone's career/livelihood.
You fire fast because despite hiring slow you have realised that you fucked up, and made a bad hire. Once you understand the hire is bad it is in both yours and your employee's interest to fire fast - albeit you must always do this with the utmost respect for the person being fired.