"Hire slowly, fire quickly" is an oft-repeated maxim for a good reason.
As for the most humane way to fire someone in a startup, I have a few thoughts (borne from several misfires earlier in life):
1. Sit down with your employee and give them a formal warning. "You're not performing up to our expectations, and if things don't improve within 14 days, we're going to have to replace you." Give them a list of specific items to work on.
2. On the day of the firing, always have two people from your company present in the room. This mitigates a surprising number of "he-said/he-said" type disputes, and also (surprisingly) leads to a less contentious conversation. If it's just the two of you, the odds of someone saying something they'll regret increase substantially.
3. Be generous with severance. I've never regretted giving an employee 4 full weeks of severance, no matter how junior.
4. Be direct. Resist the urge to treat it like a high-school breakup (i.e. "It's not you, it's me.") I have always been advised by HR professionals to never mention "cultural fit", as it's a gateway to wrongful termination suits. However, I have been ignoring this advice for about a decade. I believe an honest firing trumps a legally "safe" firing.
5. Most importantly: don't plan on doing anything else for the rest of the day. Take your team out and order drinks. If your other team members want to go home after lunch, by all means let them. In a small team, terminations can be very emotional for everyone, and nobody feels good about working right after their coworker got canned.