First, be open about it to any potential partners and investors.
Second, are there any partnership options with the patent holder? Most people are smart enough to know a patent that is just held isn't all that valuable. Killing a startup won't help a company that isn't really in the field.
Third, an extremely cynical patent lawyer tells me the whole system is extremely fungible. This might mean that you can get out of infringement easily.
Forth, have any others tried to go against the patent and failed?