It is always possible that rent seekers will appear but remember, a good parasite never kills its host, and at present you are unlikely to be worth the energy to attach to (no offence intended).
As someone who has been sued by non-practicing entities related to a patent, it doesn't break the company. They just want money and they make it affordable. They sue you immediately in order to start the negotiations and show their seriousness. Often their license fees are explicitly set to make it cheaper than fighting them in court specifically so that you won't challenge the patent's validity - so you can settle and move on. It is a tax and it can be unfair, but it won't kill your company.
Honestly, can you share one example where a patent license fee killed a start-up company?
I am not a lawyer.