They may yield this time. Especially if you are an important member of the team with some exclusive knowledge (i.e. you're the only one who knows the codebase for component Foo, and if they lose you they are screwed). But, they will likely be bitter about it. Try to force you to share your knowledge in the next few months and kick you out.
Again, I agree that it makes perfect sense to you as an employee to maximize your income, it's just business. But a lot of business owners, especially in startups, may take it personally ("How dare he want to leave us? How about our mission of making the world a better place? He's only in it for the money ????!!!").
So, long story short, I would amend this advice with not disclosing that you have an other offer and not setting an ultimatum.