A start-up needs to have an exit to pay back investors. A company that isn’t hoping to exit is just a lifestyle business. Both are valid options, but very different tactically.
I used to work for a 100-year-old Japanese corporation. They absolutely refused to go into debt, or seek investors. It could be challenging, getting them to loosen their pursestrings.
But they survived depressions, recessions, and even being bombed in WWII. They had a massive implosion, a few years back, that would have killed most companies (it resulted in me being laid off), but it looks like they are getting their feet under them again.
It depends on the kind of investors you want to attract for your startup. Some investors are interested in building a slow-growth, long-term, stable company, while others prefer a higher-risk approach and will expect fast results. IMHO choosing the right kind of capital is just as important as focusing on the product.