I agree that design shouldn't be the focus if you are at the early, bootstrapping stage. You should instead focus on user traction and shaping your product based on user feedback. You can always work on the design part later on.

That said, for a consumer facing product, user experience and design does become important after a point - but it need not be your priority at the very early stage.