1) Except when you don't have the money. Not everyone trying to build a business goes the funded startup route...
2) Except when it does. There are (a small percentage of) businesses where speed does matter and caching everything is not an option. Think exchanges, ticket sales, real-time analytics off the top of my head.
3) Yes, you want to quickly iterate. Sometimes this means using readily implemented libraries for the non-core parts of your businesses. Sometimes it means avoiding readily implemented libraries that do not fit your requirements and/or overcomplicate what you are trying to do. Sometimes it means avoiding the cognitive overhead of all that cruft a framework has and you don't need. Sometimes it means using a typed language and letting a compiler do the most basic testing for you...
4) To me, this is a classical exploration/exploitation tradeoff. You will be less productive using anything new. New language, new framework, new ideas drag your productivity right now. But they may increase it in the long term, even if you end up going back to whatever you were using before. Find some budget to explore and make the most out of what you learned, but don't overspend on it.