It is a balance. Turning down work is dangerous, and the generally opposite of why you're in business. But giving other firms too much leverage will kill you just as dead as insufficient work. I started getting nervous when one company was over 50% of our income, but there was disagreement about urgency and what exactly to do about it with my partners, and we dithered until the bomb went off.
For people who are reading this quickly, this is the magic that makes saying no ok. If you are just turning down client requests they'll stop asking. Most people are extremely busy so they really appreciate having a "go-to guy" they can send any requests even vaguely related to what you do.
I guess I should add, this was in the 2000s. We started the company in the aftermath of the dot.com crash. Getting work was a major challenge, and turning down anything felt like asking to be made homeless. It is a very different market now.