Vacation: I can never take a vacation (or that's what it feels like). I think this is fairly typical of most startup/contractors since you don't get paid if you don't work, but I wish it were different. This is probably due to lack of experience which will change over time.
Networking: I have some legacy clients that get my team through the months but 100% of our business is repeat or legacy. This is extremely problematic and even though I do a lot of networking, it seems to all for a loss. Finding new clients is definitely the hardest part for me.
Hiring/Contracting: Finding good talent in local towns is tough. Always have to do coding review on developers to ensure correct methodology, look at past project experience, etc. The hardest thing to determine (in my opinion) is how hard someone works.
As others have stated... finding work = hardest problem. It is a bitch to say the least.