Does harder hiring in the SF Bay Area hurt or does it merely help weed out things that don't have a chance for some other reason?
As far as later stage employees go, successful silicon valley companies go elsewhere for what they can get elsewhere.
I think that you should start where you think that you have the best chance of success. I don't think that "harder to hire" or "more expensive" captures sufficient detail, but it's your company, so that's your call.