If you're not working on something that is currently hot in Silicon Valley, there is little reason to be here. It's not like Portland, where you decide that you want to live there first and then decide what you'll do for a living. But if you do want to work in one of those hot sectors, there really is no place to do so other than Silicon Valley; nowhere else do you get the same combination of talent, capital, experience, open-mindedness, and risk-taking.
It doesn't surprise me that people are moving out (even beyond personal experience, where basically all of my friends have remarked that they're considering moving to Colorado, or Seattle, or Portland, or North Carolina, and my wife & I are considering it too). We're at the end of a technology cycle, so nobody really knows what the next one will be or if it's something that'll interest them. But there are a lot of fascinating things going on with drones, robotics, self-driving cars, AI, satellites, and hardware, so if any of them seem interesting, Silicon Valley might be a good place to check out.