I definitely suggest you attend Startup Weekend, because you can use it to try on different roles without fully committing to a startup. The fact that you can understand code and technical issues, without being a developer yourself, positions you really well in my opinion. If you want to be really early, consider being founder. If not, a role like community manager (1st marketing person often) could be a great fit for you... that usually includes customer service, sales, marketing, and a lot of the non-technical jobs rolled into one. You can decide where to focus later.
I've done it 2x now (Pelago, Twilio) and it's possible. Like everyone in the startup, you mostly have to be crazy dedicated to working hard/smart as needed, and build amazing trust with your team.