Start ups are moving quickly and trying to get to the next milestone. That often means they really want people on or off, but not off "thinking about it."
That said, it sounds like you already made up your mind right? You are having second thoughts so own that and say "Thanks, but no thanks." and keep looking and/or doing what ever it is you are doing.