If you have time to write beautiful code at a startup, I might be suspicious. If something is ugly but not broken, why spend the time to fix it (versus releasing a new feature)? If you do everything right the first time, more power to you, but I think the prior suggests otherwise.
"Look for fundamental problems and propose fundamental solutions. Shoot as deep into the software stack as plausible/possible."
Not that I explicitly disagree, but the deeper you go, I suspect the higher your implementation risk.