I eventually made the decision that my house is not an investment, it's a home, which will I will use for shelter, and that I do not plan to sell for 20+ years. So long as I can afford to make the mortgage payments and not be house poor, I will be reasonably happy and this should work out.
Through that lens I figured even if the housing market were to collapse, I will have 20 years to make up the difference.
Right or wrong, this helped me frame it in a way to so I remove the timing aspect.