You can go wherever you want. The house isn’t going anywhere, it’s still registered as yours and you still pay taxes for it, with a discount if you designate it as your primary residence.