Turning the AC or heating completely off when you're not at home is never recommended. To save power, you just let them run at temperatures a few degrees outside your comfort zone when your away, and it will still have enough thermal inertia to quickly get back to comfort levels when you're back without the risk of condensation/mold.

Where I live, this is even written in my rental agreement, that when I leave home for long winter vacations, I must leave the heating ON.