I remember the same misconception existing about fluorescent lamps in the past. "They consume more when turning on, so the gains vs incandescent are negated". Which was verifiably false to me at the time by checking the label on a lamp.
Air temperature is only one contributing factor for comfort in a home. Surface temperatures take longer to achieve and the larger the gradient the longer it can take.
There are multiple factors to take into account, including the highest temperature the home gets to, that is dependant on location and weather patterns.
If you keep the temperature in the home to 28-30 degrees then later cooling to lower temperatures before bed time you can reduce costs and energy expenditures than if you let it rise to 34 degrees or more and only lowering it when you get home. Or, what you save in costs you expend in otherwise like poor sleep.
Lowering it advance, on a schedule makes more sense if you want surface temperatures to also be reduced.
The blanket disabling of air conditioning when not home is not the only solution, and often is sub-optimal.