I'm not so busy that I don't stop to eat lunch, so we would still go out on days where I didn't have quite so much to do and could afford the extra hour or so that eating somewhere other than my dining room requires, but then he would do things like remember he needed to stop somewhere else and could we just do that while we're out so he doesn't have to drive all the way back into town? (We live in a fairly rural area) So a trip out for lunch would turn into lunch, grocery store, his business (he owns a store here in town), bank, etc. and suddenly the hour and a half I'd planned to take off had turned into three or four.
I can't say I blame him. I enjoy spending the time with him and it's nice to get out and do some things together during the day while the kid's in school, but it just isn't practical most days. Now we very clearly plan what we're going to do when we go out on the days I'm working and he's not.