Politely mentioning that you don't want to work on holidays could be a great answer if they ask why you're looking to change companies. You could just say something like: "I'm looking for a job that allows me to spend holidays with my family." (and leave it at that)

It's a totally normal expectation, the explanation doesn't reflect poorly on you or your past employer, and it sounds like it is genuinely important to you.