Curious why you didn’t feel it was worth it to be honest with them in that moment? It’s not like they could stop you from leaving.
I left a job recently and had a great conversation with my boss about what went wrong and how I think they could improve and after talking to some old coworkers, it seems they have taken a lot of what I said to heart.
I get it not wanting to talk about it, but at the same time I think that feedback is the most valuable and I would want someone to tell me if I was in their situation.