It's the same reason I won't buy something through my Alexa. I want to see the price, I want to make sure it's not 50% smaller than the last time I bought it, and I want to compare. The same is true for food.
I work for a POS company and was asked about the ACP (Agentic Commerce Protocol) and I told my boss we should skip it for now because of this exact reason. Not anti-AI at all, in fact I've built tools for our clients to manage their menus via LLM ("Add Pumpkin Spice as a seasonal flavor to all my coffee drinks", "Mark lettuce as out of stock", etc) but ordering food does not feel like a good fit for agents as things currently stand.