Wouldn't you want your robot precisely designed to the spec of your kitchen as it throws hot oil around, rather than figuring it out on the fly with computer vision? As another commenter pointed out, robots have been frying potatoes at immense scale for decades, without a single LLM or AI in sight. Just "normal" computer programs can do it.