Currently, we have highly specialized apps which do exactly what they need to do. They were created by someone who cares about making the program which works right and reliably. Most programs have years of bug fixes and deterministic algorithms. Many of them run locally and waste a minimum amount of energy to do the calculation.
But this is different - this encourages you to make one-off apps for each little thing you want to do. Imagine someone making a "bill splitter" app every time they go out with friends and how much energy that would waste... Just to do "14+16" that you could do in your head.
Also imagine it coming up with new UIs every time (depending on the prompt/context), and even potentially gaslighting you and doing calculations wrong because no one ever tested it before.
But overtime I'm sure it would arrive at a solid, user tested bill splitter that you can just tweak and customise as much or as little as you like, saves it to your profile and there you are, a reusable, fully customisable tool.