The plants don't simply grow in the pot, you had to get potting soil which had to be transported and was most likely packaged in plastic.
You might have driven somewhere to get the seeds, etc.
The pots had to be made from clay (needs quite some energy to be fired in a kiln) or plastic.
And your scenario might be even one of the more optimal ones, with no additional land use...