Grass puts Nitrogen into the soil. There are other plants, but grass is the most well-known. Things like corn or grain take nitrogen out, and use it. In nature these plants grow centimeters apart, so there is no problem. There's a nitrogen cycle. However if you want much denser grain, you can only have grain.
So now we've got agriculture, and it requires density. Large plots of land where you've got nothing but grain. Nitrogen goes out, doesn't come back in. Grain grows for 2-3 years, then refuses. It dies, because it cannot use the nitrogen from the atmosphere. It can only get it from the soil.
So we found a solution a very long time ago (it's in the bible, surprisingly, although we have figured out slight improvements since then). Do agriculture in a 7 year cycle: leave blank, grain, grass + animals, grain, grass, grain, grass + animals. There are many cycles used in various places, but they're all quite similar.
Fertilizer shortcuts the process: so why does it increase the yield 3 times ? Because you can cut out the grass years and the blank years. Not because the plants grow faster, because you can grow 3x as many of them.