Ocean fishing requires very little energy input, but provides huge amounts of ready to eat food output. You can't replace that in many parts of the world. Not every place has good farm land for one, or the technological infrastructure, or even political stability.
Give you and example. Egypt went from a population of 8 million to 80 million in the last century. Do you think they can grow all that food in the desert? or do they rely on fishing and on global imports of food items...like grain from Ukraine. How will the poor do when you take out one of their sources of food. While simultaneously the price of energy is sky rocketing and global trade is under threat do to conflict and as countries begin to hoard fertilizer and food for their own populations, in anticipation of the the coming winter.