It is commonly understood that plants do not have the same mechanisms to experience a range of emotions (pain, happiness, sadness, etc.) as fish, birds, and other animals do.
Further, if we want to abstract away from a mechanistic understanding and focus purely on a reduction of suffering, then veganism should be your goal. Most crop production is simply for animal feed. Eliminating animals from our diets would dramatically decrease the amount of plants we destroy.