No, growing plants out in the sun is much more efficient than capturing the light with a solar panel and then growing the plants elsewhere.
Or rather, too much sun for available water. Installing solar panels high enough that goats don't bump into them, and planting shade-tolerant arid plants, can absolutely support grazing.
Some grasses grow with partial light. With the right inter-panel spacing, could it be that a grazing animal might require more space, but not to an unreasonable degree? (I assume there are other considerations for the pasture...ease of finding the animals, moving them around, etc)