Are you near the city center, or out near the suburbs? It's common for a bus to be crowded on one part of the route and nearly empty at the end.
And this is a fundamental issue with transit. It _has_ to travel as mostly empty on avereage to be useful.
...no? In the heart of downtown, with people riding in both directions, I don't see how it follows that it has to mostly empty to be useful. If it's full you can't get on so that's not useful, but anything less than totally full should still be useful.
Its not a meaningful issue. The passenger density is higher on even a mostly empty bus than cars, and the pollution is lower as well.