What you are writing is exactly why I threw game theory into the mix. Game theory show strategies that given a certain set of rules are beneficial for the player or all players (for interesting example see Nash equilibrium). The example you have given in first paragraph, that forbidding slavery creates a better environment for the whole of it's people is exactly just that - showing one strategy to be more beneficial than the other. It's not showing that slavery is "objectively" wrong. I remember seeing a nature documentary where two species were explained to live in something that could be seen a master/slave symbiosis which benefited them both. (As a sidenote, I'm all against slavery and I do not accept causing harm to others no matter how beneficial it would be to myself).

You say that "people tend to live better lives" and "I would like to live in" etc. This is very subjective. As for quantifying wellbeing of members, we may ask if it's better for 100% of society to feel well, or is it better for 80% of society to feel fantastic and 20% miserable and in fear. It's again a matter of opinion and we can use mathematical concepts like game theory to better explain using formal methods why some moral values would be beneficial for us and other's not. That's why I threw it into the mix.

You also say that some opinions are better than others. This general statement is very dangerous and was used throughout the ages by proponents of slavery, racism etc. They definitely claim that their opinions are better than yours while you do otherwise. The thing is, some opinions are better to achieve certain goal given some set of circumstances, rules and participants. Acknowledging that opinions and moral values are not objectively right or wrong doesn't make us crazy to pursue those that we see as moving the society we live in in the direction we want.