When assessing arguments like this I find it useful to substitute animals for a hypothetical population of hominids with cognitive abilities typical of nonhuman mammals. If an island populated by such people were discovered, and they turned out to generally live nasty, brutish and short lives for some reason, would that justify farming them for meat?
The answer is clearly no. If you were to ever encounter a hominid of any cognitive ability who was being farmed for meat, you would be horrified and would report the situation to law enforcement. That individual's treatment by the operators of the farm would qualify as brutal regardless of how much people of the same background tend to suffer out in the world.
I don't think there is a valid reason to treat hypothetical hominids with cognitive abilities typical of nonhuman mammals differently from actual nonhuman mammals.