Machine learning approaches are powerful because they can take a “small” set of labeled data and use that to make billions of new inferences at low cost. The problem is that even when these data sets are fully representative of the “real world,” the “real world” is rife with biases and unfairness and machine learning takes these unfairnesses and gives them a huge Tensorflow-powered megaphone.
For example, let’s say you wanted to use machine learning to create a system for employers to make sure that they aren’t hiring likely drug dealers. If you get a data set of every single American drug dealer’s mug shot, you’d get a disproportionately black population because white kids selling ecstasy to their high school classmates rarely get arrested and booked. The data set is not “biased” —- it is complete —- but the enforcement of the law is biased. But now, rather than using the development of this software as a moment to remove unfair bias, a machine learning model will amplify it, causing lots of black men to be denied legitimate employment opportunities, which will actually increase the chance that they sell illegal drugs to support themselves.
This machine learning doesn’t just amplify the biases in the data set, it amplifies all of the biases and other imperfections in all of the events that fed into the creation of that data set.