Part of the reason white collar firms outsource is that benefits usually need to be uniform across all employees. If you are a white collar firm with generous benefit packages, it becomes comparatively expensive to hire blue collar workers yourself compared to outsourcing. Decent health insurance (~$20K/year) is more than the federal minimum wage ($14.5K/year). I'm not suggesting this is moral, but it is reality. At a factory, with many workers at similar pay levels, this won't be as much of an expense.

Unfortunately, a second reason you see outsourcing are job functions where many of the workers in a geographic area are undocumented. Companies can't hire undocumented workers directly, but will allow a middleman to do so on their behalf.