As far as I know, hogwild doesn't really scale to very large systems and doesn't get used in practice, but maybe that's just because it's most useful on CPUs, which are inefficient for other reasons