Materially, in the US slaves often had higher material 'quality of life' than northern (free) factory workers. They had a better diet, lower pollution, better housing, and more time off. The plantation owners effectively viewed them as an investment. Really, if you want to talk about underpaid and overworked, a more apt comparison would be factory and mine workers in the 18th century.
The horror of slavery was that you had no choice in life. Who cares if you are fed well if someone else can decide to split your family up. Someone can kidnap you, drag you across an ocean (where you have a high chance of dieing) and put you in a country where you can't speak the language. Someone else can decide to move you to a new home. Someone else can decide whether or not you are allowed to read. Someone else can legally beat you.
The Japanese salaryman/women can quit her job if they want to. If they don't, that is their choice, but it is a choice. That is a massive difference. Any comparison between the two really papers over the horror of slavery. Much like any Nazi comparison, it drags the quality of the discussion down.