Both are selling your body for (relatively) high pay. Both have you burn out usually by your mid-late 40's. Both are dangerous work. Both have high degree's of human trafficking*. Both subsist mostly on young naïve workers. Both have no education requirements. Both have success predicated greatly on genetics (strength vs. looks). Both are the oldest professions in the world.
Really the most substantial difference I can see is in how society values men vs. women, dispensable vs. protected. Which is of course the same way humans valued men and women for pretty much all of history.
*Trafficking is tricky because in the US it's basically if you do (illegal) sex work and have a boss you are being trafficked. Basically everyone in construction would be "trafficked" then, so instead I go with a high degree of desperate illegal immigrants who will do really any work (ripe for exploitation).
Edit: I should note that my take is that prostitution should be legalized, regulated, unionized, and not stigmatized. It should be on the same social rung as labor.