Having seen this image since inception, I never noticed Mt. Fuji in the background.
In this case, the boats are fast (each has a bunch of crewmen) and were used to catch valuable fish. And the boats on the right have two people not at work (barely discernable in TFA's recreation). Those people were on break, getting ready to replace tired oarsmen. That way, the boat could be moving at all times.