Weight is proportional to the cube of body size (length) while muscular strength is proportional to body size. The strength/weight ratio is inversely proportional to the square of body size. As size increases strength increases proportionately but the power/weight ratio goes down rapidly.
Fleas are an extreme example: they can broad-jump like no person. Shorter people, like fleas, have an advantage in some sports (e.g., gymnastics) because of their higher power/weight ratio. This also makes a difference in endurance events where shorter people can sometimes perform exceptionally well because they expend less energy per mile.
Don't waste your money on old myths. Serious swimmers today do weight training. Big people are stronger than small people generally. If you don't believe me, pick a fight with a mean guy who's taller than you. Or better yet, pick a fight with a mean, tall competitive swimmer.