Samuelson's argument basically comes down to the fact that the investments that maximize growth don't maximize your utility unless your utility is log(wealth), and to him it was completely obvious that people should act to maximize their own utilities.
However, there have been a couple recent developments that somewhat undercut his line of thinking. First of all, to the extent that people's behavior can be described by utility theory, we all roughly have isoelastic utilities (https://en.wikipedia.org/wiki/Isoelastic_utility) with a parameter not too much greater than one. As a result, the ideal investments for most of us are near Kelly.
Second, there's a body of theory that asks what the optimal strategy for investment is given a certain number of years, and it turns out that the ideal strategy is to follow the Kelly approach until near the deadline and then to switch to the investments that maximize your own utility as the time to act grows short. This work is known as turnpike theory for reasons that never really made sense to me.
There are still concerns about Kelly investments, but Samuelson's argument isn't really one of them.