For this, you're perfectly correct.
> It’s not brain size that limits them but how their brains develop with use, much like how muscles grow with exercise.
Here, the anser is yes, but like for muscles, biology will create constraints. If you're male, you may be able to bench 200kg, but probably not 500kg unless your biology allows it.
> If brain size and calorie consumption directly drove intelligence, we’d expect whales
As you wrote later, there are costs to developing large brains. The benefits would not justify the costs, over evolutionary history.
> Brains can large like arms can be large but big arms do not necessarily make you strong -- they may be large due to fat.
A chimp has large arms. Try wrestling it.
> and factors like brain size follow from that
Large brains come with a significant metabolic cost. They would only have evolved if they provided a benefit that would outweigh those costs.
And in today's world, most mammal tissue is either part of a Homo Sapiens or part of the body of an animal used as livestock by Home Sapens.
That's evolutionary "success" for you.