It's not only about the resources allocated to your project. Bigger images also leads to longer deployment times, which increases the feedback loop for issues. Not to mention increased costs associated with storage, bandwidth, layer caching, etc.
I'm quite opinionated here, but these things only matter at significant scale. Most one-person-SaaS or SMEs never really graduates out of the first pricing tier for this stuff.