Pretty much. When I hear someone call their project a "startup" instead of a "business" I assume they're trying to scam the world.
If you want people to think of David to the incumbent Goliaths, something with a mission to introduce something new and breakthrough to the world, a conduit for the wild dreams of youth to come alive, call it a startup.
If you want people to think stodgy, established, nice stable place to work and pay your mortgage off for 30 years, but no changing the world, call it a business.
Like any branding, that will affect who approaches you and what opportunities you end up with.
Just look at YC. Only two YC backed companies have ever gone public - DropBox and PagerDuty.