I'm not certain it has anything to do with the physical world; after all Industrial Engineering is considered a valid disciplined that is entitled to an Engineering license, and many of the IE's I know might be better characterized as mathematicians.
Still, I tend to agree with your assessment of the term Engineer, although it's largely because unlike other Engineering fields, Software Engineers in the US seem to be happy to hold the title without having to meet any of the legal requirements [1]. A PE license, while not insurmountable, is not exactly the easiest title to acquire. Efforts have been proposed to make Software Engineering part of this licensing process, but it seems to have a lot of push back. Perhaps the solution is to have legal weight (perhaps like 'Lawyer' or 'MD') behind the title, as countries like Canada have (the US has this to some extent, but it's not nearly as strong).