It is legal to show up at a port of entry without any sort of anything. If all it allows you to do is show up at the port of entry it's a worthless scrap of paper.
Commercial carriers such as airlines are also not allowed to transport people who need ESTAs/visas and do not have them to the US, so in practice it's relatively difficult to even get to the port of entry without a visa.
The visa is immaterial to the fine unless it conveys something beyond merely allowing you to present at a port of entry, which is allowed to basically everyone.