They are, and if you flip one hardware switch inside you go into diagnostic mode and can print out the IP and all config information and trivially reconfigure without any sort of security or logging. The days of expensive dedicated lines are long gone. Most ATMs run off of the bank location's LAN, which given the quality of regional banking IT is not secured all that well.

A lot of the machines I worked on in remote locations (in highrise offices away from bank locations) were hooked up through the building's wifi via a wireless router sitting unsecured on top of the machine, just out of sight. I mean, you have easy physical access to the connections in that case.