In theory, you could hack on the Tor client so that clients and servers did an SIP negotiation prior to connecting on the desired port. You'd then run something on the host which would act like a port-knocking daemon, temporarily allowing new connections on a port only in response to a request from the Tor client, and only to the SIP peer in that message.
(Or, Tor could just present itself like a network interface, giving each N-proxied-peer a virtual IP that changes whenever it regenerates its identity. "Hidden service" connections would be regular IP-to-IP communication. For "public" connections, exit nodes would need to be running SOCKS proxies, and then there could be an anycast IP address that picked a proxied-exit-node at random. Then you'd just set that as your plain-old SOCKS proxy in your browser.)
This is not at all correct. The IP address of the hidden service is masked in exactly the same way that its clients' IP addresses are. That is, the client and service connect across the Tor network to an client-chosen onion router known as the 'rendezvous point', through which they set up a shared circuit.
See here for more detail: https://www.torproject.org/docs/hidden-services.html.en
Or here for the technical specification of the hidden service protocol: https://gitweb.torproject.org/torspec.git/blob/HEAD:/rend-sp...
The point I was making was that the proximate node to the hidden service--the last one in the onion-routing chain--connects to its destination by using its public IP to talk to the hidden service's public IP. From the perspective of the hidden service, the node proximate to it in the onion-routing chain is a regular Internet peer, which is impossible to distinguish from any other regular Internet peer.
In the end, what Tor gives you is a proxy (to a proxy, to a proxy.) And, from the server's perspective, there's no difference between a proxy and a regular client. It can't tell, by the IP, that the client it's speaking to is a proxy. And because of that, you cannot, at the server-level, block non-proxied clients from speaking to you. Because you don't know which those are.
Thus it is absolutely fine to firewall off all inbound connections on the host running the hidden service, as it will only be making outbound connections - and even those are to a limited set of IP addresses as defined by the guard nodes it has chosen for entry into the Tor network.