But how does "exit" by itself print that pretty little message telling you that, although it's obvious you wanted to exit, it's not going to let you? It's not just a function reference if it shows up that way.
In CPython, exit is actually an object of type site.Quitter (the actual class is implementation dependent). site.Quitter overrides the __repr__ special method (which is called by the interpreter on any expression typed in the shell to print the result). site.Quitter also overrides the __call__ special method, so when an object of type site.Quitter is called, the overridden __call__ method invokes system exit.
>>> exit_class = type(exit) #gets a reference to the class
>>> my_exit = exit_class('bye') #the arg is used to print the message
>>> my_exit
Use bye() or Ctrl-D (i.e. EOF) to exit
>>> my_exit()
<python shell exits>
>>> class ExitFunc:
... def __call__(self): __builtins__.exit()
... def __repr__(self): return "That's how!"
...
>>> exit = ExitFunc()
>>> exit
That's how!
>>> exit()