It uses libpam-google-authenticator, looking at the source for google-authenticator.c[0] (line 156-7) it appears to use the Google Chart Infographic[1] API (which as a side note has been officially deprecated).
Can't quite work out how it is displaying it, I'm not competent enough with plain old C, but I'm sure someone can follow the code and chime in.
EDIT: Looks like it uses libqrencode and if that's not found it just uses Unicode block elements for actually displaying it from Google's API.
[0] http://google-authenticator.googlecode.com/git/libpam/google...