They actually have a set of files and libraries that they ignore (.DS_Store/jquery/boostrap/etc)

https://github.com/github/linguist/blob/master/lib/linguist/...