One drawback is that it doesn't follow the standard $MANPATH directory structure (man/manX/Y.X where X is the section number and Y is the manual page) but instead uses a single flattened directory (man/Y.X).

I filed an issue about this here: https://github.com/defunkt/gem-man/issues/8