The general setup seems to be that you have a public facing LB which just forwards all incoming traffic to any available fabio instance which handles the dynamic routing. fabio does not solve the public VIP problem for you.
Disclaimer: I'm the author.