For those who are interested in replicate the very same architecture in your environment, I am working on a similar solution. Please check it out
https://github.com/luizbafilho/fusis
It is a control plane for IPVS and adds distribution, fault tolerance, self-configuration and a nice JSON API to it.
It is almost done, but it needs documentation on how to use it.
Moreover, for those who like numbers, I did some benchmarks using a 16 cores machine with two bounded 10Gbit interface
Scenarios:
1 request per connection with 1 byte: 115k connections/s
20 requests per connection with 1 byte: 670k requests/s
20 requests per connection with 1 megabyte: 14Gbps
It scenario tests one specific aspect of the load balancer.