You can turn any switches into ghetto load balancers by running BGP on some hosts and advertising a /32 into your switches.
You want BGP on your edge router, where your transit connections, these are what do the ECMP towards your LBs which speak iBGP. I'm not sure where you would use a cheap switch in this setup, certainly not at the edge.
Many recent data centers run eBGP between all switches. I can't explain the rest without diagrams.
Huh? Then you must not understand what you are talking about very well. Are you talking about running BGP to ToR switches? What does that have to do with ECMP or load balancers? Its just an L3 design and its not new.
I think they mean hashing algorithm for cumulus ECMP routing, that serves downstream LBs