The unnecessary flood of multicast packets these Google devices send is causing your AP to drop to its lowest speed and broadcast those packets.
Its the equivalent of having an 802.11B client that only syncs at 1Mbps and is constantly downloading. There is no airtime left for anyone else!
Things you can try: Block 802.11G and lower clients (increases your minimum WiFi transmission speed), convert multicast packets to unicast (LEDE is apt to support that), or throw the chromecast in the dustbin!