Having static IPs on your management devices on a separate subnet and vlan from your regular host ports should mostly mitigate this.
I'm responsible for netsec at my company and I've never seen any indication of the BMC network activity on primary network ports after I've configured the BMC in BIOS to use it's dedicated network port.