It makes (business) sense to build your own gear when you operate at scale. Typical use cases are TORs and small table routers used for data center Clos topologies. Both often make use of merchant silicon which come with SDK's.
I mean, these ToR switches really are super cheap if you buy direct from ODM. For designing your own ToR, you'll need at least a team of 5-6 people each making $400-500k annually at a company like Google or FB. How much is the cost savings per ToR switch in capex? $10/switch? In opex? I think it might make opex sense.
You'll build and run your own mgmt plane OS for sure, but that's tangential to building your own hardware.
Control-plane/management-plane software is often written by these companies or by contractors.
NICs are another story. MSFT and AMZN both build their own NICs using either FPGAs or custom silicon.