Nah, it's not that simple.
Work is an amazing place to bond with people, if you all enjoy the work and find value in it. I'm still friends with a bunch of folks I met in 2007 when we all managed the 3rd level infra for a bank. We meet for beers, we go to each other's houses, we're best men at each other's weddings. Just yesterday evening I inter-continentally called one of them for a catchup.
Can it be toxic? Of course! Is it always toxic? Don't be silly.