Bee colonies are fragile. If the bees so much as 'sense' a disease, seeing the bodies doesn't matter specificly because an infected bee generally leaves the colony to die, there can become a mass exodus as they may believe there's disease in the actual colony. This presents a problem if you're continually losing your bees due to moving your colony, because if you get unlucky you could trigger an exodus.
What's more bee's follow a small oligarchy, each of which hold the ability to trigger a swarm. Let's call these 'Simon's' because they're the ones who relay the queen's orders. So what happens when the bee's believe Simon says 'leave' because he went missing? These oligarchs have the ability to trigger a swarm to found a new colony, so losing these few key bees is truly dangerous for a colony.
It sounds rather trivial, but these few bees can control a huge portion of a hive's population. These are the bees a hive needs to retain, just as much as the queen, but they cannot be contained like a queen (IE regular size not fat and need to leave the hive to trigger natural behaviours like gathering). The even bigger problem is that these bees cannot found a new colony, as all the females aren't sexually mature.
Another problem is that a new queen has a mating flight, often multiples. These queens are born to replace the existing queen, which if this queen gets lost (because some moron human moved the hive 50km) it will found a new hive instead of taking over the existing hive. This means if there isn't enough time to prepare, the hive can literally die off because it's incapable of preparing a new queen in time (it takes ~3 weeks from the egg being laid to the mating flights).
Bees are an agrarian society, and if the towns keep moving the farmers will never find it. The problems are really as complex as the questions as to why a town becomes abandoned, so really it's amazingly complicated and we as humans are screwing around with it.
Solution: The best solution is likely to maintain static hives on farmland instead of relying on mobile hives. It may be less cost-effective, however it is likely much more sustainable. I know if I was a farmer, I'd be trying to provide support for my own bee hives to form.