The thing is there are derivatives that are sometimes non-linear, things with triggers and barriers. When some large enough fish has one of these (eg by taking the other side vs a customer) they have an incentive to move the rate in whatever way they can. Whether it's getting someone to submit a bad rate or sitting on an FX cross, it can be worth it.
Before I went full quant I was often looking at the screens manually. You'd often see at around the WM/Reuters fix that the price would move strangely. An unusually large move would happen with no apparent news. You'd get a rumour from a broker, but who knew how they knew? And quite often the move would fade after the window closed.
This would happen even in exchange traded options. If you knew the specifics of the settlement window, you would know when the price would go wonky. You wouldn't know who, but you knew that is wasn't a normal time in the market.
The LIBOR was a bit less obvious, only really clear to me in hindsight. Swaps don't have a common schedule like listed options, so there's a fixing every day that could be in someone's interest to influence. Same goes for FX, but since LIBOR is a bunch of opinions (as opposed to trades) it's not as obvious.