One scenario, and I think what people were paranoid about, is that you could make your cuts really arbitrary, and select a region where you by accident have too few simulation events and too many data events - say "Select all events where the first muon has momentum > 15.6 GeV, but not between 20.1 GeV and 20.3 GeV, unless there are 3 jets, in which case the selection is...". You just try out so many regions until you find something. For this reason you need to keep every step of your analysis justifyable, and a BDT is somewhat more explainable than a neural net (it's essentially just a trained list of simple cuts). Another thing that is frequently done is a blind analysis, where you only look at the real data when everything is finalized and locked in.
There are other issues beyond cherry picking that determine which methods you can use. E.g. the simulation doesn't reflect all correlations between variables, and it doesn't simulate the hardware triggers perfectly - so your simulations might behave different wrt. certain cuts than your data.
Nevertheless, it is not like you are banned-banned from trying different methods. When I was in particle physics (ATLAS and D0), we tried all kinds of creative things including NNs. But you have to convince hundreds of collaborators to put their names under your paper and you have to pass internal peer-review, which can feel a bit conservative and frustrating.
Why did CERN require so much selectivity? Was it a lack of funding? Are particle collisions particularly expensive? Are you competing for "collider time"? I am curious what the limiting factor was.
I think I would lose my mind if I needed hundreds of collaborators to support an experiment.
It was perfectly fine to work on any crazy ideas you wanted to. But if you wanted to publish as part of the CERN collaboration (and if you use CERN non-public data and CERN infrastructure, I'm not sure it was possible to publish as an "independent" anyway), there is a lot of protection to ensure that your article is at a good scientific standard.
As explained in the previous comment, scientists were nervous about "black box spurious pattern", "experimental artifacts" or "elsewhere effect". This was a good thing. The limitations were not there because of budget or access to infrastructure, but because people were conscientious of doing good science.
Remember the NN examples where the NN is given pictures of horses and pictures without horses and had to classify them, and it turned out one criteria was to detect the signature, in the bottom right, of the photographer that specializes in horse pictures? It meant that if you were giving a new picture of a bird but with that signature, the NN would say "it is a horse". Now, how do you make sure that something similar does not occur when treating billions of collision events? The categorized data, especially for the signal, is coming from simulation. What if your NN detects that in the signal, the pixel number 231 is not "hot" like the one in real data (because it turns out that, as sensor can be sometimes, this pixel is hotter in the real detector than it is in the simulation), and uses that to select the signal? Or that the NN overfits part of the data (the majority is classified on sensible cuts, but for more extreme cases, it classifies as signal events where there is an electron with pT between 24.5678 and 24.5685, but as background if it is before or after, because it happens to be what the training data looks like)?
Because of that, there were a lot of cross-checks and precautions being taken. One common approach was to compare more black-box-y methods with other methods that are more easily explainable. That was certainly a good thing. And this was not being seen as a chore: no one was approaching it as if people being good scientists where being personally against your new method. Proving that your new method was checking all the boxes of the scientific qualities was not seen as a bad thing, it was seen as the fun part of building new robust scientific advances.