Not old enough for the dotCom (36) but my educated guess is that because the internet was new and exciting, companies were prepared to train and take risks.
Training for AI now requires specialist skills which can be syphoned from within an existing organisation or from those with existing skill sets.
Unlike the latter where anyone can pick up a programming language. Not everyone can do quantum algebra mechanics or whatever AI/ML uses.
Plus the cost of running AI is expensive. SME's don't have the resources to hire those for new jobs; AI kit and training. So it's easier to hire from within then outside.
AI is a rich boys toy.