It's not 'AI' (what actually is?), but mostly hand-crafted state-machines trying to achieve game-specific goals, which directly inspect the current game state instead of trying to build their own 'world model' (of course with restrictions like fog-of-war, visibility-checks and so on so that the cheating isn't too obvious).

Really no that much different from what the Pacman ghosts did, just developed further from there (especially during the Golden Age of strategy games during the late 90's and early 00's).

With this base, scalability is achieved through fairly traditional performance optimizations (mostly using the right algorithms and data layout, especially for path-finding and 'visibility checks').

Also in strategy games, AI is often layered like a military chain of command, where the "commander AI" only makes high level strategic decisions and only has a very 'sparse' world model, while the lowest level 'soldier' AIs are mostly occupied with pathfinding but only know about their immediate surroundings.

Also, each game genre has their own highly specialised, hand-crafted "AI" algorithms. A first-person-shooter AI is completely different from a car-racing-game or realtime-strategy game.