Here's my attempt to explain the whole thing:
Almost all modern AI is a search problem; We're just trying to find a clever way to search the problem space.
Most supervised learning methods are a search problem for best programs (models) that explain the data; We're just trying to find a clever way to express the model and efficient algorithms to search the problem space.
Many popular ML algorithms (NN, SVM, etc.) are an optimization problem; you just want to find parameters to maximize some performance metrics. But you don't want to search all the parameters exhaustively. You want something that can be improved incrementally over time.
NN is popular because it's flexible (can have thousands of inputs/outputs) and can be built incrementally. The secret sauce is its performance metrics (loss function) being differentiable. This way, you can incrementally improve the model via gradient descent.
NN is popular also because in theory you can stack as many layers as you want, giving it more power to learn complex relationships. It's applicable to many tasks.
CNN/RNN/LTSM/Transformer are just variations of these. They have different structures, but all operating on the same principle: differentiable loss function and iterative training methods.