What other methods are there for training a neural network? Even blind evolutionary algorithms (assuming each generation is initialized to be somewhat 'near' or 'similar' to its progenitors) are effectively descending a gradient.
How can you perform a GD on non continuous function? Genetic algorithms and evolutionary methods can explore disjoint domains and functions hard from analytical pov.
Just because they can be used for gradient descend doesn't mean they use it.