If you are interested in this kind of thing, there are a whole host of "derivative free" optimization methods out there. One I've seen used in particular at a previous job, the Nelder-Mead (downhill simplex) method, always struck me as pretty elegant and intuitive once you've seen it explained. https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method)