Back then I modelled the quantum circuit as a set of unitaries (by parametrizing them through their generator), that operate on one or two qubits, set a limit to the amount of steps and the amount of controlled gates and then threw different optimization algorithms at it. I got the best performance using simple dense neural networks. What's cool is that I could generate a training set really quickly since I could just randomly build tensor products of unitary matricies to create billions of unitaries of up to 7 qubits in minimal time and then just see how close I can get given a fixed length for the quantum circuit and a fixed number of control gates.
I really liked this approach and it was fun to work on. However it was ultimately limited as the size of the matrices scales exponentially with the number of qubits.