It is not clear why they switched from GBT to ANNs. I would have thought GBTs would work better on the kind of structured data they are dealing with.
In the image comparing logistic regression and decision trees, it seems to suggest they only used linear features for logistic regression (?). Otherwise, shouldn’t there be a non linear classification boundary?
They write in the text that their model featured nonlinear transformations. I think the picture shows a simplified view and as the plots don't have labels it's not clear which space is regarded.