I like TensorFlow with Keras much more than using DeepLearning4j, but DL4J is callable from Armed Bear Common Lisp. I have simple examples in my book that you can read free online: https://leanpub.com/lovinglisp Unfortunately the example in the book is a little weak because I wrote the deep learning code in Java and just called my Java code from Common Lisp. I should modify the example to build models, train, and evaluate using the DL4J APIs directly from Lisp.
EDIT: in the book example, I just call train and evaluate entry points in my Java code from Lisp so I can train new models from Lisp but can’t create new models with new architectures without modifying the Java code. I really should change that.