Hacker News Reader

Comment by mark_l_watson | Hacker News Reader

Great read. I have 30 years combined experience in symbolic AI and neural networks and my current day job is managing a deep learning team. I could not agree more with: “”I think it is far more likely that the two — deep learning and symbol-manipulation-will co-exist, with deep learning handling many aspects of perceptual classification, but symbol-manipulation playing a vital role in reasoning about abstract knowledge.””

My personal time projects are mostly combining symbolic AI and deep learning, but I am still trying to find a non-Python solution, including Haskell TensorFlow bindings, Armed Bear Common Lisp with DL4J, and exporting trained Keras models to a Racket environment - all plausible hacking environments but none feel ‘just right.’ If you are working on the same ideas please get in touch with me.

View on HN

I've just been trying out a few deep learning frameworks recently, and found the julia framework Flux to be enormously fun, flexible and generally just extremely hackable. If you like lisp you might also find it a pleasing language - even though it doesn't look it on the surface, there's a lot of lispy inspiration hiding underneath the skin.

Thanks Chris, I will look at Flux.

EDIT: make that a double thanks, Flux looks great: concise and expressive.

Here is an attempt in Clojure https://m.youtube.com/watch?v=phA4bMjKvCY

Thanks I am watching this right now.

EDIT: thank you so much, great talk! Even though the Juji system does not directly link TensorFlow models into Clojure, the architecture of using Kafka to manage low level black box model calls (they treat trained models as functions as I do in my Racket hacks) is a good idea. I also like the rule/template language they designed. Great work.

this also looks good: https://github.com/hswick/jutsu.ai

Amen. I'd love to find a way to access Tensorflow directly from a native compiled Common Lisp like CCL or SBCL. This would be ideal for combining symbolic AI with DL.

I would like the same.

I like TensorFlow with Keras much more than using DeepLearning4j, but DL4J is callable from Armed Bear Common Lisp. I have simple examples in my book that you can read free online: https://leanpub.com/lovinglisp Unfortunately the example in the book is a little weak because I wrote the deep learning code in Java and just called my Java code from Common Lisp. I should modify the example to build models, train, and evaluate using the DL4J APIs directly from Lisp.

EDIT: in the book example, I just call train and evaluate entry points in my Java code from Lisp so I can train new models from Lisp but can’t create new models with new architectures without modifying the Java code. I really should change that.

Hi Mark, I want to correct one thing. My company is behind dl4j. We are also the second largest contributor to keras. We have near perfect model import (with keras contributors maintaining it). Dl4j also has TF import and a native graph runner: https://github.com/deeplearning4j/dl4j-examples/tree/master/...

Java itself tends to be fairly verbose I know, but not only are there wrappers but for inference if you want to use python for basics and then us for inference. We enable several ways to do that. Folks tend to oversensationlize these "us vs them" framework "fights".

Rather than do that, why not just use what's good for each use case? Python for training, java for deployment?

Beyond that, you could also consider building your own wrapper maybe? In clojure, there's jutsu.ai for example: https://github.com/hswick/jutsu.ai

Thanks for the info! Sorry if I sounded like I was knocking DL4J: I have used it in examples for two books I have written.

I use Keras at work so that is why I prefer it - I am very used to Keras, and to a lesser degree TensorFlow.

Just tonight I was experimenting with jutsu.ai, and your advice of training on Keras and running on DL4J is good advice. I hacked up a little code to import Keras weights into Racket, but I can only process dense layers, and not RNN or complex architectures.

Thanks! Usually dl4j is targeted at data engineers, not python folks. We actually have the end to end system working.

Can you hint the problem with a python based solution? What doesn't feel right about it.

Pure Python is slow as a snail compared to both the underlying GPU code that has to run in deep learning applications, and the tuned, compiled code you would use for symbolic reasoning.

Naturally, I guess I thought there was something syntax related or otherwise based on the way he said "doesn't feel right"

Python is great for short scripts and programs, but I don’t like it for large systems.

Yes, I found it is easy to run into memory or performance problems when code size becomes larger. Especially when the system is multi-threaded.

Reply on news.ycombinator.com