Quick shout out for DyNet! The API is so simple and intuitive even a two year old (a prodigious one) can use it.
http://dynet.readthedocs.io/en/latest/index.html
If you're doing the sort of work that would be published at ACL or EMNLP, DyNet is a really good choice.