Check out Neanderthal[0], it seems to have support for some subset of sparse matrices and it's faster than ND4J[1] to boot.

[0]: https://neanderthal.uncomplicate.org [1]: https://dragan.rocks/articles/18/Neanderthal-vs-ND4J-vol1