Chemception/Chemnet do just that with OK performance. https://arxiv.org/abs/1706.06689 https://arxiv.org/abs/1712.02734