Hi NMCA, I'm Raza one of the founders of Humanloop. I totally agree that transfer learning is one of the best strategies for data efficiency and it's pretty common to see people start from large pre-trained models like BERT. Active learning then provides an additional benefit, especially when labels are expensive. For example we've worked with teams where lawyers have to annotate.

In terms of breakthroughs in recent years, some things I'd point to would BALD (https://arxiv.org/abs/1112.5745) and its applications in deep learning as well. There has also been progress in coreset methods (https://openreview.net/forum?id=H1aIuk-RW).

I think that you're right that it used to be much to hard to get active learning to work. Part of what we're trying to do is make it easy enough that its worth the benefits.