I wonder if a two step process could work better than this, first use a variational autoencoder or simple an autoencoder then use it to train the labeled sampled.

In (1) there is a full example of using the two step strategy but using more labeled data to obtain 92% of accuracy. Someone can try changing the second part to use only ten labels for the classifying part and share results?

(1) https://www.datacamp.com/community/tutorials/autoencoder-cla...

Edited: I found a deep analysis in (2), in short for CIFAR 10 the VAE semi-supervised learning approach provides poor results, but the author has not used augmentation!

(2) http://bjlkeng.github.io/posts/semi-supervised-learning-with...