word2vec yields better representations than PMI-SVD. If you want a better explicit PMI matrix factorization, have a look at https://github.com/alexandres/lexvec and the original paper. Explains why SVD performs poorly.

If you are looking for word embedding for production use, checkout fasttext, lexvec, glove, or word2vec. Don't use the approach described in this article.