With respect to LDA, there is a new paper on arxiv, Two SVDs Suffice: Spectral decompositions for probabilistic topic modeling and latent Dirichlet allocation [1].

[1] http://arxiv.org/abs/1204.6703