I'm writing a blog post with some more reasoning but my view is that it can be useful for certain simpler tasks (eg unstructured -> structured, basic summarization) and not more complex things (eg generation).

The tricky thing is that finetuning makes a big difference, and while it should be possible to hotswap LoRA adapters (at some cost to performance), I haven't figured that out yet.

Not as good as GPT-4 of course but nor far from 3.5 if you just need to reword whatever returned by the retrieval. It's like losing 20 IQ points which might be still better than most support interactions I had.