You'll still need to re-tune your prompt and the hyperparameters when switching models. So the actual effort of switching models is not improved much if at all.
This. I'll still use langchain for token/cost counting and some nice abstractions on top of the LLM, and the document loader system is semi-useful, but all of the retriever/chain stuff abstracts away the most important part - the prompt.
I've tried to switch out prompts between LLM's and I've had to change every bit of code provided by laying chain with a different implementation between them. It is an entirely useless abstraction. The prompt template is not at all transferrable.
Hopefully as LLMs get smarter, you'll need less prompt engineering and stuff will "just work" across models.