I recommend using https://ollama.ai/ if you dont care about openai compatibility.