Interesting, admittedly, I am slowly getting to the point, where ollama's defaults get a little restrictive. If the setup is not too onerous, I would not mind trying. Where did you start?
Assuming you have llama-server installed, you can download + run a hugging face model with something like
llama-server -hf ggml-org/gpt-oss-20b-GGUF -c 0 -fa --jinja