An inference server for AMD GPUs, in C++ and HIP

Just posting this here as it helped me get incredible performance out of my 2x r9700 setup that I didn't think were possible when using llama.cpp and vllm