You can use llama.cpp, it runs on almost all hardware. Whisper.cpp is similar, but unless you have a mid or high end nvidia card it will be a bit slower.
Still very reasonable on modern hardware.
If you build locally for Apple hardware (instructions in the whisper.cpp readme) then it performs quite admirably on Apple computers as well.
Definitely try it with Ollama, it is by far the simplest local LLM tool to get up and running with minimal fuss!