IMHO, this would be a perfect application to build using Elixir and Phoenix LiveView. I think it would provide really robust realtime capabilities, and fits well with things like binary pattern matching that Elixir and Erlang handle well.
I agree with you on LiveView as well. I believe it would prove to be more reliable than client-side JavaScript.