Right now Deepseek's official hosting is cheaper than everyone else who can manage to run the model, including Deepinfra. I haven't seen any good hypotheses as to why other than their large batch size and speculative decoding.
Gemini 2 can purportedly do this, you can test it with the Spatial Understanding Starter App inside AI Studio. Only caveat is that it's not production ready yet.
I probably won't ever use this but the docs look really nice. Specifically: the header combines a blurry backdrop filter with something like `background: radial-gradient(transparent 1px, white 1px) center / 4px 4px` to make a background of transparent dots.
There are a lot of interesting options. Gemini 2 Flash isn't ready yet (the current limits are 10 RPM and 1500 RPD) but it could definitely work. An alternative might be using a fine tuned model - I've heard good things about OpenAI fine tuning with even a few examples.
Interesting eval but my first reaction is "using Mixtral as a judge doesn't sound like a good idea". Have you tested how different its results are from GPT-4 as a judge (on a small scale) or how stuff like style and order can affect its judgements?
The web tries to do a lot of things, even some of the stuff you named. There's a Bluetooth API in Chrome, server-sent notifications via service workers, and some attempts at VR support.