597 karma · joined February 6, 2021
Contact: me [at] anu.ninja
I was able to fit the model with decent speeds (30 tokens/seconds) and a 20k token context completely on the GPU.
For summarization, the performance of these models are decent enough. However unfortunately in my use case I felt using Gemini's Free Tier with it's multimodal capabilities and much better quality output made running local LLMs not really worth it as of right now, atleast for consumers.
I've used Moonlight + Nvidia Gamestream with ~40ms RTT and couldn't feel a difference in competitive shooters, so total latency must be pretty low.
Does it have something to do with the bandwidth requirements? (1 stream v/s potentially hundreds)
1. Recursively lookup DNS, so domains will have to be blocked at the registrar level, since DNS is unencrypted, it can be blocked at ISP level as well.
2. Use a protocol alternative to DNS, a good mature example is GNS. It aims to replace DNS, with a built from group up, modernish protocol. Using a DHT and public-key cryptography.
3. There are "block chain" solutions to the whole domain problem, look at Handshake, ENS etc.
And it doesn't expose ADR/carrier phase.
1. https://learn.microsoft.com/en-us/previous-versions/technet-...
that does little to counter the real problem, as the chance of successfully guessing the pin on first try is still 1/1e5, which gives ~69k attempts for a 50% chance of correctly guess that pin, Which is like ~2 hr at 10 pin attempts/second. Having request throttling helps tremendously, but shouldn't be the only deterrence in place. Moreover it should be implemented in a way that it does not become a way for DoS attacks.
reminds of this particular xkcd [1], How someone fucks up that bad is beyond me.
And even after that won't the company itself have to start putting telemetry and stuff themselves since prexisting maintainers definately won't do that. It feels pretty much like a Hostile takeover than acquiring. How do they even benefit from it
However on the bright side performance.now() has a much worse resolution on most browsers, making timing attacks harder.