Feel free to reach out.
(currently at https://github.com/mmastrac/diffgemma but not in a releasable state yet)
50,078 karma · joined July 3, 2007
Playing with fun tech.
Mastodon: @mmastrac@hachyderm.io (https://hachyderm.io/@mmastrac)
Email me: matthew@mastracci.com
LinkedIn: https://www.linkedin.com/in/matthewmastracci/Feel free to reach out.
(currently at https://github.com/mmastrac/diffgemma but not in a releasable state yet)
FWIW I'm getting a hardware max of 20 tok/s (approx topping out the GPU's compute) on my custom local diffusiongemma port running on an M3.
I've been going smaller.. I have a custom-quantized Rust port of DiffusionGemma (26B) that seems to perform better (in responses) than benchmarks seemed to indicate and reasonably fast for its model size. Works really well on a 36GB mac as well for both prefill and generation.
It's been interesting learning about the balance of factors for performant metal kernels on unified memory.
Should have a repo up on github in the next few weeks.
The C2 IP (89.124.107.161) and malware-serving git repo (144.124.244.92) are both hosted on VDSINA in Russia, so not sure if there's anything to do there.
But I think the point is that they should be cheap to set up, and because of the short life, never really contain anything except the potential to compute when needed, not important data.
Luck surface area. I really owe so much to Om. I really can't imagine where I would be without that chance.
IMO, Deno has always been more methodical, more focused (maybe too focused?) on standards. But now the Deno team is on the right track: using Claude extensively to improve the node.js compat which was absolutely herculean if not impossible before AI. [+]
On the other hand, Bun has always played a bit fast and loose, chasing metrics at the cost of stability. Access to abundant AI has sent that project off the rails.
Disclaimer: Former Deno engineer - I'm obviously going to have some biases. All IMO of course, but if you ask me I'd still bet on Deno in the long term, and I personally still use it for any .ts projects.
[+] There might be a dozen people in the world that know how sensitive and subtle the timing and ordering in the JS event loop is and how meticulous just this single part needs to be for major node.js projects not to completely crap themselves.
There's a decent amount of knowledge around pre-main work in Rust, but I think this is one of the first attempts to walk through mutable link sections, which open up a pretty wide world of optimization, IMO. Even without mutability, I figured there isn't nearly enough documentation on these approaches out there.
(1) https://docs.rs/scattered-collect/0.20.0/scattered_collect/m...
(2) https://docs.rs/scattered-collect/0.20.0/scattered_collect/s...
I wrote https://crates.io/crates/type-mapper as a way to work around those limitations but it is _very_ painful TBH.
I think I'd feel more comfortable if I could drop an .rs file into the eBPF dir instead of a .c one. It's already a Rust project! :)
And for some reason I was expecting this to be a kernel-accelerated webserver - if that could be done safely using eBPF that would be amazing!
Also, single-threaded? Forking and sharing an incoming connection queue is basically trivial on Linux, that should be literally just a few lines, even with Rust. Use SO_REUSEPORT and the kernel will do the rest.
FWIW, if you're going to push for io_uring, you should also be pushing kTLS IMO, you'll drastically simplify your design if you can avoid pumping userspace SSL after the handshake.
It's surprising how little information exists out there about proc_macros in general.
Complex proc macros absolutely do slow builds down. In many cases, a proc macro only need to be a stub that can delegate to a declarative macro.
You may not need to use syn/quote, but if you are doing any sort of processing/parsing of Rust code you pretty much need to.
FWIW, I really hope that the Rust project focused on finer-grained token matching in declarative macros so we can migrate most proc_macro code away. The macro system is powerful, but nowhere near where it needs to be.
In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...
This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?
And you're also assuming that all the pings are being returned by this box.
The AI popup is the worst and will hallucinate answers from Reddit comments. I specifically had it ask me a nonsense question which was literally just someone's Reddit comment suggesting a follow-on topic B to the search topic A. The AI mode will _sometimes_ be useful enough to prompt into doing the search and summarization for me and get me just enough info and some links to continue the work myself.
avi2ude? av2go?
I do not believe that engineers who are tokenmaxxing are truely productive and I have not seen any evidence whatsoever (perhaps the opposite).
I've personally found that with the right flow and codebase knowledge, that's achievable with sustainable levels of effort.