kamranjon

LM Studio (which prioritizes MLX models if you're on Mac and they are available) - I have it setup with tailscale running as a server on my personal laptop. So when I'm working I can connect to it from my work laptop, from wherever I might be, and it's integrated through the Zed editor using its built in agent - it's pretty seamless. Then whenever I want to use my personal laptop I just unload the model and do other things. It's a really nice setup, definitely happy I got the 128gb mbp because I do a lot of video editing and 3d rendering work as a hobby/for fun and it's sorta dual purpose in that way, I can take advantage of the compute power when I'm not actually on the machine by setting it up as a LLM server.

I'm not sure if you're just unaware or purposefully dense. It's absolutely possible to get those numbers for certain models in a m4 max and it's averaged over many tokens, I was just getting 127tok/s for 700 token response on a 24b MoE model yesterday. I tend to use Qwen 3 Coder Next the most which is closer to 65 or 70 tok/s, but absolutely usable for dev work.

I think the truth is somewhere in the middle, many people don't realize just how performant (especially with MLX) some of these models have become on Mac hardware, and just how powerful the shared memory architecture they've built is, but also there is a lot of hype and misinformation on performance when compared to dedicated GPU's. It's a tradeoff between available memory and performance, but often it makes sense.

I have always wondered if the neural engine could be used for training - pretty excited for part 3 of this to see if the juice is actually worth the squeeze

This is a great idea, but the models seem pretty outdated - it's recommending things like qwen 2.5 and starcoder 2 as perfect matches for my m4 macbook pro with 128gb of memory.

I thought this part of the write-up was interesting:

"This is, I think, in contradiction with the idea that LLMs are memorizing the whole training set and uncompress what they have seen. LLMs can memorize certain over-represented documents and code, but while they can extract such verbatim parts of the code if prompted to do so, they don’t have a copy of everything they saw during the training set, nor they spontaneously emit copies of already seen code, in their normal operation."

Can't things basically get baked into the weights when trained on enough iterations, and isn't this the basis for a lot of plagiarism issues we saw with regards to code and literature? It seems like this is maybe downplaying the unattributed use of open source code when training these models.

I'm really interested in using this but wonder if the unique architecture means that it will not be able to be converted to a GGUF and used by ollama or llama.cpp? I certainly would understand that the observability features would require some custom tweaks, but I'd just like to try it out on my local ai server (basically just ollama + tailscale) and see how it works as a regular model.

It would be pretty incredible if they could host an embedding model on this same hardware, I would pay for that immediately. It would change the type of things you could build by enabling on the fly embeddings with negligible latency.

I haven't quite figured out if the open weights they released on huggingface amount to being able to run the (realtime) model locally - i hope so though! For the larger model with diarization I don't think they open sourced anything.

I think it's driven from an understanding that being constantly connected is potentially unhealthy for ones mental health. Not saying that's 100% true but I think enough people feel it that they are looking for solutions in things like dumb phones, digital detox, purpose-built devices etc.

I understand your lament about people just being able to be bored, but I think this gets at something deeper, that when we aren't bored and do want to do something, we are often distracted because of the nature of smart phones.

I got a Mudita Kompakt for similar reasons, I can sideload the few apps that I really need. The one I use the most is Kiwix, as I've downloaded the entirety of english wikipedia, so I can basically look anything up (useful when reading) without requiring an internet connection. I also have signal on there. I really like it and it's my dedicated phone now.

Since switching to a dumb phone I've gone down the same path. I already was an avid photographer but I've added a typewriter and a nice pen/notepad which has gotten me writing again. I've also read more books than I've read at any other time in my life, it's really incredible how much time I was wasting.

I think one of the values of (what appears to be) AI generated projects like this is that they can make me aware of the underlying technology that I might not have heard about - for example WebTorrent: https://webtorrent.io/faq

Pretty cool! Not sure what this offers over WebTorrent itself, but I was happy to learn about its existence.

Asahi is one of the projects I support monetarily cause I really hope that one day I can run linux natively on my M4 max with GPU acceleration. They did an amazing job with M1 and M2 - great to see they are still pushing forward after the departure of Alyssa Rosenzweig, who did a lot of the work on the GPU support for those.

Edit: Here is their donation page if you're interested in chipping in as well: https://opencollective.com/asahilinux

I wonder if it was trained on anime dubs cause all of the examples I listened to sounded very similar to a miyazaki style dub.

I read the release but didn't quite understand the difference between a next-edit model and a FIM model - does anyone have a clear explanation of when to use one over the other? I'd love if there was a sublime plugin to utilize this model and try it out, might see if I can figure that out.

I have an Olivetti Lettera 22 typewriter and it's the perfect machine, just immaculately designed. But one thing that absolutely floors me and I have no idea how they did it - is they have infinite, programmable tab stops on a completely manual machine (no electricity). So you can set as many tab stops as you want, and then hit the tab key and it will jump between all of the stops in order. It's great for creating lists of things or for creating simple tables. How the machine is able to remember your settings, and allow you to jump between and clear you tab stops completely mechanically is just so cool to me and seems like a marvel of engineering.

That's a pretty crazy requirement for something to be "useful" especially something that runs so efficiently on cpu. Many content creators from non-english speaking countries can benefit from this type of release by translating transcripts of their content to english and then running it through a model like this to dub their videos in a language that can reach many more people.

It's worth noting that the code in question was also open sourced and permissively licensed by the original author as he stated in the thread[1]. I guess this isn't really about licensing at all, just the original author seems to think it was rude, and also doesn't want to accept any of the apologies that have been offered.

[1]: https://github.com/Modernizr/Modernizr/pull/684#issuecomment...

I'm curious if you would have the same opinion about code shared on stack overflow?

You posted the code to a public blog page, with no attribution in the code or request of attribution from others, no license, and seemingly intended to share it freely with the world.

Then you got an apology, and a second apology.

I'm confused about what you think you're owed?

The explanation makes perfect sense, the headers were obviously just copied with no malicious intent. What is it that is still bothering you about this?

"Prerequisites: ... intro-level math (e.g. derivative, gaussian)"

Anyone got recommendations for learning resources for this type of math? Realizing now that I might be a bit behind on my intro-level math.

Just wanted to maybe provide a slightly different perspective, but I recently went through this process of pulling back from being socially active and it was for more than just one reason.

I wanted to focus on my health, both mental and physical, this meant going to the gym every morning and making time to read and getting rid of social media.

I also wanted to reduce my consumption of alcohol which typically was fueled by social events and always seemed to throw a wrench in taking care of my health (hard to get to the gym in the morning when you were drinking the night before, and for me it was even after just 1 drink).

What I realized was that many of the people I was spending time with, they oriented their communal time around drinking and for me that's pretty detrimental to my goals. After pulling back from social activity, I've felt so much healthier, happier and optimistic about life.

I get the same exact phone calls as you're describing, and I generally weigh the events I'm being invited to with what the focus of the event is - if the goal of the event is to just get together at a bar, I don't go. I think many of my friends feel that I've lost my way, but it's difficult because I sort of see them in the same light.

What I do hope to do eventually is to cultivate some new friendships, because I am missing that social aspect of my life, but for now I've sort of got a good thing going and I'm not too concerned about rushing it into being.

I think they expire in March 2026 if the NVIDIA stock drops to $140 a share? Something close to that I think.

That's a good point, we had sort of the precursor to this already and yea likely driven by google themselves. It seems that every time incentives are aligned purely for profit we end up with situations like this where they inevitably run a good thing straight into the ground.

Does it seem, I'm not sure, ironic maybe? That the main example here is "An app that writes blog posts" - "Researches a topic and writes a blog post about it" - that the company who helped champion the network effects of the internet and surface truly useful search results is now helping to destroy that very same thing they built their entire business on?

I use the Mudita Kompakt specifically cause it allows sideloading so I can still have a few extras. Right now I have Kiwix and Libby. It works really well.

I have a $10 a month plan from US cellular with only 2gigs so I try to keep everything offline that I can.

Honestly it's mostly the news... so I draw the line at browser, I'll never install a browser, that's basically something I can do when I sit down at a PC. I read quite a bit and I like to have the ability to look up a word or a historical event or some reference from something I read using Kiwix and it's been great for that, just needed to add a 512gb micro sd card. And Libby I just use at the gym when I'm on the treadmill.

It'd be great if you could add it to Kiwix[1] somehow (not sure what the process is for that but 100rabbits figured it out for their site) - I use it all the time now that I have a dumb phone - I have the entirety of wikipedia, wiktionary and 100rabbits all offline.

https://kiwix.org/en/

I was interested to try Date.now() since this is mentioned as being the only part of the Date implementation that is supported but was surprised to find it always returns 0 for your microquickjs version - your quickjs variant appears to return the current unix time.

You may not have seen this part: "Tied embeddings: We now tie the embeddings between the encoder and decoder. This significantly reduces the overall parameter count, allowing us to pack more active capabilities into the same memory footprint — crucial for our new compact 270M-270M model."

At this point - you would think that cheaters could be detected on the server side by either training a model to flag abnormal behavior or do some type of statistics on the movement patterns over time - is a client-side anti-cheat really required?