sipjca

Contributions are always welcome! There’s a WIP diarization PR rn, and after it’s merged would love to have support if it fits well into the interface. And if not would love to figure out a good interface for it

Ha amazing, love to hear it

Yep, but I am in the process of also porting NVIDIAs Sortformer for multi speaker diarization as well :)

I’m not sure how many specific models will be supported as the library is more focused on transcription specifically. But the models which support diarization natively must be supported I think. And parakeet multitalker was the primary driving force for this change

Thanks! What an excellent question, I’m not sure I have a good answer. I kind of became an open source maintainer by accident as Handy became popular

Certainly I am very lucky that quite a few people donate to Handy, and also some people and organizations who sponsor the work I do

To be honest I just love contributing to open source and wish to continue to do so. So anyone who supports this is good to me. Organizations which believe in OSS and push it forward are typically most aligned with me

Of course you can always email me (contact@handy.computer) and we can discuss in more detail

Yes, I’ve put a PR up on pypi for extra storage for CUDA but it has not been accepted yet afaik

If there’s any issues or improvements on the bindings I would love help to make the DX the best it can be

It very much depends on the hardware! An M4 max is being compared against a Ryzen 4750U with an integrated GPU!

The M4 max has probably 10x the compute and memory bandwidth hahaha

Yep the latest version has support! Virtually all of the SOTA open models are supported by Handy including the streaming ones like

Nemotron Streaming

Parakeet Unified

Voxtral Mini Realtime

If something you want is not supported, open an issue on transcribe.cpp!

Thanks :)

More or less yes, for whisper.cpp, just trying to make local transcription more accessible to anyone building an app, etc

Hey, yep author and maintainer here! Certainly sponsors help and the wonderful community who donates to Handy as well! Mozilla AI was very helpful in getting this work off the ground. It was a pipe dream for me to build for Handy and they helped to sponsor me so I could make time to take this project seriously and get a v0.1.0 release out the door

I agree this should be everywhere and I hope to distribute libtranscribe some day properly so it is more a system library! It will take time to stabilize but I think we can get there

Hey! It’s actually in progress right now, probably will come this week :)

Hey! It’s actually in progress right now, probably will come this week :)

Yes

for real

Hasn't pretty much everyone from Nuvia left QC at this point?

I’ve been doing something similar with less dedicated workflow and generally works great

fwiw because of the relatively few activated params offloading to system RAM is quite feasible, you can see the endless amount of people doing this on r/localllama with qwen3.6 35a3b

fair enough, i guess minimizing that surface area is important to begin with

imo being digital native means that migrating to any machine should be basically trivial. working with the flow of the machines rather than customizing and ricing them because your a cool computer person or whatever

i just want my computer to work. any config i have on my machine can be rebuilt by just doing the work i need to do.

my primary work machine was stolen last year so i was forced to go through this quite literally with a new machine rather than hypothetically or by my own will

im more surprised that more people don’t treat their computer as disposable anyway.

that it could just be wiped at any moment and it wouldn’t matter. shit happens, could be stolen, broken, whatever. the computer should be able to be thrown out the window and continue to live life.

to be clear, i don’t think upgrading and disposable in this way is good, but it being wiped at any moment shouldn’t be a concern

i grew up wiping my machine every year anyway, so i guess it’s just a habit

is the computer that sacred?

inference code is effectively trivial to port at this time

everyone understands cuda well enough anyway

thats incredible

Wondering similar. It certainly can run beyond 30 seconds but at some point I believe the output should degrade

Plus you could do actual batch inference instead. Or if you must carry forward the context you could still do it linearly, but the mem usage shouldn’t just explode

Thank you!!

I just don't have the bandwidth to run another project, maintaining Handy is hard enough on it's own, especially for free!

I didn't just dismiss for no reason, I am a human! I have needs and I can't just sleeplessly stay in front of the computer putting out code. If I had more time I would, but alas.

Someone could easily vibe code an iOS version in a few hours. I could do the same but I do not have time to support it.

Of course they are! Both are important and will be around and used for different reasons

I don’t think it’s about literally shrinking the models via quantization, but rather training smaller/more efficient models from scratch

Smaller models have gotten much more powerful the last 2 years. Qwen 3.5 is one example of this. The cost/compute requirements of running the same level intelligence is going down

This is an argument, but it’s also fundamentally comparing a computer that works out of the box to one that doesn’t.

That wasn’t the point. You’re a person who runs arch, that means most likely your requirements for a computer are VERY different than the target for this Mac. There’s always some other computer you can buy, but most people will just buy the Mac

Yes