sipjca

Isn’t it that hardware catches up and becomes cheaper? The margin on these chips right now is outrageous, but what happens as there is more competition? What happens when there is more supply? Are we overbuilding? Apple M series chips already perform phenomenally for this class of models and you bet both AMD and NVIDIA are playing with unified memory architectures too for the memory bandwidth. It seems like today’s really expensive stuff may become the norm rather than the exception. Assuming architectures lately stay similar and require large amounts of fast memory.

it seems like the value is that you don't need another tool to composite the text. especially for users who aren't aware of figma/photoshop nor how to use them (many many many people)

ty! if there's any way I can help just lmk, always happy to lend a hand or an ear

ive approached the same thing but slightly differently. i can run it on consumer hardware for vastly cheaper than the cloud and don't have to worry about image sizes at all. (bare metal is 'faster') offering 20,000 minutes of transcription for free up to the rate limit (1 Request Every 5 Seconds)

https://geppetto.app

I contributed "whisperfile" as a result of this work:

* https://github.com/Mozilla-Ocho/llamafile/tree/main/whisper....

* https://github.com/cjpais/whisperfile

if you ever want to chat about making transcription virtually free or so cheap for everyone let me know. I've been working on various projects related to it for a while. including open source/cross-platform superwhisper alternative https://handy.computer

No problem! Thank you for trying it and requesting features!

I added a issue (and comment) for this on the GitHub repo: https://github.com/cjpais/Handy/issues/47

yep I think this is the fundamental question as well, everything else is intermediate

Been working on https://handy.computer. It's similar to SuperWhisper and others, but is open source and cross platform

It uses whisper.cpp under the hood and should be accelerated on most devices using the Vulkan backend

as above, open source and cross platform, should work on Ubuntu

https://handy.computer

Open source and cross platform one: https://handy.computer

I believe the 1000x number I pulled is from SemiAnalysis or similar, using MMLU as the baseline benchmark and the cost per token from a year ago to today at the same score. Model improvements, hardware improvements and software improvements all make a massive difference when combined to make much greater than 10x gains in terms to intelligence/$

somewhat hard to say how the cards fall when the cost of 'intelligence' is coming down 1000x year over year while at the same time compute continues to scale. the bet should be made on both sides probably

but are they? on average? how do you measure this?

it's pretty easy to talk to a solo-dev or gardener

Because that is way less clean of a solution and would be a total pain to carry around

I’m the author of the post and will also be releasing an open source speech to text app in the coming weeks. It’s what I’ve been using for months, but packaging it for people

It’s going to be extremely simple, and hopefully easy enough to use. MIT licensed, free

Can you tell me more about the "anonymous federated score sharing"? Maybe something we can think about more

I totally agree with Ollama integration and if there is interest we will try to upstream into llama.cpp

Right now the main priority is just getting the data out, but in the future may have some interest in this. Or perhaps we can open an API for others to build this as well

I think you might be right, definitely interested in this feedback and creating charts and graphs that are the most useful for folks!

It's kind of just an artifact of the development. Happy to switch the default models and sizes in the future especially based on community feedback

LocalScore dev here

Thank you! I think this is quite possible! If you don't mind starting a discussion on this I would love to think aloud there

https://github.com/cjpais/LocalScore/discussions

LocalScore dev here

Llamafile could certainly be released without the GPU binaries included by default and it would slim down the size tremendously.

The extra 70MiB is that the CUDA binaries for LocalScore are built with CuBLAS and for more generations of NVIDIA architectures (sm60->sm120), whereas Llamafile is built with TinyBLAS and for just a few generations in particular

I think it's possible to randomize weights with a standard set of layers, and maybe a possibility for the future

Ty for pointing this out, the results are taken from the db based on LocalScore, I will make some modifications to make the sorting better here

adsb.lol airplanes.live

adsbexchange was also bought out by private equity recently, these two seem to be some of the larger fully community driven resources.

I mean the whole company is betting on AI, why wouldn’t they use AI to generate the image?? Fundamentally it doesn’t matter if it was AI generated or not, most people don’t care and the people that do won’t impact their bottom line

lol what models are you using? I just tested a handful (down to 1B in size) and all gave beef

It’s still not great. I drove I5 North for the holidays and at one stop there was an hour+ long wait at Electrify America. Only 3/4 stalls working and a queue 5 or 6 deep to charge. At the same stop there were 12+ Tesla chargers and no issue with wait and not fully occupied

I am very interested in how small these arrays can be. From talking with a friend with cochlear implants, I would assume this could help dramatically with the right signal processing to help him hear.

it is not, at least in llama.cpp/llamafile

https://benchmarks.andromeda.computer/compare

Love this! Have been doing something similar with HAProxy + Cloudflare Tunnels, but would love to move off it at somepoint. Super curious to give it a run soon. Thanks for sharing!

I love this site, I’ve used it countless times. So useful and super quick to see results and just understand weird svg elements

Thank you to the dev

fwiw as a consumer I have a tenstorrent card in my machine