first time i’ve found a fellow manitoban on HN greetings !
401 karma · joined January 12, 2020
The moment I received my first packet on a cut-up wired headset I used as a transceiver makeshift tenceiver it felt like something clicked and I just began to understand more how the universe works. I recommend projects of this type and wish more folks did it.
I've ran a ML hosting platform (banana.dev) last few yrs.
Not being able to get GPUs has stressed me out a lot. So took a crack at solving it.
This is a super alpha product, please don't trust it with sensitive workloads or production applications yet. Down to chat if you'd like to use this at scale!
Would love honest feedback, and feel free to AMA! -kyle
Bit stuck in progress following Vercel template here https://vercel.com/templates/next.js/tier
But I've already built out my own stripe usage-based billing app b4, and it sucks way more to debug that, so I get what you're trying to do here.
hey HN, we use used to run an ML consultancy for a year that helped companies build & host models in prod. We learned how tedious & expensive it was to host ML. Customer models had to run on a fleet of always-on GPUs that would often get <10% utilization, which felt like a big money sink.
Over time we built infrastructure to improve GPU utilization. Six months ago we made a pivot to focus solely on productizing this infra into a hosting platform for ML teams to use that would remove the pain of deployment and reduce the cost of hosting models.
We deploy on A100 GPUs, and you pay per second of inference. If you aren’t running inferences you pay nothing. Couple points to clarify: Yes, the models are actually cold-booted, we aren’t just running them in the background. We boot models faster due to how we manage OS memory. Yes, there is still cold-boot time, it’s not instant but it’s significantly faster (e.g., 15 seconds instead of 10 minutes for some transformers like GPTJ).
Lastly, model quality is not lost on Banana because we aren’t doing traditional weight quantization or network pruning which makes networks smaller/faster but sacrifices quality. You can think of Banana more as a compiler + hosting platform. We break down your code to run faster on GPUs.
Try it out and let us know what you think!
We're building toward a GPT3 level moment in computer vision, and here's our V0 (demo video linked).
It's called Carrot. Request access here: https://banana-dev.typeform.com/carrot
We are starting with a Visual Question-Answer model, and plan to expand its capabilities to be increasingly general purpose over time as we build in common CV features and upscale the quantity of parameters.
This is a hybrid of vision and language models, which can extract semantic meaning from images and query against it using natural English. This v0 runs on 13B parameters, with 18B and 34B model iterations coming in the pipeline.
The API is in beta, so jump into the waitlist linked above to get early access.
Relevant project (mine): https://audiblogs.com (lets you listen to any blog posts in your podcast player. It's like Audible for blogs.)
Let's you listen to any web article in your podcast player
Could you elaborate on whats missing from Pocket? And 2) Is there anything on your mind we could add to make audiblogs 10x better for you?
And def thinking about firefox support, thanks for the input.