1 karma · joined August 17, 2024
We’ve all had to open a door with our hands full and thanked the wheelchair button by the door, or tried to watch a movie with the flu and a stuffed head so we snapped on CC.
The main problem is that many teams think they’re “that guy (or girl)” who can manage a complicated piece of infra as a small team and will be saving money. If you spend 10% of your time doing devops work, you spend 10% less time on your actual product (you might as well be binging netflix). A majority of these teams convert (IME) to a hosted offering once the right price and SLAs and DevExp are there. The only part of your sales pitch that matters to these teams is “this isnt worth your time to self host when we have a great offering thats cheaper and faster to get setup”. If you have a logo from a big co that has the talent to self host but are buying your product, then your work is done.
I know smart people work there, some friends included, but OpenAI was never a product-led startup, they were a research org. Ignoring the closed vs open debate for a moment: for all the talk of focus, focus, focus from YC, it seems unfocused to ship their models from a home-rolled platform when they could just get MS to foot the engineering and infra work. Is any of their platform stuff a real asset to OpenAI proper? Have MS do the cloud stuff, just pass them product design.
just my 2c from the outside.
Fyi me and a friend made a demo app of tinystories we just submitted: https://news.ycombinator.com/item?id=36960333 - from testing it out, anything with <500ms latency and 15 tok/sec feels real-time, and that makes a world of difference in UX
After reading the TinyStories paper (using small language models of 1m to 33m parameter size to generate short stories), me and a friend decided to port it as a test to see what performance of tiny LMs look like on mobile devices. The stories it generates aren't great or anything, it's really just a toy tech demo to play with. We used HuggingFace's exporters library to do this.
You can try it out here: https://apps.apple.com/us/app/tinystories-on-device/id645149...
(we are not affiliated with MS research or the authors)
- they want to make benchmarking easier by using AI systems
- they want to automate red-teaming and safety-checking ("problematic behavior" i.e. cursing at customers)
- they want to automate the understanding of model outputs ("interpretability")
Notice how absolutely none of these things require "superintelligence" to exist to be useful? They're all just bog standard Good Things that you'd want for any class of automated system, i.e. a great customer service bot.
The superintelligence meme is tiring but we're getting cool things out of it I guess...
Descendant models = models fine tuned from an ancestor on one particular domain, e.g. by partitioning your training data by subject or source
Democracy = some weighted mix of the descendant models is used to find the next token
I was thinking of a VS code extension that lets me put files or directories “physically” on my desk. Itd be emulating the way that I read text books. Instead of keeping notes and bookmarks physically in the book I could drop sticky notes that represent files directly on my desk and show a preview by by just shifting my eye focus to the AR/VR note.