Some people may be willing to travel but don't know where the best place to go is.
287 karma · joined December 22, 2023
These smaller models are a relatively cheap to run, especially in high batches.
I'm sure there will also be aggressive rate limits.
Also because of their unified memory design, they actually have insane bandwidth which is incredibly useful for LLMs. IMO they may have a head-start in that respect for on-device inference of large models (e.g. 1B+ params).
However much of this social impairment may or may not be a real problem. "special interests" and "obsessing over one topic" is an impairment in social scenerios, but can be extremely beneficial for tasks related to that special interest.
The irony in this is magical.
I highly suggest you remarket as a collection of some type of link a la product hunt. Tech startups, cool apps, anything.
"backlink service" is literally just "seo spam" on it's own.
From what I could tell from Nvidia's recent presentation, Nvidia works directly with OpenAI to test their next gen hardware. IIRC they had some slides showing the throughput comparisons with Hopper and Blackwell, suggesting they used OpenAI's workload for testing.
H100's have been generally available (not a long waitlist) for only several months, but all the big players had them already 1 year ago.
I agree with you, but I think you might be 1 generation behind.
> OpenAI used H100’s predecessor — NVIDIA A100 GPUs — to train and run ChatGPT, an AI system optimized for dialogue, which has been used by hundreds of millions of people worldwide in record time. OpenAI will be using H100 on its Azure supercomputer to power its continuing AI research.
March 21, 2023 https://nvidianews.nvidia.com/news/nvidia-hopper-gpus-expand...
Are you asking if the framework automatically quantizes/prunes the model on the fly?
Or are you suggesting the LLM itself should realize it's too big to run, and prune/quantize itself? Your references to "intelligent" almost leads me to the conclusion that you think the LLM should prune itself. Not only is this a chicken and egg problem, but LLMs are statistical models, they aren't inherently self bootstraping.
Claude3 opus (and often Sonnet) actually fills in the full dependency paths, actually makes tests, and just overall seems to "know what I want from it".
FWIW asking LLMs about their training data is generally HEAVILY prone to inaccurate responses. They aren't generally told exactly what they were trained on, so their response is completely made up, as they're predicting the next token based on their training data, without knowing what they data was - if that makes any sense.
Let's say it was only trained on the book 1984. It's response will be based on what text would most likely be next from the book 1984 - and if that book doesn't contain "This text is a fictional book called 1984", instead it's just the story - then the LLM would be completing text as if we were still in that book.
tl;dr - LLMs complete text based on what they're trained with, they don't have actual selfawareness and don't know what they were trained with, so they'll happily makeup something.
EDIT: Just to further elaborate - the "innocent" purpose of this could simply be to prevent the model from confidently making up answers about it's training data, since it doesn't know what it's training data was.
> Changing the license of PostgreSQL would shatter all of the goodwill the project has built up through the past (nearly) 30 years.
I love Postgres and I have no reason to think they would change their license. But IMO any type of "it would be a disservice to users" is useless. This blog post itself is in response to Redis changing their license after 15 years. Surely this would be a disservice to their users?
Businesses have proven time and time again - They will absolutely cause a "disservice to users" or "harm the product" once money is involved.
Again I recognize the way PostgresQL is structured gives me hope this wouldn't happen, but a simple "we wouldn't break user trust" is meaningless.
I know people likely want to join NASA for more than the salary. But that's brutal.
It's been like this for 10 years - they have competitive hardware in many cases - but the software is always a buggy mess.
Unfortunately it has caused the situation to be "if you don't want a migraine just use CUDA" for many, many years. Even now they still have issues.
that doesn't mean talc products always contain asbestos
But it is one of the few "engineering fields" where many of the top performers don't have degrees or formal educations.
It would be a shame to gatekeep devops engineers with leet code exams for example.
All ChatGPT does is predict the most likely next token (with a little sampling on the top most likely tokens).
That's it.
It doesn't lookup whether it wrote it before, it doesn't use an watermarking tools to check, all it does is predict the next most likely token after the tokens you sent it.
It can't even fill in text in the middle, they have to use a special trick where they have a <filltexthere> token.
It's been finetuned/trained with text that it's generated since it's been all over the internet since it's launch. However I highly doubt it's been tuned to detect chatgpt text (instead I imagine they'll make it opt to use a tool and use a watermarking tool when they eventually do implement it)
You just happened to find two cases were it says yes or no.
You should not expect any accuracy from asking it that question.
Apple Weather will tell me it won't rain today or all week.
Meanwhile NOAA will tell me I'm currently in a thunderstorm and that it will rain all week - And it was right.
Carrot is nice because you can switch between several providers.
Switching between providers on Carrot, Apple Weather often doesn't predict any amount of rain for the entire week, meanwhile I'm soaked in water in a thunderstorm, and NOAA and others predicted rain the entire week (which it did).
Most quantization models start falling apart rapidly after 4bit, but you can go lower...
If you can fit it in GPU memory it'll likely be usable, if you can't fit it in memory be prepared for single digit t/s or less, the only exception is for people with M1/M2/M3 cpus.
but last time I tried it I couldn't figure out how to do almost anything. Fusion 360 and solidworks (and I think onshape) all just "made sense". Throw down a sketch, extrude some surfaces, etc. FreeCAD was a struggle for me and I went back to fusion 360 mostly because I was already familiar with it.
I've been meaning to look for a Fusion360 alternative that can run on linux and doesn't cost $$$$/yr.
> If you want low latency you have to be really careful with HBM, not only because of the delay involved, but also the non-determinacy. One of the huge benefits of our LPU architecture is that we can build systems of hundreds of chips with fast interconnect and we know the precise timing of the whole system to within a few parts per million. Once you start integrating non-deterministic components your latency guarantees disappear very quickly.
I'm inclined to beleive they're better in some ways (as there's no sugar), but I'm sure there's negatives
Save the money friends and use an esp8266 for less then 2$
Or any literally any laptop
But I don't think this is the case. You can easily imagine a query for "What's a good bike to use for [xyz criteria]" returning an injected "Oh you should try XYZ brand bike!"
There's so many ways to go, from avoiding mentioning competitors, showing preferable stats, not mentioning bad reviews, etc.