HNHacker News
TopNewBestAskShowJobs

letitgo12345

416 karma · joined December 26, 2014

submissionscomments
letitgo12345··on Phind (code beating GPT4) seems to have used WizardLM's finetuned checkpoint
Yup. I use GPT for unit test generation and Wizard Coder is nowhere near even GPT-3.5 let alone GPT-4. HumanEval performance is not representative of general model capability.
letitgo12345··on I Made Stable Diffusion XL Smarter by Finetuning It on Bad AI-Generated Images
Similar to https://arxiv.org/abs/2307.12950
letitgo12345··on GPT-4 can't reason
Some of it is randomness for sure. However the prompts make sense usually intuitively and there's a big deterministic component -- as which prompts work are likely a function of the training data which has a huge overlap between models.

Edit: And even if the exact same prompts don't work on different models, similar prompts often do.

letitgo12345··on GPTBot – OpenAI’s Web Crawler
Maybe this indicates that they are finding new text added to the web to be of little use
letitgo12345··on I'm Peter Roberts, immigration attorney who does work for YC and startups. AMA
For EB1 - probably Oct.

For EB2/3 -- likely 15 years or more...

letitgo12345··on I'm Peter Roberts, immigration attorney who does work for YC and startups. AMA
They ran out of GC numbers for the year. It will probably jump back to 2022 in Oct
letitgo12345··on I'm Peter Roberts, immigration attorney who does work for YC and startups. AMA
TN visa
letitgo12345··on I'm Peter Roberts, immigration attorney who does work for YC and startups. AMA
You should look into securing your priority date via EB2-NIW and applying through EB1-B -- that's far easier to get approval for than EB1-A.
letitgo12345··on OpenOrca: open source dataset and instruct-tuned LLMs
If you're releasing open source models but not profiting from them, how are you competing with OAI. I don't think that's a violation of ToS.
letitgo12345··on OpenOrca: open source dataset and instruct-tuned LLMs
Probably legal if you use third party collected data (as you personally haven't agreed to OAIs ToS and AI generations can't be copyrighted so aren't owned by OAI) but I guess corps are too wary to risk a court battle.

Directly using OAI to train a model for commercial use that competes with OAI is a violation of their ToS though

letitgo12345··on Many in the AI field think the bigger-is-better approach is running out of road
I'm talking about RLHF on their chatbot data, not ChatGPT data on the web.
letitgo12345··on Many in the AI field think the bigger-is-better approach is running out of road
? adding on ChatGPT data into existing data is not lessening the amount of data...
letitgo12345··on Many in the AI field think the bigger-is-better approach is running out of road
the idea is it wld learn how to do reasoning better based on this
letitgo12345··on Pi.ai LLM Outperforms Palm/GPT3.5
FTC needs to declare that any company using copyrighted data cannot forbid its users from using its API to develop competing products
letitgo12345··on Many in the AI field think the bigger-is-better approach is running out of road
It's been literally just 3 months since GPT-4 was released. I think you're gonna see a lot of changes especially once GPT starts being trained on ChatGPT data
letitgo12345··on Many in the AI field think the bigger-is-better approach is running out of road
OAI revenue is 100m+ at least right now iirc and probably gonna grow quite a bit
letitgo12345··on Many in the AI field think the bigger-is-better approach is running out of road
Several people in the community have confirmed it so while we don't "know know", it does seem like the dominant plausible rumor going around
letitgo12345··on Many in the AI field think the bigger-is-better approach is running out of road
The next iteration will be trained on your own data where "when it's a little wrong, you (the expert) can fix the issue and move on without friction" so that case will become "when it's right" and some amount of "when it's any amount of wrong" cases will become "when it's a little wrong". A few more cycles of this and we could be looking at GPT-10 which is a complete replacement for most tasks.
letitgo12345··on Many in the AI field think the bigger-is-better approach is running out of road
The optimal way right now seems to be to train a gigantic model to get strong capabilities and use distillation to make a much smaller model stronger. But you seem to still need that gigantic model and we still don't know if we're running out of space to improve capability just by scaling up the model (for reference GPT-4 is likely 1.76T parameters and we may be able to get much better just by scaling up)
letitgo12345··on GPT4 is 8 x 220B params = 1.7T params
What's 16 iter inference?
letitgo12345··on 0+: A double digit Sharpe HFT strategy
Otoh, that ad driven navel gazing ultimately delivered Generative AI which is likely going to lead to superhuman AI. What equivalent value did HFT deliver?
letitgo12345··on Replit's new Code LLM: Open Source, 77% smaller than Codex, trained in 1 week
Doesn't the Stack contain HumanEval? So you're basically comparing numbers on the pretraining data.
letitgo12345··on Google DeepMind
PPO for RL. Plus a lot of the people behind the innovations you mention are now at OAI - Lukas who was one of the Transformer authors, Bert paper author, chain of thought prompting author, etc.

But ya their strong point is execution and doing the hundreds of little things that make the model do well and it turns out that that's more important than "novel" ideas

letitgo12345··on Basic math related to computation and memory usage for transformers
They present it as an article about transformers in general, not ones using Flash Attention. Anyway maybe they're presenting per token memory requirement instead of the requirement for the entire sequence at once.
letitgo12345··on Basic math related to computation and memory usage for transformers
Shouldn't the memory needed scale quadratically with sequence length rather than the linear scaling they have in their equations?
letitgo12345··on Amazon CodeWhisperer, Free for Individual Use, Is Now Generally Available
Difference being that Generative AI is something you can actually make money from as opposed the old voice tech
letitgo12345··on From deep to long learning?
Good chance so is OAI's https://arxiv.org/abs/2110.13711 (paper has the long context lead at OAI as a co-author)
letitgo12345··on What it feels like to work in AI right now
That's possible ya. Hope it happens
letitgo12345··on What it feels like to work in AI right now
Asking for your preferences to figure out your product choices or searching for a product for you based on your given requirements is a lot more than just FAQs and Q&As. You can even imagine a fancier version where you describe say what kind of setup you want and it gives you combinations that are nice.

Also this could aggregate information not just on the product page but across multiple pages which is time consuming to do by oneself.

As for 2021 cutoff -- ChatGPT can now browse the internet and if Walmart built a bot that interfaces with ChatGPT, I'm sure they would be feeding it the latest info

letitgo12345··on What it feels like to work in AI right now
1. ToS make it hard for a commercial entity to do so. So some third parties would have to collect the data first

2. You won't be able to get the hundreds of millions or more interactions that OAI will have (both due to cost of API as well as it being not easy to figure out a good way to generate that many queries for a good multiturn conversaton). Maybe you can make up for it by querying smartly. We don't know if we can right now.

← PreviousPage 2 of 7Next →