HNHacker News
TopNewBestAskShowJobs

llm_trw

3,929 karma · joined February 12, 2024

omni_vision_ai@proton.me
submissionscomments
llm_trw··on Quiet Quitting: Why Employees Are Demanding Fairness and Boundaries
I have a $30k computer at home.

Are you going to give me one like that at work?

If not I'm less productive there and only showing up to polish your ego.

My corporate escorts rates are higher than my working rates accordingly.

llm_trw··on When AI Promises Speed but Delivers Debugging Hell
How much did the hardware to train gpt4 cost?
llm_trw··on When AI Promises Speed but Delivers Debugging Hell
Because ignoring a heroic effort from all the women in India the number of Indian developers does not double every 4 years.

The number of flops a gpu can output on the other hand does.

llm_trw··on When AI promises speed but delivers debugging hell
>The memory and compute requirements to develop and run these models make no sense

There is the story that von Neumann flew off the handle the first time he saw an assembler.

>>How dare you waste compute cycles on this frivolity? Just use machine code like everyone else.

llm_trw··on When AI Promises Speed but Delivers Debugging Hell
49 20 72 61 74 68 65 72 20 74 68 69 6e 6b 20 74 68 61 74 20 74 68 65 20 6e 61 6d 65 73 20 6f 66 20 6b 65 79 77 6f 72 64 73 20 6d 61 74 74 65 72 20 61 20 6c 6f 74 2e
llm_trw··on DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
As we have seen here it won't be a Western company that saves us from the dominant monopoly.

Xi Jinping, you're our only hope.

llm_trw··on DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL
The word you're looking for is copyright enfrignment.

That's the secret sause that every good model uses.

llm_trw··on French police free kidnapped Ledger executive
https://xkcd.com/538/
llm_trw··on PhysicsForums and the Dead Internet Theory
For the tech discussions I'm interested in burning cpu/GPU cycles for proof of work is a good way to make replies expensive enough that only people who care will post then.

Another option is a web of trust.

It's finally the year of gpg!

llm_trw··on New Tesla Model Y
To misquote Douglas Adams:

>In other words—and this is the rock-solid principle that underpins their entire facade—their most profound character flaws are entirely masked by the sheer distraction of their glaringly obvious, superficial character flaws.

This quote was transformed using a transformer.

llm_trw··on NIH hit with freezes on meetings, travel, communications, and hiring
To a first approximation all breakthroughs are ignored for years until someone else steals them.

Transformers weren't scaled at google because it would eat their ad business. We could have had current level AI in 2020 if they'd been serious about it. They weren't and everyone who worked on it left when the gold rush started without having done anything meaningful in google.

Computer vision, which is the field I've been working on the past few months, is even worse.

Perverts in their basements are getting better than sota image recognition so their wifus have realistic vaginas, and writing tutorials which provide more working insights than any scientific paper.

This isn't a new thing. I dropped out of my PhD working on something similar to AlexNet in 2009 because everyone thought I should just use gradient boosted decision trees like everyone else at the time. I couldn't get $10,000 grant to buy a machine to run the training on.

It isn't just me. Yann LeCun's networks were running the postal service, but he couldn't get a paper published anywhere because it wasn't a hot area to publish in.

But, yeah. Tell me more about how great Western academia is.

llm_trw··on Young men now spend more of their free time alone than any other group
In 2015 Melbourne there were 5 different groups meeting about Linux weekly and one group about every programming language in the top 20 of stack overflow.

The endless lockdowns destroyed pretty much all of civic society there and the place hasn't recovered, literally everyone I know from that period left after the lockdowns ended.

llm_trw··on Young men now spend more of their free time alone than any other group
Two years of lock downs killed most of civic society. It used to be that there were several interesting meetups every day. Now you'd be lucky to find one once a month.
llm_trw··on Young men now spend more of their free time alone than any other group
>Far from being a US-specific trend, this is sweeping the western world. The share of young people on either side of the Atlantic who regularly meet up socially with friends, family or colleagues has dropped sharply. In Europe, the share who don’t even socialise once a week has risen from one in ten to one in four.
llm_trw··on Young men now spend more of their free time alone than any other group
Is this alone or 'alone'?

I remember in 2000-2006 spending a ton of my time with friends at a lanshop, because our computers were terrible.

It would have counted as in person time, but once we got better computers and internet we just started hanging out online with webcams and voice chat instead.

Nothing much changed apart from spending a lot less money and going to bed at a more reasonable time - the 24 hour discount was probably not the best for our sleep schedules.

llm_trw··on NIH hit with freezes on meetings, travel, communications, and hiring
Yeah, I can replicate work out of a paper quite easily if half or more of the contributors have Chinese, Indian or Russian email addresses.

A lot of small to medium sized companies fighting for mind share also release one script deploy pipelines for their work, which are better than the academic releases.

It's western universities and big tech that releases at best bit rotten code and at worse no code what so ever.

llm_trw··on NIH hit with freezes on meetings, travel, communications, and hiring
The progress isn't happening at Western universities.
llm_trw··on NIH hit with freezes on meetings, travel, communications, and hiring
In ml basically every paper from a Western university is either useless to industry or not replicable.

It is very much a crisis and one that academia has no interest to solve.

In a sane world we'd start with figuring out what the mystery meat in the version of pytorch used for training is and work up from there. There is zero funding for building up the required infrastructure for doing actual science instead of whatever the fuck you call the current mess.

Basically anything on a more complex dataset than cifar is probably a statistical fluke because of hyper parameter tuning. To find this out you will need to spend three months building up all the bit rotten code that doesn't work from the paper. To draw the conclusion that everyone is full of shit you'll need to waste years of your life.

llm_trw··on Tensor Product Attention Is All You Need
It addresses b too since decompositions are always smaller than the original tensor. It's usually the case that memory access is also slower than matrix multiplications so this will be faster. Burning flops to save memory movement.
llm_trw··on Tensor Product Attention Is All You Need
The attention economy and its consequences.
llm_trw··on Tensor Product Attention Is All You Need
Only if the paper is handwritten.
llm_trw··on Metacognitive laziness: Effects of generative AI on learning motivation
> But when they came to writing, Theuth said: “O King, here is something that, once learned, will make the Egyptians wiser and will improve their memory; I have discovered a potion for memory and for wisdom.” Thamus, however, replied: “O most expert Theuth, one man can give birth to the elements of an art, but only another can judge how they can benefit or harm those who will use them. And now, since you are the father of writing, your affection for it has made you describe its effects as the opposite of what they really are. In fact, it will introduce forgetfulness into the soul of those who learn it: they will not practice using their memory because they will put their trust in writing, which is external and depends on signs that belong to others, instead of trying to remember from the inside, completely on their own. You have not discovered a potion for remembering, but for reminding; you provide your students with the appearance of wisdom, not with its reality. Your invention will enable them to hear many things without being properly taught, and they will imagine that they have come to know much while for the most part they will know nothing. And they will be difficult to get along with, since they will merely appear to be wise instead of really being so.

--Socrates on writing

llm_trw··on Kimi K1.5: Scaling Reinforcement Learning with LLMs
You mean the charity* foundation** Open***AI?
llm_trw··on UK's hardware talent is being wasted
Up to 4 layers is pretty standard these days. It's more than 8 where you start running into issues.
llm_trw··on More than 40% of postdocs leave academia, study reveals
ML papers by Western universities barely touch on the problems that practitioners face.

The only papers I see that are routinely useful have half the authors having a .in or .cn email at the end with the rest having Indian and Chinese names in US institutions.

The only western papers which aren't extended advertisements for their company are from people who are making something for themselves.

For example the best paper on image classification I've ever seen was posted on a private discord and was about better labeling the parts of a vagina as part of a stable diffusion training pipeline.

I used the methods without change and got better than state of the art for document segmentation.

llm_trw··on Solving the first 100 Project Euler problems using 100 languages
Huh, that's surprisingly not actively misleading.
llm_trw··on ROCm Device Support Wishlist
>NVidia would have to seriously fumble.

Worked for AMD in the CPU market.

That said I'm deeply worried about anyone whose based their company on amd gpus. The only reason why they do well in hpc is because there's an army of dreadfully underpaid and over performing grand students to pick up the slack from AMD. Trying to do that in a corporate environment is company suicide.

llm_trw··on UK's hardware talent is being wasted
We used to have shops in schools back when funding for education went to education instead of administrators.

That's not been the case for a generation so no one under 70 even knows what we've lost.

llm_trw··on ROCm Device Support Wishlist
We go from:

Support is coming in three months!

To

This card is ancient and will be no longer developed for. Buy our brand new card released in three months!

Every damned time.

llm_trw··on ROCm Device Support Wishlist
Imagine nvidia supported only the 4090, 4080 and 4070 for cuda at the consumer level. With the 3090 not being supported since the 40xx series came out. This is what amd is defending here.
← PreviousPage 7 of 34Next →