Chief scientist of major corporation can’t handle criticism of the work he hypes
statmodeling.stat.columbia.edu
statmodeling.stat.columbia.edu
Why Meta’s latest large language model survived only three days online - https://news.ycombinator.com/item?id=33670124 - Nov 2022 (119 comments)
It's too bad because you really do want leaders who listen to criticism carefully and don't immediately get defensive.
When is “improved structure prediction” useful or important?
Often a process is simplified or distilled down to a sound bite for the general population. Then we simply repeat without understanding the details.
THe problem is those distilled soundbites get learned by the next generation and they try to apply it. At least I will give AlphaFold/DM credit for correcting their language - originally they claimed AF solved protein folding, but really, it's just a structure predictor, which is an entirely different area. Unfortunately, people basically taught computer scientists that the Anfinsen Dogma was truth. I fell for this for many years.
> It states that, at least for a small globular protein in its standard physiological environment, the native structure is determined only by the protein's amino acid sequence.
Seems like "no true scotsman". If you present a counter example, they'll go "but this is only true for "small", the one you gave me isn't small.
Do you say "we're going to give you a problem that is unsolved but likely has a general solution, and you have a chance of making progress, publishing, and moving on to a postdoc" or do you say "THis is an impossibly hard problem and you will only make a marginal improvement on the state of the art because the problem space is so complex and large"?
You say the first because it gets the students interested and working on the problem, only to learn many years later that the simplified model presented was so simplified it wasn't helpful. I fell for that and spent years working on drug discovery, structure prediction, etc, only to realize: while what Anfinsen said was true, it only applies to about 1% of protein space. It's not so much a "no true scotsman" as "some scotsmen wear quilts, and others have beards, but neither of those is sufficient to classify an example as scots".
Given that improved structures has sped up drug discovery, I can see where the mistake is made (X has improved Y, therefore X will improve Y)
I'm sure you know your stuff, and that you have a lot of experience with proteins that haven't helped with drug discovery or engineering, but it sounds like this is indeed a mismatch between predictions rather than facts.
It very well could be the case that speeding up certain problems by multiple orders of magnitude really does help with drug discovery, and this isn't factually inconsistent with the fact that solving those problems hasn't turned out to be useful so far in this area.
This isn't my field but I could grab a bunch of random jars off a shelf and pour them into capsules. No matter how fast I can do this won't improve medical outcomes for patients.
Wasn't your claim that the AI process hasn't been shown to generate a useful drug [1]?
For someone who dedicated their career to ML, they'll naturally try to solve everything in that framework. I observe this in every discipline that falls prey to it's own success. If there's a problem, those in the industry will naturally try to solve it with ML, often completely ignoring practical considerations.
Is the engine in your car underperforming? Let's apply ML. Has your kid bruised their knee while skating? Apply ML to his skating patterns.
The one saving grace of ML is that there are genuinely useful applications among the morass.
Very similar to crypto evangelists boldly proclaiming the world of finance as obsolete. Rumours of you understanding how the financial system works were greatly exaggerated, my dudes.
This assumption has proven to be very fragile, but I don't think the AI bigwigs have accepted that yet. Still flush from the success of things like AlphaZero, where this thesis was more true.
The biggest defeating problem for pure AI teams is that they don't understand the domain well enough to know if their data sets are representative. Humans are great at salience assessments, and can ignore tons of the examples and features they witness when using their experience. This affects dataset curation. When a naive ML system trains on this data, it won't appreciate the often implicit curation decisions that were made, and will thus be miscalibrated for the real world.
A domain expert can offer a lot of benefits. They could know how to feature engineer in a way that is resilient to these saliency issues. They can immediately recognize when a system is making stupid decisions on out of sample data. And if the ML model allows for introspection, then the domain expert can assess whether the model's representations look sensible.
I'm scenarios where datasets actually do accurately resemble the "real world", it is possible for ML to transcend human experts. Linguistics is a pretty good example of this.
1) The AI expert is auxiliary here, and the domain expert is in the driver's seat. How can it be otherwise? You no more put the AI expert in charge than you'd put an electronic health record IT specialist in charge of the hospital's processes. The relationship needs to be outcome-focused, not technology-focused.
2) The end result is most likely to be a productivity tool which augments the abilities/accuracy/speed of human experts rather than replacing them. AGI being not that sciencey of a fiction, we aren't likely to be actually diagnosed by an AI radiologist in our lifetimes, nor will an AI scientist make an important scientific discovery. Ditch the hype and get to work on those productivity tools, because that's all you can do for the foreseeable future. That might seem like a disappointing reduction in ambition, but at least it's reality-based.
This is called the frame problem in AI.
“What are you doing?”, asked Minsky.
“I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied.
“Why is the net wired randomly?”, asked Minsky.
“I do not want it to have any preconceptions of how to play”, Sussman said.
Minsky then shut his eyes.
“Why do you close your eyes?”, Sussman asked his teacher.
“So that the room will be empty.”
At that moment, Sussman was enlightened.
Closing your eyes doesn't make the room empty. And in the same way not programming preconceptions into the neural net doesn't make the preconceptions go away?
I realize explaining a joke or something like this takes away some of the charm (sorry), but would love to get the point :)
I never found them laugh-out-loud funny myself.
This is probably a reference to those.
These technical mimics of that structure echo a time when a lot of people, relatively, were experimenting with disregarding personal subjectivity in favor of direct experience and deeper Truth. In the modern tech versions, that is rarely if ever part of the story?
More generally, it disingenuously disregards the fact that the definition of the problem brings with it an enormous set of preconceptions. Reductio ad absurdum, you should just start training a model on completely random data in search of some unexpected but useful outcome.
Obviously we don't do this; by setting a goal and a context we have already applied constraints, and so this really just devolves into a quantitative argument about the set of initial conditions.
(This is the entire point of the Minsky / Sussman koan.)
I get that starting from a point with "some correctness" makes sense if you want to use such information (e.g. a standard starting point). However, such information is a preconceived solution to the problem, which might not be that useful after all. The fact is that you indeed might not at all need such information to find an optimal solution to a given problem.
> by setting a goal and a context we have already applied constraints.
I might be missing your point here since the goal and constraints must come from the real world problem to solve which is independent from the method to solve the problem. Unless you're describing p-value hacking your wait out, which is a broader problem.
There is always a starting state; using a random one only means you don't know what it is.
Not throwing any stones here, because I've been guilty of the same sort of arrogance in other contexts. But I think the same thing happened a ton during Bubble 1.0 and the software-is-eating-the-world thing. And it's hardly limited to tech: https://xkcd.com/793/
For me, at least, where this came from was ignorance and naivete. Three things cured me. One was getting deeper mastery of particular things, and experiencing a fair bit of frustration when dealing with people who didn't understand those things or respect my expertise. The second was truly recognizing there were plenty of equally smart people who'd spent just as long on other things. And the third was working in close, cross-functional team contexts with those people, where mutual listening and respect were vital to the team doing our best work.
So here's hoping that the AI bigwigs learn that one way or another.
Any radiology AI that needs millions of training sets is useless in practice.
Why? I have no doubt that radiology AI might not be that useful (though radiologist friends of mine say AI is making an increasing impact on their field.) But this logic doesn't make sense. So what if an AI needs a million training examples or even a million training sets? Once your topology and weights are set, that net can be copied/used by others and you get a ready-to-go AI. There's an argument to be made that if training scale is what's needed to get to AGI, then maybe AGI is unrealizable, but that's not the same as saying a domain-specific AI is useless because it needs a large training set.
- Startup founders without any domain experience in extremely risky ventures
- Crypto bros
- Donald Trump
I consider myself - and others view me as - a hyperrational person (possibly often to a fault), and I must admit that when I hear an outlandish claim like the ones spewed by the above, I am sometimes left in a strange emotional state... a stupor?
Like, I don't quite believe the claim because I'm defensively rational, but I feel a certain dizziness and confusion (thinking to myself, "could this actually be true?") until I come back to my senses. The more outlandish and impassioned the speech, the stronger the effect.
It's made me realise we're all built similarly, from the most rational from the most gullible.
I always like to ask myself: "what are they trying to get me to do?". Whether it's a habit, voting pattern, product, etc. Then I ask myself: "is it useful? who does it benefit?"
Whether it's true or not, idrc. what's more important (at least to me) is what it does and who it benefits
(also in this case to avoid any ethical conundrums, something is useful if it makes $, beneficial to X party if it makes them $)
I'm in the field. Haven't experienced people who I could have described as bros. Can you point to one example?
But this is an interpretation after the fact, it feels exactly like you describe it. Stupor.
Edit: typo
For the case of AI analysing x-ray photos, the obvious solution would be a system that can tag photos with information about what AI thinks is going on there. And this information could be passed to the human.
This could save a ton of time and help reduce cases where radiologists missed obvious things.
My son once broke his arm. I brought him to ER, they made the photo but the two people who looked at the photo said there is nothing wrong with the arm. I asked for the copy of the photo.
A week later the swelling did not subside so I took the photo to another doctor and he pointed out an obvious fracture line.
There are many ways to deploy automation and I wonder why everybody tries to shoot for removing humans altogether when most of the time this is literally asking for problems.
The above is a common problem that UX research is interested in. I'm not sure how much it is solved, but it goes well beyond medical fields.
There was a paper a while ago about an effort at something like this. They got the AI going and noticed that the first thing it did was classify all the x-rays by race of the patient. Then they freaked out, gave up on their original project, and wrote their paper about how AI is inherently evil.
The human will still have to look at the x-ray to see if the AI missed something. 95% accuracy is not good enough, those 5% of cases are what most of their training is for, missing it can mean a lost human life. Maybe it can be used to speed up obvious diagnoses, but it cannot be used to filter and rule anything out. The amount of time a radiologist will spend looking at the x-ray will probably not be reduced, so I don't think there's money to be saved here.
A useful productivity tool could be to examine datasets after the radiologist found nothing, as a way to double-check their reading. This won't reduce costs but might marginally improve patient outcomes. Radiologists in first-world medical systems don't really miss a lot of stuff, though.
And of course for simple obvious non-life-affecting stuff like broken bones and dental x-rays, you don't need radiologists now either. Your son's x-ray was probably not looked at by a radiologist.
Isn't this very context dependent? E.g., a delay in lung cancer diagnosis may be a very big deal, but much less so for something like prostate cancer
As the CDC says on its page for prostate cancer screening[0]:
>Screening finds prostate cancer in some men who would never have had symptoms from their cancer in their lifetime. Treatment of men who would not have had symptoms or died from prostate cancer can cause them to have complications from treatment, but not benefit from treatment. This is called overdiagnosis.
>Prostate cancer is diagnosed with a prostate biopsy. A biopsy is when a small piece of tissue is removed from the prostate and looked at under a microscope to see if there are cancer cells. Older men are more likely to have a complication after a prostate biopsy.
[0] https://www.cdc.gov/cancer/prostate/basic_info/benefits-harm...
From "Discrepancy and Error in Radiology: Concepts, Causes and Consequences" (2011) (https://www.ums.ac.uk/umj081/081(1)003.pdf):
> In the 1970s, it was found that 71% of lung cancers detected on screening radiographs were visible in retrospect on previous films [4,6].
> The “average” observer has been found to miss 30% of visible lesions on barium enemas [4].
> A 1999 study found that 19% of lung cancers presenting as a nodular lesion on chest x-rays were missed [7].
> Another study identified major disagreement between 2 observers in interpreting x-rays of patients in an emergency department in 5-9% of cases, with an estimated incidence of errors per observer of 3-6% [8].
> A 1997 study using experienced radiologists reporting a collection of normal and abnormal x-rays found an overall 23% error rate when no clinical information was supplied, falling to 20% when clinical details were available [9].
> A recent report suggests a significant major discrepancy rate (13%) between specialist neuroradiology second opinion and primary general radiology opinion [10].
> A recent review found a “real-time” error rate among radiologists in their day-to-day practices averages 3-5%
> In patients subsequently diagnosed with lung or breast cancer with previous “normal” relevant radiologic studies, retrospective review of the chest radiographs (in the case of lung cancer) or mammogram (in breast cancer cases) identified the lung cancer in as many as 90% and the breast cancer in as many as 75% of cases [11].
> A Mayo Clinic study of autopsies published in 2000, which compared clinical diagnoses with post-mortem diagnoses, found that in 26% of cases, a major diagnosis was missed clinically [11].
A radiologist's diagnosis is not image classification, it's reality classification maybe. (That's quite poetic).
Watching an educational Youtube video about endangered tiger habitats is not the same thing as segmenting possible embedded pictures of kitties and poachers or whatever, and classifying them as such. There's, like, a lot of additional context.
A (good) radiologist is interpreting the images in light of the patient's history, symptoms, and other tests, with the goal of forming a diagnosis and treatment plan. This is rather different from taking an MxN array of pixels and trying to decide if it contains a tumor.
For example, about 10% of people have small gallstones. If an ultrasound incidentally detects some in a healthy, asymptomatic person, nothing happens. The exact same images, coming from a patient with a history of upper-right abdominal pain and jaundice, probably lead to a referral for surgery instead.
What you're describing is the general problem of informed diagnosis, which is also classification, but typically takes into account a great deal of qualitative information. Few if any ML people are working in this area because there is no golden data and it's nearly impossible to evaluate in a quantitative way.
I can certainly imagine that radiologists would love (say) a tool that automatically flags low-quality or mis-oriented images, but that's not at all where the hype is at.
For what it's worth, I agree that blockchain itself is a first-generation technology and sucks relative to other things, like giant vacuum tube computers did. However, the concepts it enables (smart contracts) have as much promise as the idea of software programs running on personal computers back when most people wondered why you need them, since they do very little but play pong.
When I wrote the following article for CoinDesk in 2020, I didn't want to say "blockchain voting", I wanted to say "voting from your phone". Because there are far better decentralized byzantine-fault-tolerant systems, than blockchains. But that's what they ran with:
https://www.coindesk.com/in-defense-of-blockchain-voting
In it, I say:
For every technology we use today, there was a time it was laughably inadequate as a replacement for what came before.
And that's really, the crux of the issue. It happens slowly, and then all at once. Yes we need to listen to guys like Moxie who are skeptical, but we need to also then go and have a discussion from different perspectives, not just one specific perspective. It has even become fashionable in many liberal circles to be against the type "tech bro" typified by HN, including VCs and Web 2.0 tech bros. So before you downvote, realize that most of you would be on the receiving end of it in other echo chambers, due to this phenomenon of thinking there's only one best narrative.
People like Moxie are much more interesting and interested, because they say they' love to be proven wrong. And I am also open to substantive discussion:
https://community.intercoin.app/t/web3-moxie-signal-telegram...
I imagine it's the same with AI claims about traditional fields. Where have we heard that before? "Yes it's cute and impressive but these guys don't really understand what the experts know about chess."
That is just success bias. How many times did people try out perpetual motion machines? The rest of the article tries to make the converse that if something is inadequate today, it will be the replacement in the future.
When do you consider something to be not worth spending time on?
Or even turning lead into gold! A ton of famous scientist (i.e. Newton) were very busy with that idea but we end up only knowing them for other side projects/discoveries.
The abbreviation "i.e." stands for "id est" and means "that is". So what you have written means "A ton of famous scientist (that is, Newton)"...
Now while I'm sure Newton had gravitas (hah!), I doubt it amounted to a ton.
You want to use the abbreviation for "exempli gratia" which is "e.g." and means "for example".
It's a common mistake but easy to remember with eg-xample. :-)
Mark Twain lost all of his money on a wide variety of speculative investments, most of which I would call fairly reasonable.
Getting in on the ground floor does you no good if it's the wrong kind of ground floor, or if it's the right kind of ground floor but the one next door ends up going to the moon while yours falters, or even if it's the right exact ground floor but you go bankrupt investing too early and the person who buys it from you rides it to the moon.
For example, I want to have a centralized authority which can override fraud or a mistake in a financial transaction. I want laws to apply, I want them to be written by elected humans.
I'm not even sold on the value of any kind of electronic voting in general elections, since trust in the process is so vital here that in my mind the horrible inefficiencies of pen and paper and a bunch of humans manually tallying up votes in a thousand school gyms until 11 pm is actually quite okay. I'll pay for that with my taxes, no problem. Now you add blockchain into the mix, and I don't know what problem it solves that does not have superior alternative solutions.
And so on. But I'm gonna stay open minded. Technology X might one day find the perfect problem to solve, or I might realize I was stupidly wrong about technology X for some time.
I'm lazy, you see. Conveniently forgot to mention that.
1. Communities around the world issue their own currencies independent of the dollar (believe it or not, this is as important for financial stability as not just relying on Facebook’s server farms in California)
2. They give their members a UBI in the local currency that they can only spend locally
3. The amount of daily UBI to give out is determined by the vendors in the community voting on it (monetary policy)
4. Have each vendor tagged with “food”, “clothing” etc. and apply taxes to make negative externalities more costly and withdraw money from the economy (fiscal policy)
5. Calculate statistics on how money flows locally in the smart economy and have economists make recommendations about the fiscal and monetary policies, while the population continuously gets to adjust them up and down based on these recommendations
6. Hooking up all such communities to a decentralized exchange called Intercoin, where the central currency is only held by real KYCed communities and isn’t easily susceptible to pumps and dumps by speculators and banks like Goldman Sachs, and also encourages recurring and sustainable value exchanges between communities
7. Allowing tourists to buy the local currency, and allowing individuals around the world to donate to the community and see stats how the money is being used
More info here for instance:
https://intercoin.org/communities.pdf
If this is too vague here is a concrete example:
https://www.youtube.com/watch?v=tgODUgHeT5Y
Or the Today show wondering what the point is:
https://www.youtube.com/watch?v=UlJku_CSyNg
Technology empowers individuals and smaller communities. That's what it does throughout history. Personal computers. Personal printers. Now you can send email instead of relying on a centralized post office. VOIP disrupted Ma Bell monopolies in our lifetimes, instead of $3 a minute phone calls you can now do full videoonferencing for free. The Web instead of gatekeepers at radio, TV, magazines, newspapers, etc.
In all of those cases, you could question why "on a computer" matters. Who needs email when there are phonecalls? Who needs the Web anyway when there is email? Who needs online dating sites when there are matchmakers?
Nathan Myrhvold at Microsoft told people at Excite that "search is not a business".
Economist Paul Krugman wrote that by 2005, it would become clear that the Internet's effect on the economy is no greater than the fax machine's.
Well, these smart contracts are programmable, and you program against one widespread platform, like JVM or in this case EVM. That's a huge benefit. You can have programmable money, programmable elections, without having to deal with thousands of APIs, or as you are saying "normal" physical currency. Governments in fact also want to phase out cash, and even bank credit, and create centrally controlled "CBDCs" so your "normal" money is also under attack by your governments. China and USA and Canada have already done it and they'd love to be able to freeze people's accounts, restrict them from getting on trains, etc. It may be preferable to incarcerating them later on for years, as we do in the USA.
https://www.nist.gov/publications/blockchain-technology-over...
What they all have in common is cutting out an inefficient rentseeking middleman that people have been forced to trust. Yes that includes FTX, Binance, Coinbase and governments.
Here is an example… how would you do this internationally without crypto?
The real problems are making global payments directly to the people on the ground without waste and having confidence in how they are spent.
This works with the existing infrastructure in every city — similarly to point-of-sale machines that VISA and Mastercard did a lot of work to set up over decades. Back then you would be asking why the world needs credit cards and payment systems when there were perfectly good cash based systems and charities on the ground.
Anyway, the vendors sell food, the person shows up and buys the food. We know how the money was spent. The people help people.
To try to show you by analogy… it is as if people said that there should be a decentralized and uncensorable network for uploading videos taken by people’s own dashcams, phones, etc. of rockets hitting buildings, detention campa etc. But you’d keep saying that the Associated Press and the current centralized media is perfectly adequate and reports everything we need to know, and if people wanted they could upload their videos to Telegram or some other adhoc solution that isnt designed like the news agencies. Why decentralize anything? Because the people DON’T get a good system otherwise
And real vendors don't want cryptocurrency magic beans anyway. They want useful currency like dollars or euros that they can use to pay their own suppliers.
Technology empowers individuals and smaller communities. That's what it does throughout history. Personal computers. Personal printers. VOIP instead of $3 a minute phone calls. The Web instead of gatekeepers at radio, TV, magazines, newspapers, etc.
In all of those cases, you would probably say "the real vendors don't want the Internet anyway". Who needs email when there are phonecalls? Who needs the Web anyway when there is email? Who needs online dating sites when there are matchmakers?
Nathan Myrhvold at Microsoft told people at Excite that "search is not a business".
Economist Paul Krugman wrote that by 2005, it would become clear that the Internet's effect on the economy is no greater than the fax machine's.
You'd be in good company ... a lot of smug people have always said this newfangled stuff is totally useless because people are perfectly fine using the "useful" systems they've always used, not "magic beans" like this new programmable money.
In order to have an actually good discussion, we need to look at the thing and go past the "well someone criticized the Internet too and now look at it".
So taking the idea of the blockchain. What makes the blockchain a "different thing"? The differential aspect is that it allows untrusted nodes to join in a distributed architecture. It's not the only database that exists and it's not the only distributed one either. So any claims of new features brought by blockchain should justify why they need the "untrusted distributed nodes" part. If they don't, we can assume that those new features don't really need the blockchain: either it's already been done, it's not an use case people are too interested in or it's not viable due to other reasons (economical, technical apart from storage, political...). In the case of blockchain claims, most don't actually justify the need for the untrusted distribution. For example, smart contracts: it's just a fancy word for "computer program" only that it runs on a distributed trustless architecture. But is that really needed? My bank already runs computer programs that execute loan payments, for an example of things people try to implement with smart contracts.
Compare that with personal computers or even AI. PCs allowed data manipulation, storage and calculations at a capacity that was not previously available. Of course the first computers wouldn't have enough power to do things that a wide array of people would find useful, but "low power" isn't a fundamental aspect of personal computers in the same way that "low bandwidth" isn't a fundamental aspect of blockchains.
Now elections, roles, permissions and credit balances may hold significant total value.
Various jurisdictions now start to require you to hold surety bonds, get audits etc. Suppose you manage payments volume of $20 million a month for a teacher marketplace. One of your developers can just go into the database and change all the balances, salami-slicing money to themselves. Or someone can go and change all the votes.
How can the users trust elections, or that you won’t abscond with the money one day, or get hacked, like FTX and MtGox?
Web3 solves this with smart contracts. For the first time in history, we can guarantee (given enough nodes) that it is infeasible to take actions you are not authorized to do. The blockchain is readable by everyone - but more importantly, only authorized accounts can take individual actions that do a limited amount. It’s truly decentralized.
The alternative is to build elaborate schemes where watchers watch the watchers — and the more value is controlled by the database (in terms of votes or balances) the more risk and liability everyone has. Why have it?
Have teachers be paid by students using web3 smart contracts and tokens. Your site becomes merely an interface which contains far more low-value things.
As for data, you can store it on IPFS with similar considerations. Read this:
https://community.intercoin.app/t/who-pays-for-storage-nfts-...
As you can see, my company and I have been giving it a LOT of thought, and not distracted by ponzi schemes. I am able to articulate exactly when you need Web3 and IPFS.
Most people won’t have the knowledge or the time to verify the contracts. They will trust your word that they can’t be used to scam them. Smart contracts can still have failures too. And as long as the blockchain doesn’t control the real world, it won’t guarantee anything there (such as people making multiple wallets to manipulate the votes).
> The alternative is to build elaborate schemes where watchers watch the watchers — and the more value is controlled by the database (in terms of votes or balances) the more risk and liability everyone has. Why have it?
Why not the alternative of a regular bank account with public records? That also eliminates the risk that any mistake or manipulation stays there forever. It’s a tradeoff, not an absolute improvement.
In any case, it seems you have indeed thought about a real use case where the blockchain at least makes some sense. But that’s precisely my point, we need to be talking about actual use cases and not empty claims about potential without actually looking at why it’s useful.
or this in log4j: https://theconversation.com/amp/what-is-log4j-a-cybersecurit...
In fact it happens more and more:
http://cyware.com/news/analyzing-the-deadly-rise-in-npm-pack...
2) On EVM you can just audit a smart contract FACTORY, and then people can trust all the instances made from it. This is immensely powerful.
And then there is also regular use by people who put huge amounts of value into a protocol and it is never hacked. It is why for example people trust UniSwap liquidity pools not to rugpull them, and the same with many other protocols.
Now, to be fair, you don’t need a blockchain for that. You can use merkle trees and code signing. But the current Web sucks at it — you just have to trust the server not to suddenly send you bad code, or your “user agent” to execute JS that sends your password in the clear somewhere. And App Stores are black boxes where you just have to trust Signal’s claims or Telegram’s claims. I write about that here in much more detail:
https://community.intercoin.app/t/web3-moxie-signal-telegram...
and I give an extensive talk here as to why we need to reform the current Web:
https://www.youtube.com/watch?v=yKPKuH6YCTc
And I wrote an article here on why the current technology leads to extreme centralization and what effect that has on our society:
https://cointelegraph.com/news/how-a-web-that-lost-its-way-c...
There are real societal consequences on the largest scales, from these platform choices of tech stack.
3) Sure, in the real world, things may not match the data on the blockchain. That painting could be stolen despite what the NFT says. The house up for sale may be a fake. And someone could default on a loan that you thought would bring you revenues.
Blockchain doesn’t guarantee any of those things. However, on Aave marketplace, I am guaranteed a return, or my money back in the token I had lent. So my risk is reduced now to black swan events on the token market, so if I’m using mainstream tokens the chances of getting wiped out or losing my collateral are almost totally eliminated.
People who don’t want to use it don’t have to. But if you’re building a very big community with a lot of value at stake, would you rather put votes, roles, and balances into a central database and PHP app code, or a blockchain with smart contracts?
From personal experience I can tell you it should be the latter.
4) In your example it would be the bank that would opt to use blockchain and freely offer other indepdendent entities not under their control to run nodes snd secure it. It reduces the bank’s risk and liability!
And yes thank you for recognizing that there may be a real class of use cases!
So my general skepticism regarding blockchain is that it presents technological solutions to social problems (so it won't work). AI is different since it's a bit all over the place. In principle, it aims to solve problems that are obviously worth solving, but as long as it will fall short of its promises, the partial solutions we do get are kind of a mixed bag: if we need to keep a human in the loop or at the wheel, it's suddenly a lot less attractive. And the path to the AI being good enough to be trusted is nebulous at best. But we'll see. As for PCs, their utility has always been obvious and the roadmap clear.
People tend to elect representatives for long terms instead and then complain about them, rather than delegating their vote to experts or trying other systems like Ranked Choice Voting.
Many people complain about having to travel far to a polling booth, and disenfranchisement, whereas they could vote for their phone. Elderly and minorities in rural areas often have bad access.
If elections were cheap, people could easily engage in collective decision making of various types and choose various ways yo tally votes. None of that is possible today, we are like the people before computers, or before the industrial revolution - having a limited number of options, newspapers, etv.
We would also have more confidence in the results as we’d check using the Merkle tree that our vote was counted. It would be user friendly to do so.
And we could also implement many of the results on-chain, such as how much UBI to give out in our own community’s currency, or how much to tax transactions.
This is just the tip of the iceberg. Just see https://intercoin.org/applications
It's pointless to improve voting technology if people don't understand the ramifications of what they vote for, and they will never know that unless they invest hundreds of hours of careful study into it. If everyone does this, there is no way it won't be insanely expensive.
IMHO there is a very simple solution to all of these issues that could have been implemented a century ago: pick representatives at random. Send them all to the capital, lodge them and pay them to think about the issues full time, talk to experts directly, interview candidates for Prime Minister and other executive positions directly, coordinate with each other, and so on. Give people the time and the information they require in order to cast the very best votes they are capable of.
Blockchain may or may not make it easier to bypass the concept of having representatives decide, but this assumes we'd want to in the first place, which I am in full agreement with you: this is a feature, not a bug, so I don't wanna, so I don't need blockchain.
And the reason we localize voting, or should do so, is because of the principle of subsidiary.
https://www.npr.org/sections/parallels/2014/04/02/297839429/...
What is your expected outcome when you let the crowd decide how much taxes they pay, or how much to spend on road/water/electricity grid maintenance?
Sure you want quality here but there’s always going to be a human in the loop for this kind of work.
Any workflow with a human in the loop has this speed vs accuracy tradeoff.
While I’m not saying that speed trumps accuracy here, I don’t think you can dismiss without evidence that the tradeoff exists and speed might have benefits.
Lab work and clinical trials are incredibly slow. A single experiment testing a single candidate might take weeks (in cell lines), months (rodents) or even years (humans/non-human primates). You're going to do a bunch of them and they often require expensive reagents and/or tedious work.
Consequently, shortening the wait for a predicted structure by a few hours (or days) won't really move the needle. This is especially true if it makes your experiment, already probably a long shot, less likely to succeed.
He's supposed to agree with you, or not express an opinion? Anything else short of this would be "defensive" right?
This whole idea that defending your positions in arguments is somehow a bad thing is a really odd modern development that I never understood.
Any AI model needs to be designed with adversarial usage in mind. And I don't even think random people trying to abuse the thing for five minutes to get it to output false or vile info counts as a sophisticated attack.
Clearly before they published that demo Facebook had again put zero thought into what bad actors can do with this technology and the appropriate response to people testing that out is certainly not blaming them.
Why? There's probably plenty of usage of ML where both the initial training set, its users and its outputs are internal to one company and hence well-controlled. Why should such a model be constructed with adversarial usage in mind, if such adversarial usage can be prevented by e.g. making it a fireable offense?
Wow, not sure what to say if that's what you think are the only options. I didn't see the original response to the parent commenter, but this quote in the article, "It’s no longer possible to have some fun by casually misusing it. Happy?" doesn't bode well.
I get that in the post-Twitter world it can be heart to differentiate between valid criticism and toxic bad-faith arguments, but lets not pretend that it's impossible to acknowledge criticism in a way that doesn't immediately try to dismiss it, even if you may not agree in the end.
You interrupting the messaging ruins it, so you get some deniable boilerplate response. its not personal.
Him and Grady Booch recently had a back and forth on the same subject on Twitter where to me it seemed like he couldn’t answer Booch’s very basic questions. It’s interesting to see another person with a similar opinion.
Society is still led by vague mental imagery and promises of forever human prosperity. The engineering is right but no one asks if rockets to Mars are the right output to preserve consciousness. We literally just picked it up because the elders started there and later came to control where the capital goes.
We’re shorting so many other viable experiments to empower same old retailers and rockets to nowhere.
Being constantly sniped at probably puts you in a default-unreceptive state, which makes you unable to take on valid feedback, as yours sounds like.
For sure. If this is how he treats outside experts, I can't imagine what it's like to work for him. Or rather, I can imagine it, and I think it does a lot to explain the release-and-panicked-rollback pattern.
Like many thin-skinned hype merchants with a seven-figure salary to protect, they're going to try and block criticism in case it hits them in the pocket. Simple skin in the game reflex that will only hurt any chances of improvement.
Ideally scientists would be interested in the truth and engineers would be interested in making the system better.
And my expectations were exceeded by the first speaker. I couldn't wait for 3 full days of this! Then the second speaker got up, and spent his entire presentation telling why the first person was an idiot and totally wrong and his research was garbage because his was better. That's how I found out my field of study was broken into two warring factions, who spent the rest of the conference arguing with each other.
I left the conference somewhat disillusioned, having learned the important life lesson that just because you're a scientist doesn't mean you aren't also a human, with all the lovely human characteristics that entails. And compared to this fellow, the amount of money and fame at stake in my tiny field was miniscule. I can only imagine the kinds of egos you see at play among the scientists in this article.
At the end of the day, all of the noise of negativity and bad press is being drowned out by incredible demos. I don't know what to chalk this up to if not jealousy. Most people in the ML-o-sphere are ignoring it.
At the end of the day, all that matters is: are users using what you built?
You chose industry over academia, and that's fine. It lines up with your values. But realize that not everyone shares those values and beliefs. To some, the act of discovering a new thing is much more important than the users using said discovery. And that lines up with academia more so than the industry.
Both are different. Both are valid.
> None of it would be possible without academia though. Industry just applies academic research.
Meh, that vastly oversells academic research. Very little of academic research in computer science is actually used in the industry. It's not that the industry is ignorant, but rather that the majority of academic work is useless: They create artificial problems [1] and solve them in shoddy ways, with hand-picked benchmark results, and frequently without even publishing the source code.
It's probably not surprising, given that the typical incentive is to get a PhD. So you need a "problem" that can reliably be solved in 3-5 years and which allows you to produce 5-10 conference papers with your name on it.
[1] I'm not talking about theoretical fields – my comment is purely about supposedly practical research.
I was once watching a VC interview a snooty machine vision scientist at Johns Hopkins who was talking up how well his research was at recognizing three d things. So the VC pulled out his cellphone and took a photo of a box on the table. He asked the professor to have the software highlight the rectangular solid. Whoop. He never heard back. The software in the lab that was supposedly so great couldn't do a very basic task that wasn't from its preapproved set of tasks.
I do think that academia can be the source of some great ideas, but they often end up believing their own BS.
I worked at Google and there's just tons of stuff that never actually existed in academia and was created, launched, and then replaced by something better entirely within the company without any publications!
> At the end of the day, all that matters is: are users using what you built?
How would you measure Isaac Newton's advances in calculus and mechanics or Einstein's general theory of relativity, against say, a web app with a billion users?
If you want to steelman the GP's argument, you should compare the web app with e.g. some niche in pure math. There the trade off between novelty/interest and usefulness to people today is more clear.
I think the two are incomparable and both useful, but it's disingenuous to strawman the GP as saying web apps are more useful than relativity.
Yes, I picked those two examples for the effect or as a reduction to the absurd (not a strawman), because going only by the immediate or tangible value of what one "builds" (science isn't even built, but rather discovered) is not a good way to dismiss academia.
When most folks think of academia they think faculty, but staff vastly outnumber then. Contrary to popular belief there are legions of cold, level headed, engineers that get shit done.
A lot of the research isn't some random study of something that may or may not be useful in half a century or more, it's often immediately applicable and winds up in products or shaping government policy on a global scale. Especially the well funded ones.
But we don't hear about that stuff. We hear what the media and tech companies are currently trying to cram down our throats.
Ah yes, the Kardashian model of success
But the A.I. hype is out of hand. "A.I. Safety" research is the worst of it, as it suggests this technology is so powerful that it's actually dangerous. The other day I was almost to write a comment on HN to a post from lesswrong where they apologized at the beginning of an article critical of the intelligence explosion hypothesis because short of Scientology or the LaRouche Youth Movement it is hard to find a place where independent thought is so unwelcome.
Let's hope "longtermism" and other A.I. hype goes the way of "Web3".
I'm not really sure exactly what you mean by "feed AI inputs like race and it will make racist decisions", as that's an absurdly general claim that doesn't even really say anything useful. If you mean the model reflects the underlying stats that correlate with race, class, gender, then sure, yes, I agree the predictor will do that. To say that it's racist is an entirely subjective claim.
THe point of insurance isn't to avoid social ills, but to statistically minimize the impact of risk on individuals (while making a tidy profit for the insurer).
I'm sure if black wall street wasn't razed to the ground by a racist white mob (among several other attempts to gain stability, wealth, etc. by black people being destroyed by racist white anger) then maybe the model wouldn't need to "reflect the underlying stats that correlate with race...". But those underlying stats, and those correlations, didn't just happen in a vacuum, hewn out of the aether like a magical and consequentless thing.
Because it's fucking unfair, and humans hate feeling like they've been wronged for unfair reasons they cannot control. How would you feel if your loan got denied, you asked why, and you found it it would have been approved if you had been born in April instead of June? Don't pretend you'd love it and smile. You'd rant about it and tell all your friends how bad that lender is.
If the lender rejects you because your debts are too high or your income too low or something, or the zip code of the home you're mortgaging, that makes more sense and feels more relevant and in your control, at least.
Yeah, it's nice from the point of view of the lender, but the cost is too high to society.
Society feels so strongly about this that (in the USA) we've even passed all kinds of laws about what pieces of information are absolutely banned from being used to determine who gets a job or who gets a home. If you are a landlord or hiring manager you have to be aware of them so you can at least pretend to comply.
Sometimes I wonder if AI is already here secretly. Half of my replies on hacker news seem to basically boil down to teaching what humans are actually like.
I mean, I pay more for car insurance because I am immutably male. I don't feel great about this, but assuming that males my age are statistically more likely to be in crashes this makes total sense.
Likewise, I don't feel great that even if I were really good at basketball I'd be way less likely to make the short list of an NBA recruiter, because statistically I'm not going to be as good as someone two feet taller. It may not feel great, but it still absolutely feels fair too.
If people born in April instead of June were actually statistically more likely to be in crashes, I don't see how this is any different or unfair.
Because it’s radically unfair to do that! You’re seriously suggesting that some people should be penalized for the day they were born?
Taking the point more seriously, it’s hard to believe that the risks for whatever you’re ensuring have populations of people each month that are so different that a random person from each has a risk associated with them that birth month, as a signal, isn’t likely to discriminate unfairly almost as much as it helps. Naively, it’s likely to unfairly penalize almost half the population. Maybe you’re prepared to accept that because, hey, it’s right over half the time, but tell that to the vast population you’re penalizing.
And some things just shouldn’t be included. Health insurance, for example: isn’t it unfair to include risk of cancer caused by genetic factors in premium calculations? I guess you could say that’s different because parents could control birth month, but, man, that just seems so uncaring to me.
In the race example, let’s say there is a discrepancy in the underlying stats now, but it’s caused by structural problems in society caused by racism in the past. By using that as a signal to discriminate now it makes it more unlikely that the problems will ever be resolved (example: Black people cant get loans as easily, so can’t start businesses as easily, so the wealth gap is reinforced). If you think that’s ethical I would suggest that you are considering statistics too much and people too little.
Genetic risk of cancer cannot be included in insurance decisions (excluding voluntary life insurance) because a law was passed (GINA) preventing it.
It's not the role of the loan industry to make it easier for black people who appear to be a higher statistical risk to get loans. That's the purpose of government- to make it illegal to include genetic history in life insurance models- provided that society believes in that principle.
Another way to say that - one that appears less uncaring to readers - would be “yes, including birth month or race in insurance risk calculations would be unfair+, so it should not be done. If necessary, the government should step in to prevent this”.
+ you may squabble with the use of ‘unfair’ here - obviously it is not ‘unfair’ in the stastical sense, but that’s not what the word generally means in spoken English. ‘Inequitable’ would be better, but for some reason ‘equity’ seems to be evolving to mean ‘at all costs, produces equality of outcome’ and so is a victim to the same problem in the opposite direction. I chose the more common word.
[edit: I used insurance here, sorry - the same would apply to loans I think]
I do agree that if you expect corporations to act morally, then it would be sensible for them to be more careful in understanding and correcting long-term social ills.
It is trivial to bake an irrelevant distinction into a naive ML driven process.
Even if we set ethics aside, this is still a terrible idea because credit risk analysis is highly regulated in most cases. There is enormous regulatory risk to using birth month in a credit risk model. Particularly because hen a jury asks "what's an alternative causal model that could explain the bank's incorporate of this data point into their models?" their answer is going to be "absolutely none".
Instead, they look at who was approved, and maybe whether they repaid their loan.
So if a human making loan decisions isn't keen on giving out loans in View Park-Windsor Hills, a model trained to produce human-level performance won't be keen on doing it either.
Historically, at least in the United States, loans were unavailable to PoC, especially black Americans. Districts in many American cities were "redlined"[1], that is to say certain districts of cities were deemed "unprofitable" for banks. Redlining was policy and was targeted towards discriminating against black Americans, who were often victims to predatory loans with unserviceable interest rates, which commonly resulted in defaults. The defaults caused worse scores for people in the neighborhood and created a positive feedback loop. Historical housing loan data (and insurance data) includes this data. People today are affected by this historical data and this absolutely must be taken into account by the developers of these models.
Because of the entangled nature of real data, dropping "race" as a feature for training wouldn't solve the problem. Factors like zip codes (think of the redlined districts) would also influence the outcome[2].
Creating models, whose output can impact people so profoundly, e.g. can Jane get health insurance, calls for more reflection than just "numbers correlate!".
[1]: https://www.smithsonianmag.com/history/how-federal-governmen...
[2]: https://www.educative.io/blog/racial-bias-machine-learning-a...
Political solutions are necessary, maybe something like subsidized loans for people from formerly redlined communities to purchase and restore homes, or start businesses. Urban planning projects, like increasing mixed-zoning, pedestrian traffic, and good public transportation would help keep money in the neighborhood. Then there is the question is how to deal with gentrification and increase quality of living in a community without displacing people from that community. It takes a team of experts from various fields, the community itself, and goal-oriented cooperation.
Holy cow, so literally a horoscope? Lol.
How often have you talked to someone at an institution about some egregious wrong, and they defend it as "that's our policy?"
Computers just made the enforcement easier and with less opportunities to break out of it - for example, a sympathetic public servant no longer has the power to make exceptions since "the computer won't allow it."
An example given was a state wanted to spend less on low income benefits. They setup an overly complex computer system that decides eligibility and requires participants in the program to fill out multi page forms 100% accurately on a regular basis or their benefits are suspended immediately. There was no way to appeal this decision, only to re-apply.
Employees of the state agency no longer had any insight into why a form was rejected, and could not speed up the process to resubmit it because the participant had long term medical issues and needed potentially lifesaving prescriptions filled. As a result, many people who would otherwise be eligible for benefits had them denied.
This lead to a marked decrease in spending on social services and was considered a win by the state.
Anyone going through this rigamarole, that's always my advice. Keep pressing, they'll eventually accept. It sucks, but you have to play the game.
That isn't to say that corrupt public servants don't exist, and that they don't look nearly identical if you were to take a brief glance of their actions without looking at their reasoning, but that doesn't mean they're the same people or causing the same end effect.
"Sympathetic" and "source of inequity" are subjective. Letting someone get extra points on their test because of their race is equitable to some, unequitable to others. Who decides?
Equity, by its very definition, is unequal. Every single thing done in the name of equity is unequal. On purpose. Because the "job" of equity is to right inequalities in the system.
In practice it creates more freedom, because users have no choice in their top authorities (I'd have to move to another state or country if I don't like the laws) but often have a lot of choice in bottom level authorities (if the guy at the DMV is mean I can go to a different location)
I'm not saying it's strictly good, but it's much more complex. IF you are someone who truly believes the top level policy is good and perfect and has no problems, then yes, any kind of bottom level control.
But if your are someone who is harmed by the top level policies, you now have an escape hatch by which a public servant can help you overcome this.
For example once in NY I tried to register a car I had bought out of state and one DMV worker told me I could not because the previous owner was named Johnathan Smith but had signed his name as John Smith. Ridiculous, but it's possible they were following the letter of the law and an automated system would have said the same.
I simply went to a different DMV where a different worker took a more favorable interpretation. If it had been a piece of software making this comparison I would have been screwed.
I owned a vehicle for 15 years and it broke down on me, as a result I dropped insurance on it as I wasn't sure if I wanted to keep it or not.
fast forward 3 months and it turned out to be a relatively cheap repair so I went to get it tagged (it had expired while uninsured) and the woman at the DMV absolutely hammered me on the paperwork I had to show her to the point that even the agent at the insurance company was confused about what she wanted.
I go to another location and lo-and-behold it's no longer a problem. Maybe she was in a bad mood, maybe she didn't like the way I looked. I have no idea, but the point you're making is absolutely relevant in day to day life.
And if I can pontificate slightly further, inequities exist in life, there is no way to completely remove them. Giving autonomy to low level people and squashing the ones that abuse that autonomy is absolutely better for people overall, even if it isn't perfect.
I'm making this number up, but I swear that 90% of the time when I come across a company that's absolutely floundering under the weight of its own poor decisions, it's because those decisions are enforced by software such that people CAN'T work around it.
What ends up happening is doing anything crosses 15 silo's and each of those silo's absolutely enforces it's authority such that no single person is able to bridge the gap between even two of them.
What makes it hard is that there are legitimate reasons why you don't want a single person being able to write code, push it to production, open firewalls, open access to databases, add ACL's, etc. Depending on the industry, the need for controls should exist, and because there's SOME legitimacy there it gets pushed across the threshold past reasonableness to absolute pain for everyone involved.
---
And if I may, that's where the most dangerous solutions lie. When there's a legitimate use of them but they can be abused.
Well, you can solve some things. Internet and zoom went some way to solving "how do we work in lockdowns?"
https://www.cureus.com/articles/107920-increased-incidence-o...
The real social problem we ought to be solving is stripping bureaucrats and politicians of their ability to impose lockdowns on us at all. And there is no technical solution for that.
That's not logical. There's no valid need for traffic, but I appreciate Google Maps traffic reports.
So yea, inequality is a social problem, but it is amplified by technical problems. We’ve seen this time and time again, the Internet at large being the most glaring example.
We shouldn’t ignore this amplification effect while we wait to solve the social problems. Furthermore, computers are often viewed as some unbiased decision machine, which makes the problems worse.
Towards the beginning of the pandemic there was a lot of this sort of stuff, say: https://www.nature.com/articles/s42256-021-00338-7
It boggles my mind how anyone can think otherwise. Existential dangers of superintelligent or even non-intelligent AI are the long-term result of the dangers of AI being developed and misused over time for human ends.
It's the exact same argument behind why we should be trying to track asteroids, or why we should be trying to tackle climate change: the worst-case scenario is unlikely or in the future, but the path we're on has numerous dangers where suffering and loss of human life is virtually certain unless something is done.
What sort of AI catastrophe do you think would happen?
The fact that we don't know how close we are is itself dangerous. It's like doing gene editing on pathogens without a proper understanding of germ theory and biosafety. That's where we are with AI.
Some AI dangers are certainly legitimate - it's easy to foresee how an image recognition system might think all snowboarders are male; or a system trained on unfair sentences handed out to criminals would replicate that unfairness, adding a wrongful veneer of science and objectivity; or a self-driving car trained on data from a country with few mopeds and most pedestrians wearing denim might underperform in a country with many mopeds and few pedestrians wearing denim.
But other AI dangers sound more like the work of philosophers and science fiction authors. The moment people start predicting the end of humans needing to work, or talking about a future evil AI that punishes people who didn't help bring it into existence? That's pretty far down in my list of worries.
Your ability to read this sentence right now when we have never met and may not even be on the same continent was once in the domain of science fiction. Don't underestimate technological progress, and specifically, don't underestimate the surprising directions it could go.
Some fantastical AI predictions will happen, most probably will not, and some utterly terrifying ones no one foresaw will almost certainly happen. The unknown unknowns should worry you, and AI is full of them.
Sure, but where should that rank in my worries relative to 'designer babies' and 'rise of authoritarian states as economic powerhouses' and 'corporations that can commit crimes with impunity' and 'rising medical bills' and 'widening gap between rich and poor' and 'far right extremism' and 'water shortages' and 'economic crisis wipes out my savings' and 'cyber warfare targeting vital infrastructure' and 'rising obesity' and 'voter suppression' and the many other things a person could worry about?
You know what would be useful for cavemen to ponder? The safety of fire. Or you know, just staying alive because there are more dangerous things out there.
The current state of so-called "AI" is our fire. It's impressive and useful (and there are real dangers associated with it) but it has no bearing on intelligence, let alone superintelligence. It's more likely that a freak super-intelligent human will be born than that we accidentally produce a super-intelligent computer. We produce a lot of intelligent humans, and we've never produced a single intelligent computer.
As it stands, we don't have the understanding or the tools to do anything useful wrt safety from a super-intelligent AI. We do have the understanding and tools to do something useful about asteroids and climate change.
That's just wrong. This very paper we're discussing has AI safety factors built-in: the AI is not supposed to be able to lie, and yet it exhibited deceptive behaviours anyway. That falls squarely under AI safety, and is a pretty useful observation to inform future attempts.
> The current state of so-called "AI" is our fire. It's impressive and useful (and there are real dangers associated with it) but it has no bearing on intelligence, let alone superintelligence.
That's conjecture. We don't know what general intelligence really is, so you simply cannot gauge how close we are. For all you know we could be one simple tweak away from current architectures, and that should be terrifying.
> As it stands, we don't have the understanding or the tools to do anything useful wrt safety from a super-intelligent AI.
Even if that were true, which it's not, we certainly won't develop that understanding or those tools if we don't start researching them!
> We do have the understanding and tools to do something useful about asteroids and climate change.
Suppose you had a time machine and went back to the 19th century to explain the dangers of asteroids wiping out all of humanity. They didn't have the launch capability or the detection abilities we do now, but if they were sufficiently convinced of this real threat, does it not seem plausible that they could be motivated to more heavily fund research into telescopes and rockets? Couldn't we plausibly have reached space sooner and be even better prepared in the present to meet that threat than we are now?
Instead of a time machine, couldn't we just use our big brains to predict that something might actually be a serious problem in the future and that maybe we should devote a few smart people to thinking about these problems now and how to mitigate them?
This all just strikes me as incredibly obvious, and we do it in literally every other domain, but somehow say "AI" and basic logic goes out the window.
This kind of elasticity in language use is the sort of thing that gives AI safety a bad name. You can't take AI research at face value if it's using strange re-definitions of common words.
This is not a redefinition, the harm results from the standard usage of the tool. If the AI is being used to predict the possible future behaviour of adversarial countries, then you need the AI to be honest or lots of people could die. If the AI concludes that your adversary would be more friendly towards its programmed objectives, then it could conclude lying to the president is the optimal outcome.
This can show up in numerous other contexts. For instance, should a medical diagnostic AI be able to lie to you if lying to you will statistically improve your outcomes, say via the placebo effect? If so, should it also lie to the doctor managing your care to preserve that outcome, in case the doctor might slip and reveal the truth?
There are no actual safety issues with LLMs, nor will there be any in the foreseeable future because nobody is using them in any context where such issues may arise. Hence why you're forced to rely on absurd hypotheticals like doctors blindly relying on LLMs for diagnostics without checking anything or thinking about the outputs.
There are honesty/accuracy issues. There are not safety issues. The conflation of "safety" with other unrelated language topics like whether people feel offended, whether something is misinformation or not is a very specific quirk of a very specific subculture in the USA, it's not a widely recognized or accepted redefinition.
AI safety is not a near-term project, it's a long-term project. The what-ifs are exactly the class of problems that need solving. Like it or not, current and next generation LLMs and similar systems will be used in safety critical contexts, like predictive policing which is already a thing.
Edit: and China is already using these things widely to monitor their citizens, identify them in surveillance footage and more. I find the claim that nobody is using LLMs or other AI systems in some limited safety critical contexts today pretty implausible actually.
None of what is remotely plausible?
And there's lots of actual, real scientists who think AI risk is a real thing we should be concerned with. Both within the field (e.g. Stuart Russel) and outside of the field (e.g. Stephen Hawking.) Are all of these scientists also talking with zero basis in actual science?
The intelligence is what determines the shape of these networks so they can even learn a useful pattern. That’s still gonna be humans for the foreseeable future.
Current AI research largely isn't focused on general intelligence, but the possibility remains that it could still spontaneously emerge from it. We can't even quantify how likely that is because we don't understand intelligence, so whatever intuition you have about this likelihood it's completely meaningless and not based on any meaningful data. We're in uncharted waters.
For a while I ran a project that ran a (scientific) binary on every Google machine in production using all the idle cores on the machine to do protein design (it worked!). At the end of the project we seriously considered making the binary instead a learning system that had access to 'sensors' (IE, signals from Google production machines) and 'outputs' (the ability to change production parameters) and just let that run autonomously. I figured Google prod is one of the few computational/networking systems with enough inherent complexity and resources that spontaneously emergent general intelligence was not inconceivable. However, the project was deemed too risky by SRE, rightfully so. Not because of emergent intelligence but emergent disruption to serving ads.
You missed the terrifying aspect. It's not terrifying that general intelligence emerged from optimizing for reproductive fitness specifically, it's terrifying that general intelligence emerged while nature was stumbling around randomly in the dark trying to optimize for a complete unrelated property.
Guess what we're doing with machine learning?
General intelligence didn’t poof into existence it happened over half a billion years and even then it came from fungi which is itself an intelligence.
Why millions of years? Do you agree or disagree that humans can develop technology faster than nature could evolve it on its own? It took maybe 70 years for computers to go from not existing, to matching humans on visual recognition tasks.
As you just acknowledged, it took nature at least billions of years to evolve humans. Is not focused technological development obviously orders of magnitude faster at evolving intelligence than nature? Does it not then follow that artificial general intelligence is a lot closer than an argument based on natural evolution might imply?
What exactly is not logical about this argument?
Personally I figure that once you have a complex enough system with feedback regulation and self-preservation, something like general intelligence occurs spontaneously through the normal mechanisms of evolution (probably more than once in different kingdoms) because it provides a heritable survival edge.
I agree that the simplest properties of neural networks are an emergent property of multicellular life.
We must do something.
This is something.
Therefore, we must do this.
2. AI safety research attempts to mitigate some AI dangers.
3. Therefore we must fund AI safety research.
Yup, checks out.
The priests say virgin sacrifices pacify the gods.
Therefore we must sacrifice a virgin on every new moon. While we are at it, we should also build a palatial mansion for the wise priests.
Yup, checks out.
An argument that depends on equating priests and scientists is an argument that fails.
Edit: for instance, here's a computer science professor and neurologist who published a book about all of the serious dangers of AI, and calling for funding safety research:
https://www.amazon.com/Human-Compatible-Artificial-Intellige...
Furthermore, AI safety programs are hiring AI researchers and computer scientists to actually do the work, so are you claiming that these people don't sincerely believe in the work they're doing?
These two things are in conflict. We could ignore both asteroids and climate change and according to the best known science there'd be very little impact for vast timespans and possibly no impact ever (before humanity is ended by something else like war).
Yes, also for the climate. Look at the actual predictions and it's like a small reduction in GDP growth spread over a very long period of time, and that's assuming the predictions are actually correct when they have a long track record of being not so.
Really stuff like asteroids and climate is a good counter-argument to caring about AI risk. Intellectuals like to hypothesize world-ending cataclysms that only their far sighted expertise can prevent, but whenever these people's predictions get tested against something concrete they seem to invariably end up being wrong. Our society rewards catastrophising far too generously and penalizes being wrong far too little, especially for academics, NGOs etc. It makes people feel or seem smart in the moment, and they can punt the reputational damage from being wrong far into the future (and then pretend they never made those predictions at all or there were mitigating factors).
That's just incorrect. The Tunguska event was a nuclear-weapon scale asteroid. These are predicted to happen once every hundred years or so. If it happens over a populated city millions would die. If a person wrongly concludes this was a surprise nuclear attack, maybe everyone would die, to say nothing of the real risk that the asteroid itself could be big enough to wipe us all out.
There's a lot of uncertainty around climate change, but changing climate patterns will certainly change resource allocations (fresh water, arable land, etc.). This will lead to shortages in places that were once abundant, which could easily lead to wars in which millions die.
> Really stuff like asteroids and climate is a good counter-argument to caring about AI risk.
This. This boggles my mind. Long-tail risks exist, and burying your head in the sand and pretending they don't just places millions of lives at risk, and potentially the entire human race. You don't have to think these are top priorities, but to dismiss them as complete unimportant is frankly bonkers.
Which had very little impact on humanity because it exploded in the middle of the tundra.
"These are predicted to happen once every hundred years or so"
What is predicted exactly, by whom and how were these predictions validated against testable reality given the postulated rareness? If they're so common then why is it so hard to name the last 10? I think in reality these events are very rare and will almost always happen over the oceans, deserts, poles etc where not many people live.
"Long-tail risks exist, and burying your head in the sand and pretending they don't"
They exist and I am not pretending they don't. I am saying that this style of reasoning in which an extremely unlikely event is unfalsifiably and arbitrarily assigned near infinite downsides in order to justify spending time and resources on it, is problematic and as a society we are far too generous towards people who do this.
"I can't name such events therefore they're not worth thinking about" is a ridiculous argument.
The most recent one happened less than 10 years ago and almost 1,500 people were injured, and we completely missed its approach:
https://en.wikipedia.org/wiki/Chelyabinsk_meteor
Luckily, it again happened near a depopulated zone, but the damage was still extensive.
> I think in reality these events are very rare and will almost always happen over the oceans, deserts, poles etc where not many people live.
Most car accidents will happen to bad drivers, so if you're a good driver you don't need to wear your seatbelt, amirite?
The fact that an easily preventable event typically happens without much damage is no consolation when that's not the case.
> They exist and I am not pretending they don't. I am saying that this style of reasoning in which an extremely unlikely event is unfalsifiably and arbitrarily assigned near infinite downsides
Your mistake is thinking the likelihoods assigned are arbitrary and unfalsifiable. By your logic, COVID-19 was unlikely as most outbreaks are small and isolated, and the likelihood of a global contagion so unfalsifiably remote it's not worth thinking about. Therefore pandemic preparation is a waste of time and money. Now that 6 million people have died, that view doesn't look so rosy in hindsight.
The calculations on asteroid threats have been done based on known data, and even with our current preparations we still miss some potentially devastating ones like Chelyabinsk:
https://en.wikipedia.org/wiki/Impact_event#Frequency_and_ris...
(There is value in doing research and ethical analysis into AI/ML statistical algorithms to prevent hidden biases or accidental physical harm. People working in those areas are producing real benefits for the rest of us and I'm not criticizing them.)
Who are you talking about exactly? Who are these alleged grifters and who are they grifting?
> There is zero actual scientific evidence to support their claims
What claims are you talking about, exactly? Let's get specific.
I hesitate to say "safe space", but... what if a group of people wants to come together discuss AI safety? If they'd have to regurgitate all the arguments and assumptions for everyone who comes along they'd never get anything done. If you are really interested to know where they are coming from, you can read the introductory materials that already exist. If the 99.9% of the world is hostile towards discussing AI safety (of the superintelligence explosion kind, not the corporate moralitywashing kind) there is some value in a place which is hostile to not discussing it, so that at least those interested can actually discuss it.
Using ML for object detection, object tracking, or prediction on an L2-L5 driver assistant system? AI safety research sounds like a capability you'd really want.
Using ML for object detection, object tracking, or prediction on an industrial robot that is going to work alongside humans or could cost $$$ when it fails? AI safety research sounds like a capability you'd really want.
Using classifiers or any form of optimization for algorithmic trading? AI safety research sounds like a capability you'd really want.
Building decision support systems to optimize resource allocation (in an emergency, in a data center, in a portfolio, ...)? AI safety research sounds like a capability you'd really want.
Hell, want to use an LLM as part of a customer service chatbot? You probably don't want it to be hurling racial slurs at your customers. AI safety research sounds like a capability you'd really want.
Unfortunately, now "AI Safety" no longer means "building real world ML systems for real world problems with bounds on their behavior" and instead means... idk, something really stupid EA longtermism nonsense.
The whole point is that the technology itself and its capabilities are not well defined, people are constantly inventing new applications and new methods now at breakneck speed, so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.
Before 2016 or so, "AI Safety" meant the types of things I listed above: how to design AI systems that are safe to use in safety-critical settings.
Unfortunately, the term "AI Safety" has been overrun with effective altruist longtermism folks talking about something closer to https://en.wikipedia.org/wiki/Existential_risk_from_artifici... which is ill-defined and, frankly, usually completely vapid nonsense.
> so the question of how to mitigate its risks is going to be at least as squishy a concept as the underlying tech + applications.
No, no, no!!! AI safety methods -- methods that really increase confidence in real systems -- are not generic. ALL of the useful AI safety work I know of is deeply related to one or more of: the specific model architecture, the specific optimization algorithm, the specific data, or the specific application. Almost always all of the above.
Pressing the idea that AI is dangerous makes it seem like these companies are even more powerful than they are and could drive up their stock price. When the AI Safety people get into some conflict and get fired they are really doing their job because now it looks like big tech is in a conspiracy to cover up how dangerous their technology is.
Suffice it to say, ideologues desperately want control over everything AI/ML. That's the real danger.
Well that's an interesting way to misrepresent an entire important field of research based on what a few idiots said. There are serious people in that field who aren't addicted to posting on LessWrong.
Is that actually true, though? It's true that a higher fraction of the people in that community give credence to the intelligence explosion hypothesis than pretty much anywhere else. (This is what one would expect, since part of the purpose of LessWrong is to be a forum for discussions about super-intelligent AI.) But even if the intelligence explosion is a terrible, absolutely-wrong theory, that doesn't prevent the people who hold it from being open-minded and tolerant of independent thought. Willingness to consider new and different ideas is something the LessWrong community claims to value, so it would be a little bit weird if they were doing way worse than average at it. And AFAICT, it seems like they're doing fine. Some examples:
- Here [1] is a post critical of the intelligence explosion theory. It has 81 upvotes as of this writing, and the highest upvoted comment goes like: "thanks for writing this post, it makes a lot of good arguments. I agree with these things you wrote" (list of things) "here are some points where I disagree" (list of things). This may even be the original post you were talking about in your comment, except that it doesn't start with an apology.
- LW has 2 different kinds of voting: "Regular upvotes" provide an indication of the quality of a post or comment, and "agree/disagree votes" let people express how much they agree or disagree with a particular comment. Down-voting a high quality comment just because you disagree (instead of giving it a disagree-vote) would be against the culture on LW.
If you're already sure that LW is wrong about superintelligence, and you're trying to explain how they became wrong, then "those LW people were too open minded and fell for that intelligence explosion BS" makes more sense to me than anything about suppression of independent thought.
[1] https://www.lesswrong.com/posts/zB3ukZJqt3pQDw9jz/ai-will-ch...
I read the MIT Technology Review article, and I was asking myself “what is an example of Galactica making a mistake?” The article could easily have quoted a specific prompt, but doesn’t. It says the model makes mistakes in terms of understanding what’s real/correct or not, but the only concrete example I see in the article is that the model will write about the history of bears in space with the implication that it’s making things up (and I believe the model does make such mistakes). I don’t think it’s a good article because it’s heavy on quoting people who don’t like the work and light on concrete details.
Does the imperfection of a language model really mean the model should not exist? This seems to be what some critics are aiming for.
So you can get an AI model that hasn’t been hardened against these attacks to write a paper on why <racist thing> or create an image depicting <racist or porny thing> and it just does it. Because the model is just an input:output device and doesn’t have the “wait maybe I shouldn’t do that because it’s bad” feature.
And while teen and young 20 something males will get a huge laugh out of posting screenshots of it, the journalist crowd will freak out and start calling the model, researchers, and company racist.
Personally I have played with large language models and the chutzpah with which they will lie and make things up is indeed astounding (they do a good job at making them sound believable, and lie with utter seriousness and confidence). So I can see where the controversy comes from, although I agree with the other commenters that the researchers should be able to put up a bit fat disclaimer about it
That being said, I am very partial to the AI researchers here who feel like their cool demo has to be taken down because some people were misusing it. It's an unfair high standard they're holding AI demos to, compared with other technologies. It's analogous to asking Alexander Graham Bell to shut down an early telephone prototype because some jerks were using it to discuss antisemitic conspiracies.
Absolutely they can, and his employer could have kept it up. The issue is the phantasmagorical and ridiculous claims about AI-generated scientific research that LeCun peddles. When there's something concrete one can use to test these extremely bold claims, there's a way to at least partially apply a reality check to the claims, and demonstrate their ridiculousness. Which is a very useful and important part of how the scientific field evolves and advances. Feeding non-experts all these wild claims in perpetual future tense only works for so long, and it ought to be that way.
The first step to successfully publish prototypes is creating realistic expectations. That's being done all the time in papers and other ML projects. Instead Meta listed a set of features in a language model that can be summarized as "magic".
1. Summarizing papers:
Here are some examples of paper summaries: https://twitter.com/unboxresearch
I wrote the software that uses GPT-3 to produce those. You can look at the papers to judge the accuracy for yourself.
2. Solve math problems:
What is the area of a circle with radius 2?
The area of a circle with radius 2 is 4π.
3. Generate wiki articles:
Write the first paragraph of a wiki article about tigers.
Tigers are the largest members of the cat family. They are apex predators, primarily preying on ungulates such as deer and bovids. They are territorial and generally solitary but social animals, often requiring large contiguous areas of habitat that support their prey requirements. This, coupled with the fact that they are indigenous to some of the more densely populated places on Earth, has caused significant conflicts with humans.
4. Write scientific code:
Write a matlab function to plot the largest three singular values of a matrix.
function plotLargestSingularValues(A)
s = svd(A);
plot(s(1:3));
I'm not trying the others because I think they're more specific to Galactica.
The models, including GPT-3, absolutely do make mistakes. We are in the first few years of these capabilities existing at this level, so much research is needed before the results are consistently good. In my experience, though, GPT-3 is extremely useful and reliable for many use cases. (Building and using tools like this is part of my career.)
This whole incident was a case study for product management and startup school 101. I've made this exact same category of error in developing products, where I said, "hey, look at this thing I built that may mean you don't have to do what you do anymore!" and then was surprised when people picked it apart for "dumb" reasons that ignored the elegance of having automated some problem away.
If this model were really good, they would have used it to advance a bunch of new ideas in different disciplines before exposing it to the internet. Reality is, working at Meta/Facebook means they are too disconnected from the world they have influenced so heavily to be able to interpret real desire from people who live in it anymore. When you are making products to respond to data and no actual physical customer muse, you're pushing on a rope. I'd suggest the company has reached a stage of being post-product, where all that is left are "solutions," to the institutional customers who want some kind of leverage over their userbase, but no true source of human desire.
The article really fails to explain that LeCun and Marcus have been trading insults for the last few years, it's hardly LeCun snapping at some random person.
If you wrote a flashing big red warning, something like the following, couldn't everybody be satisfied? "CAUTION. This technology is still very early and may produce completely incorrect or even dangerous results. Any output by this tool should be considered false and is only suitable for entertainment purposes until expert human judgement verifies the results."
I'll give you the benefit of the doubt since the demo is now offline but there was in fact a giant disclaimer that said more or less: "DO NOT TRUST THE OUTPUT OF A LANGUAGE MODEL WITHOUT VERIFICATION."
They were trying to have it both ways, where the headline giveth and the fine print taketh away.
In a better world, this could have been released as "We trained a language model on the scientific literature. We're excited because it is starting to draw conclusions, but there is obviously a lot more to be done. See our evaluation here. Have fun and let us know what you find."
The hype is really annoying and seems bizarre--everyone who even remotely cares knows about Meta already.
https://web.archive.org/web/20221116161353mp_/https://galact...
this is just a bunch of personal vendettas imho. The model was useful
The paper was useful for that reason: as a search engine for underresearched and speculative connections within fields, and from what i saw it does well in that. For example, when brainwashing ideas or looking for other research directions to review. In fact, we already have Perspectives papers being published in journals which are speculative in nature.
This is a reasonable take on its usefulness: https://twitter.com/mad_enrico/status/1592664384031330309
> It's still not perfect but a very interesting tool to search about new topics,
Criticism is certainly helpful and necessary for advancing the state of the art, but without something to balance it, it turns into a pretty bleak place to work. I guess this is one example that feels more like a personal vendetta than a constructive criticism of the work.
Unfortunately the model is down so i cant review it.
Sure, it be spitting out these links between mechanisms, but it could have pulled them out of the same place it pulled an article about bears in space: the server's ass. And verification is going to take you far longer on something that actually seems realistic, which is a massive problem.
The usefulness of the model was in under-researched fields and questions, where it can provide actually useful directions.
It's easy to dismiss "bears in space" as not factual. It's harder to dismiss "the links between two underresearched fields in biology" without putting in an exorbitant amount of work. Work which likely is going to be useless if the model is just as happy to spit out "bears in space".
And that was what I had already said. I asked you how you could possibly trust those links it provided. Because you can't. They may very well be novel links that had never been researched. Or they could have been bears in space.
A scientific model which doesn't guarantee some degree of factuality in its responses is completely useless as a scientific model.
Yes, I do think there is value in engaging with the community for models that aren't perfect. But it needs more work before it can be framed as useful tool for scientific research.
* I mean subtle nonsense. Space bears are easy to distinguish..but what if the errors aren't as 'crazy'?
> WARNING: Outputs may be unreliable! Language Models are prone to hallucinate text. Trained on data up to July 2022
The model wasn't supposed to make science for you and any scientist who used it like that should probably not be a scientist. It's a language model, i would think people know what that means by now
Facebook's intent with the model was quite clear; the abstract of the paper says this: "In this paper we introduce Galactica: a large language model that can store, combine and reason about scientific knowledge... these results demonstrate the potential for language models as a new interface for science. We open source the model for the benefit of the scientific community."
I didn't expect it to make science for me- that would be quite impressive (it's been the guiding principle for my 30+ years in science)- but I do expect the demo to be less bad.
It's only bad if you expected it to provide scientific answers, which was not the claim. IIRC the page said something about it compiling knowledge. which it does in this article
EDIT: I mean that it compiles the story of Laika and Karelian bear dogs
In any case , i understand you may have had a falling out with Lecun before, but that is no reason why this research model should not be online for peopel to test it. Let's try to improve things rather than blocking and banning things
The statement about bears isn't just factually wrong, it generated specific details that make it appear right! At first, I was going to say that it wasn't half-wrong because tardigrades (known as water bears) have been sent to space.
This is simply a project that wasn't ready for the real world. A wiser R&D leader would have told the team standards were higher, rather than advising they put a disclaimer on it.
EDIT: since you didn't reply, but just edited (you're probabably at your reply limit depth): laika wasn't a bear dog. She was a mongrel found roaming moscow.
If such an AI is possible, and I won't promise it isn't, it will certainly not be based on a transformer model. I also won't promise it won't "incorporate" such a thing, but it can not be the base.
[1]: Not sure I love this word, I think I'd prefer "confabulate", but hey, I'll run with it.
Imagine a program that had some kind of concept of "interesting and novel mathematical proofs", and it could spit them out. But, 99.9% of the proofs were actually logically inconsistent or true but extremely uninteresting. Would that still be an interesting exploratory tool? I think so.
If they had done an adequate and accurate job explaining what, if any, plausible value or potential Galactica had, warnings would not have to have to carry so much weight. Or their PR could have focused on their tricks and optimizations, and not characterised Galactica at all.
Instead they'll be known as the people who described a science flavored word salad generator as if it was a tool useful for "summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more"
If they had, for example, positioned this as a tool for creating unscientific, but academic sounding anti-vax propaganda, people could have questioned their morals, but not the "fitness to purpose" of their tool.
Please correct me if I am misrepresenting a chain of events here.
A.. tool lands that allows one use language model to generate content. People feed it false data and share that, surprise, data it produces from the data the model is based on is false. How is this a surprise? I am still not sure why Meta would pull it? It can still be useful, but it was made not useful. I am not sure what a proper metaphor is for it, but it is almost like I give you a tool ( lets say a knife ) and you complain that the tool produces bad results when drinking soup.
What am I missing here?
<<or maybe it [Galactica] was removed because people like you [Marcus] abused the model and misrepresented it. Thanks for getting a useful and interesting public demo removed, this is why we can’t have nice things.
<<Meta’s misstep—and its hubris—show once again that Big Tech has a blind spot about the severe limitations of large language models. There is a large body of research that highlights the flaws of this technology, including its tendencies to reproduce prejudice and assert falsehoods as facts.
Also the title of this post is deliberately inflammatory. Should be more like "Head of team that spent months building complex ML system annoyed when people spend undue amounts of time criticizing it."
What's undue about it? If a team spent months producing a webapp that failed to live up to its claims and was full of security vulnerabilities and inaccuracies, nobody'd expect anyone to moderate their criticism of it. Having a Turing award doesn't mean somebody can't deliver a shit product, and I don't see why it should make them more above criticism than anyone else who delivered something similar.
Research "success" certainly takes some talent and LeCun is certainly smart. However, exhibiting that talent also needs luck, timing, and connections, all of which are subject to crazy positive feedback loops: getting an award helps you get subsequent awards, better students, etc. In a world where GPGPUs came a little sooner or later, I think we'd have totally different "superstars".
Their abstract says "In this paper we introduce Galactica: a large language model that can store, combine and reason about scientific knowledge... these results demonstrate the potential for language models as a new interface for science. We open source the model for the benefit of the scientific community."
If I was a reviewer of this paper I would ask them to add (if they haven't so) significant section to the body of the paper highlighting the limitation of the model and the ways it can be misused. Including showing examples of wrong output.
I would then ask them to rewrite the abstract to include something along the lines "We also highlight the limitations of the model including inability to distinguish fact from fiction in several instances and the ways it can be misused and outline some ideas on how these limitation could be mitigated or overcome in the future."
Isn't this the same problem that Github Copilot has?
Fundamentally it has no idea whether code works. It doesn't even know what the problem is.
It just spits out things that are similar to things its seen before, including buggy code from Github repositories.
Not sure why it's so popular. I guess it helps you write status quo code faster (the status quo being buggy and slow) -- I would rather it help us write better code.
Lecun implied on twitter that they 'll get it back. I really hope so
Well, that matches our current experience with human-written Wikipedia articles pretty closely then.
Not only did they exaggerate and hype, but they also didn't even try to solve some of the most glaring issues. The efforts on toxicity mentioned in their paper aren't even mid. They barely put effort into measuring the issue, and definitely didn't make any attempt to mitigate or correct.
Toxicity isn't really the point. Here's the point. If you can't prevent a model from being overtly toxic, then why should I believe you can give any guarantee at all about the model's output? I shouldn't, because you can't.
Galactica is just another a language model. It can be a useful tool. Facebook and LeCun oversold its capabilities and downplayed its issues. If they had just been honest and humble, things would've probably gone very differently.
In some sense, this is good news. The deep learning community -- and generative model work in particular -- is getting a much-needed helping of humble pie.
Hopefully we can continue publishing and hosting models without succumbing to moral panic. But the first step toward that goal is for scientists to be honest about the capabilities and limitations of their models.
----
My account is new so I am rate limited and unable to reply to replies. My response to the general vibes of replies is therefore added to the above post as an edit. Sorry.
Response about toxicitiy:
It's a proxy that they say they care about. I can stop there, but I'll also point out: it's not just "being nice", it's also stuff like overt defense of genocide, instructions for making bombs, etc. These are lines that no company wants their model to cross, and reasonably so. If you can't even protect Meta enough to keep the model online for more than a day or two, then why should I believe you can give any guarantee at all about the model's output in my use case? (And, again, they can't. It's a huge problem with LLLMs)
Response about taking the model down:
I'm not at FB/Meta, but I think I know what happened here.
In the best case, Meta was spending a lot of valuable zero-sum resources (top of the line GPUs) hosting the model. In the worst case they were setting a small fortune on fire at a cloud provider. Even at the largest companies with the most compute, there is internal competition and rationing for the types of GPUs you would need to host a Galactica-sized model. Especially in prototype phase.
An executive decided they would rather pull the plug on model hosting than spend zero-sum resources on a public relations snafu with no clear path to revenue. It was a business decision. The criticism of Galactica and especially the messaging around it was totally fair. The business decision was rational. Welcome to private sector R&D; it works a little different from your academic lab for better and for worse.
Also, let's not pretend that every academic and institution is not overhyping their work. If you read a bunch of academic press releases you 'd think we are on the verge of curing cancer and fusion any day now.
Because it's considered rude to attack people directly; less offensive to say "this product sucks" than "you suck".
That 's a very passiveaggressive way to say "you re overhyping"
Is that really where we want things to be? Because I strongly suspect it's a lot easier for them to make it nicer than it is for them to make the output good.
One of the things I really like about HN is the _lack_ of clickbait titles. Some titles are more informative, some less, but overall I feel like the titles are clear, to the point, and not carefully crafted/engineered to poke the lizard part of my brain in the way that clickbait titles are.
Disclaimer: I haven't read the article so I can't propose a title myself. And with a title like this I'm not going to.
https://truthinadvertising.org/articles/sidebar-ad-language-...
Yann Lecun, which I personally met a couple of times, is in a way another sort of typical character: the ever-childish researcher that likes money a lot, to the point of accepting a prestigious role in one of the most deplorable companies in the modern world (at least from an ethical perspective). He also like attention and public display of status: he can’t resist to pick a fight with Gary. From a pure research perspective he’s long dead.
The question is: do we have enough of those two? Can we move on? Thanks.
It seems like it was good at structuring the writing, both at the article and sentence levels, and for valid prompts it produced accurate responses. But if you entered "write a wiki article about the Alien vs. Predator hypothesis," it would structure it like a wiki article about a scientific topic but just put random AVP stuff it found on the internet into that structure. Why couldn't they just explicitly define which sources are appropriate to pull information from? That seems easier to me than building the actual product they made (but again, I am a layman here).
The ML hype train relies on "garbage in, good out", popularized by an influential paper published at the turn of the century [1]. Anyone with a modicum of experience in experimental science knows that any experiment is only as successful as its ability to collect good data that is representative of the problem, has manageable noise based on known controlled sources and has sufficient coverage of the problem domain to provide meaningful analysis. But of course, if ML admitted that this was a necessary requirement, it would become yet another optimization technique, admittedly new classes of optimization techniques and that promise that the machine learns something new would fall flat on its face.
During much of the 1990's and the 2000's he, along with Geoff Hinton and Joshua Bengio, ignored negative criticism by many naysayers as the three of them persisted on researching deep neural networks, which a majority of AI researchers had dismissed as a dead-end. It wasn't until the late 2000's, when Hinton showed he could train restricted Boltzmann machines efficiently, that other AI researchers started paying closer attention. And of course everyone else piled on after 2012, when a deep neural network (AlexNet) won ImageNet by a wide margin over all other methods.
I certainly got the same negative response when I worked in ML in the 90s- "computers aren't fast enough, we don't have enough data, and we don't have the algorithms" and to be honest, I didn't really have the capability to disprove the people saying that. So I appreciate that he persisted and was successful.
I agree. His behavior in this case cannot be justified.
Written language is a doorway to the full extent of human cognition; unless the problem domain is severely constrained (ie "What is the distance to Mars?"), you are very likely to fall into reflexive traps that rapidly devolve into AGI ("I think, therefore I am?").
Deep Learning was promulgated by several computer scientists, and its still early days. Information security in academia? It’s Spaf. Perhaps it could be argued that his contributions don’t have the depth and rigor as LeCun’s research, but they’re broader, more sustained, and with patient good humor.
But this relatively common pattern does not seem to work all the time.
Quantum Computing, Nuclear Fusion and AI are good counter examples.
Progress in those areas seems to follow a less than linear curve, much closer to a logarithm than an exponential.
We can only speculate about the reasons.
Are we hitting some kind of ceiling?
Life imitates art imitates life.
LeCun invented the convnet and may well have been writing scientific research papers since before the author of that sentence was even born lmfao
- LeCun has a history of getting mobbed by "AI ethics" types on Twitter, and in the past he was very deferential to these folks, and even left Twitter for a while. I wrote about some of that here: https://www.jonstokes.com/p/googles-colosseum
- The MIT Tech Review, which is the author's main source here apart from Twitter, is techlash rag, and they went through a long phase where they only published anti-AI stuff from the "AI ethics" people. Most of those writers I used to follow there on this topic have since moved on to other pubs, and the EIC responsible for this mess has moved on to run WIRED. But it seems they're still publishing the same kind of stuff even with new staff and management. They have exactly one and only one editorial line on AI in general and LeCun in specific, and that is "lol AI so racist and overhyped!" It's boring and predictable.
- LeCun has a longstanding beef with Marcus, and the two treat each other pretty poorly in public. Marcus seems to have a personal axe to grind with LeCun. Given that Marcus has been leading the mob on this, it's not shocking that LeCun got crappy with him.
- Emily Bender, Grady Booch, and the other folks cited in the MIT Tech Review piece all, to a person, have exactly one line on AI, everywhere at all times and in all circumstances, and it's the same one I mentioned above. You could code a bot with a lookup table to write their tweets about literally anything AI-related.
- Yeah, LeCun is a prickly nerd who gets his back up when certain people with a history of attacking him come after him yet again. He should probably should stay chill.
- "AI so overhyped" is a pose, not an argument, an investment thesis, or a career plan. But hey, you do you.
Anyway, I hate to be defending anything Meta-related, but this article is slanted trash, its sources haters who have only one, incredibly repetitive thing to say about AI, and the author is a hater.
Sad because I am an Ars Technica fan, but can empathize with GP's point about AI coverage being repetitive and often over-hyping results.
> Anyway, I hate to be defending anything Meta-related, but this article is slanted trash, its sources haters who have only one, incredibly repetitive thing to say about AI, and the author is a hater.
I was hoping someone else would also find the actual context and background for what is clearly a poorly written hate piece.
I have very little interest in Meta, but the amount of FAANG hate that the media knowingly perpetuates and the amount of criticism that anything launched receives, would absolutely wear someone down. Lecun is no saint, but the article is very unfairly written.
> Second, what’s the endgame here? What’s LeCun’s ideal?
This section I think is particularly exemplary - regardless of your take here, I think it’s pretty ridiculous and uncharitable to interpret “criticism of science is immoral” from LeCun’s quoted statements.
I was quite familiar with Lecun's dust up with Timnit Gebru on Twitter, and I had a lot of sympathy for him in that situation.
I think it's quite sad that so much bad-faith argument has infiltrated academia to the extent that it has. Some may say it's always been that way, but it feels worse to me now. One of my "heroes" of unbiased rationality, Zeynep Tufekci, wrote a really good Twitter thread recently about how some of these flat out liars in academia manage to continue their lies unscathed with little pushback: https://twitter.com/zeynep/status/1592210111359250432
It doesn't "source haters". It quotes MIT Tech Review and Gary Marcus precisely in order to provide context for the subject of the “article” (blog post): LeCun's bizarre "this is why we can't have nice things" statement. This petulant remark seeks to shut down negative feedback as a class, regardless of its merits. That's what the blog post is about. The quotes are there so that the quote from LeCun makes sense, not because they're legitimate criticism.
> Emily Bender, Grady Booch...
These people are not mentioned in the linked post, which tells me you're pattern matching on "techlash" and posting a bunch of only vaguely related context (and a medium self-link).
> He should probably should stay chill.
Exactly?! That's the point of the blog post. It's in the title of the article. It seems obviously true, and I'm not sure how any of what you say adds up to a robust conclusion that "the article is slanted trash" and "the author (Andrew Gelman?!?!) is a hater" other than you don't like some of the quotes.
> and in the past he was very deferential to these folks, and even left Twitter for a while
How many times has he quit Twitter now? Three IIRC. Seems like he needs some coaching on the following through with promises.
https://www.technologyreview.com/2022/11/18/1063487/meta-lar...
Twitter-quotes both of them and others
Are these a new type of luddite? Why aren't there "computer ethicists" complaining about real issues with the use of technology in general? I'm being tracked at all times without my consent. These "AI ethicists" are happy to use platforms like Twitter that track you, and even reward you for giving them more info (e.g. phone number).
Every week there's a random here post about "some AI detection system closed my Gmail account / took down my Android app / froze my Square funds", and Hacker News is seen as the semi-official tech support line for companies who have turned to biased AI to cut costs.
A lot of what AI ethicists are saying is that "if we hook these AI systems up to safety-critical systems, anyone who doesn't fit the model is going to be labeled an outsider", and I don't know why we shouldn't repeat it as many times as it takes to get people to listen... accounts are still being banned, lives are still being ruined.
To counter this with "think about what progress AI has been making!" is missing the point. "Sure, it Markov-chain'd some random facts about space bears and cited random people with papers it made up who are now caught in the cross-fire of machine hallucination, but think about the progress! It could format its fiction to look like a TeX paper and add some random squiggles that look like math expressions!" is not the slam-dunk defense you think it is.
I would agree with this if I ever saw these self-appointed AI-ethicists focus on these kinds of harms. But, at least in my experience, is usually focused on the exact same set of concerns that 90% of the time has "intersectionality" somewhere in the criticism.
Yes, I'm being a bit unfair and snarky, but I'd be more willing to pay more attention to some of these criticisms if I felt it included more of the harms you bring up than just what I feel has become a constant bone to pick. I agree with the GP when he wrote "You could code a bot with a lookup table to write their tweets about literally anything AI-related."
You have the choice to avoid google accounts and limit the destruction a google AI system can do to you.
You don't have a choice to not be born black, and not be put in jail for longer just because you are black.
Why don't you care that millions of people will be hurt by these things, and care more that an app developer gets locked out of the app store? Apple hasn't put anyone in jail.
These AI ethicists are complaining about all of this, but of course they yell more loudly about sexism and racism, because, you know, those are fairly serious things that should be addressed first???
I don't think they need to be original, I think they need to bang their drum loudly. "Oh, that women's suffrage movement won't shut up about how they don't have a voice in policy that governs their life, can't they talk about something else for once" isn't an indictment of the people complaining, it's an indictment of the people not listening.
[0] https://www.propublica.org/article/machine-bias-risk-assessm...
[1] https://www.propublica.org/article/yieldstar-rent-increase-r...