[1] https://inis.iaea.org/records/fe51q-17w28/files/35015774.pdf
9,025 karma · joined April 27, 2016
[1] https://inis.iaea.org/records/fe51q-17w28/files/35015774.pdf
Surely you mean on the side of extractors? The Ottoman Empire practiced mass movement of people (sürgün), basically settler colonialism; earlier Umayyad and Abbasid caliphates are among the largest empires in history, and their population was mass converted to Islam.
Sure, you replace "Jews" by "Zionists" and then every trope is ok. It's Zionists who have the power to influence or direct US government behavior (i.e., control it).
> Two members of congress? Out of hundreds?
Well yeah, to show something possible it's enough to show one example.
> Yes!
How is being elected to Congress political suicide?
> It's not even debatable that the US is absurdly pro-Israel.
I'm not debating that. I'm debating the idea that this is somehow doing of evil Zionist lobby or that repeating antisemitic tropes is not antisemitic because you euphemize Jews by Zionists
But how is it different from what arena or matharena does?
> That sort of thing is not showing broad intelligence anymore than a person both knowing a chess player and a poet is having broad intelligence.
The claim is that these problems require somewhat broad intelligence by themselves, as opposed to specialization into specific task while unable to do anything else.
I think a general model that can
- finish nethack, doom, zelda and civilization,
- solve the hardest codeforces/atcoder problems,
- formally prove putnam solution with high probability, not given the answer
- write a PR to close a random issue on github
is likely to have some broader intelligence. I may be mistaken, since there were tasks in the past that appeared to be unsolvable without human-level intelligence, but in fact weren't.
I agree that such benchmarks are limited to either environment with well-defined feedback and rules (games) or easily verifiable ones (code/math), but I wouldn't say it's super narrow, and there are no non-LLM models to perform significantly better on these (except some games); though specialized LLMs work better. Finding other examples, I think, is one of the important problems in AI metrology.
> So, yes, we are kind of left with vibe checks, but in theory, we could do more; take a bunch of models, double-blind, and have a big enough, representative group of human evaluators score them against each other on meaningful subjects.
You've invented an arena (who just raised quite a lot of money). Can argue about "representative," of course. However, I think the SNR in the arena is not too high now; it turns out that the average arena user is quite biased, the most of their queries are trivial for LLMs, and for non-trivial ones, they cannot necessarily figure out which answer is better. MathArena goes in opposite directions: narrow domain, but expert evaluation. You could imagine a bunch of small arenas, each with its own domain experts. I think it may happen eventually if money flow into AI continues.
- live benchmarks (livebench, livecodebench, matharena, SWE-rebench, etc)
- benchmarks that do not have a fixed structure, like games or human feedback benches (balrog, videogamebench, arena)
- (to some extent) benchmark without existing/published answers (putnambench, frontiermath). You could argue that someone could hire people to solve those or pay off benchmark dev, but it's much more complicated.
Most of the benchmarks that don't try to tackle future contamination are much less useful, that's true. Unfortunately, HLE kind of ignored it (they plan to add a hidden set to test for contamination, but once the answers are there, it's a lost game IMHO); I really liked the concept.
Edit: it is true that these benchmarks are focusing only on a fairly specific subset of the model capabilities. For everything else vibe check is your best bet.
It's not about whether it's good or bad idea. To make follow-up experiments you need to first reproduce the original experiment. That's why faking "big" experiments like Schön could never work.
> Microsoft built so much work and spent so many resources building on top of it. It's not foundational to the entire field but it is foundational to a lot of follow-up research.
Will all due respect, a single group (even large one) doing a single type of experiments (even important and complicated one) is not a lot of research. Also, Microsoft knew about data manipulation, that why they moved the experiments in house. They didn't do experiments under assumption that the early Majorana papers are correct, then they wouldn't need to develop their own complicated (and somewhat controversial) protocol to detect Majoranas. It was quite clear for everyone that regardless of data manipulation people were too optimistic interpreting Majorana signatures in these early papers.
This research wasn't foundational to a lot of work. Most of important/foundational works in quantum (doesn't matter if computing or general, I'm not sure which one you meant) are verified. How can you possibly base your experimental work on someone else's work if you can't replicate it?
[1] https://fixupx.com/CraigGidney/status/1907199729362186309
Whether not doing everything to maximize your GPA is "stupid" (literally or figuratively) is a good question too.
But even if your assignments influence your GPA it's rarely the only thing that does, and not doing assignments will harm your ability to perform in midterm/exam/whatever.
I agree that making assignments not designed with external sources in mind significantly impact the final grade is not ideal. I think this is minor and easily fixable point rather than some failure of the whole education system.
Why would they be stupid? Were people before LLMs stupid for not asking smarter classmate/parent/paid contractor to solve the homework for them?
Large part of education is learning about things that can be easily automated, because you can't learn hard things without learning easy things. Nothing conceptually changed in this regard, like Wolfram Alpha didn't change the way differentiation is taught.
Do you think that the whole "diversity statement" and DEI shticks were exactly tools of removal of people with different political views from academia?
US academia political activity is what made it attractive target for Trump administration. I'd prefer that it didn't happen or that US didn't have a president willing to destroy the academia, but I'd also prefer that universities would act as non-partisan entities. I think Boaz Barak summarized it well [1]
On a separate note, US searching/detaining/refusing entrance to other countries' nationals happened all the time [2,3,4,5,6,7], I don't think it's newsworthy or should change your position on visiting US.
[1] https://www.thecrimson.com/article/2025/3/28/barak-harvard-m...
[2] https://www.straitstimes.com/asia/east-asia/us-entry-horror-...
[3] https://firrp.org/wp-content/uploads/2024/09/September-2024_...
[4] https://immigrantjustice.org/press-releases/three-travelers-...
[5] https://www.vice.com/en/article/the-us-keeps-mistakenly-depo...
[6] https://www.propublica.org/article/can-customs-border-protec...
[7] https://thepienews.com/indian-government-gives-student-trave...
[1] https://www.documentcloud.org/documents/25894225-dhs-documen...
It is very cool to hear from you!
> In my opinion, the citations above do not represent a balanced view of the Majorana field status but are rather negative.
That's true, but the goal of the citations was to demonstrate there are some negative opinions too. Maybe together with positive OP these form a balanced view.
I understand that it can be very unpleasant to have people like Frolov or Legg trying to prove you're wrong, but I think it shoudn't be personal (from either side). Trying to find alternative explainations is part of science. And Frolov did turn out correct in past, and we did think we found Majoranas when in fact we didn't, and this part of the story can't be just ignored. Citing Feynman "The first principle is that you must not fool yourself and you are the easiest person to fool". While it's tempting to dismiss the critics as broken record, I think it would both increase the credibility of the studies and improve the science if their criticism was taken at the face value. Answering specific points publicly would also create more balanced picture. I'm not aware of the responses to the cited opinions that I could cite to "balance out."
> We published two experimental papers recently that went through a rigorous peer review process.
Peer review is important, but is not the answer to specific claims, eg that TGP accuracy is overestimated, or (if we take Henry's word for it) the promised errata that never came out.
> Finally, we have exciting new results that we just shared with many experts in the field at the Station Q conference in Santa Barbara.
I've read about it from Das Sarma's twitter [1]. It does indeed sound exciting. If you're able to manipulate, store, and read out quantum data from qubit, then I think people will have easier time to agree you have one. There is of course question of non-Clifford gates, but that's a separate problem.
> We will share more broadly at the upcoming APS March meeting.
I look forward to hearing about it. If you (or someone from your team) are interested, I'd love to meet and chat at MM. My contacts are in bio.
Edit: I've also now seen Chetan Nayak's comment in Scott Aaronson blog with some details [2].
The idea behind topological quantum computing is to utilize quantum materials whose low-energy physics looks like an error correcting code. Since these systems are very large (macroscopic number of atoms), the error rates are (theoretically) very low, ie the qubit is fault tolerant by construction, without any additional error correction. In reality, we do not know how good these qubits will be at finite temperature, with real life noise, etc.
Moreover, these states do not just occur in nature by themselves, so their construction requires engineering, and this is what Microsoft tries to do.
Unfortunately, Majoranas in nanowires have some history of exaggerated claims and data manipulation. Sergey Frolov's [1] twitter, one of the people behind original Majorana zero bias peaks paper, was my go-to source for that, but it looks like he deleted it.
There were also some concerns about previous Microsoft paper [2,3] as well as the unusual decision to publish it without the details to reproduce it [4].
In my opinion, Microsoft does solid science, it's just the problem they're trying to solve is very hard and there are many ways in which the results can be misleading. I also think it is likely that they are making progress on Majoranas, but I would be surprised if they will be able to show quantum memory/single qubit gates soon.
[1] https://spinespresso.substack.com/p/has-there-been-enough-re...
[2] https://x.com/PhysicsHenry/status/1670184166674112514
[3] https://x.com/PhysicsHenry/status/1892268229139042336
[4] https://journals.aps.org/prb/abstract/10.1103/PhysRevB.107.2...
It is unfortunately unclear how good the topological qubits practically are.
The updates are in this case, at least assuming others are doing something similar, is to increase expected price if you see sell for higher price and reduce if you see the sell for lower price. To figure out how much to update would be the hard part.
[1] https://opportunityinsights.org/wp-content/uploads/2024/01/S...