HNHacker News
TopNewBestAskShowJobs

EvgeniyZh

9,025 karma · joined April 27, 2016

https://evgeniizh.com/ https://t.me/j_links https://twitter.com/evgeniyzhe
submissionscomments
EvgeniyZh··on Terence Tao on O1
There is ~3 order of magnitude more Python code in the internet than Lean code (200GB vs 200MB in the stack v2). You can't tune it "the same way"
EvgeniyZh··on Computer scientists prove that heat destroys quantum entanglement
Well the naming is a different question, but it is consistent with observation of what we call wavefunction collapse.

In Schroedinger cat experiment, the evolution is not thermalization but rather entangling the whole cat with atom state. I don't think we're really puzzled about it now, since we know that macroscopic objects can be in superposition. So cat, theoretically, can be in superposition of alive and dead until measured. I don't think there is any contradiction or confusion here physics-wise.

EvgeniyZh··on Computer scientists prove that heat destroys quantum entanglement
> evolve unitarily and do not destroy a superposition

Unitary evolution can destroy superposition of the small subsystem by introducing negligible amount of superposition into large system.

EvgeniyZh··on Computer scientists prove that heat destroys quantum entanglement
This paper talks about Gibbs states, i.e., thermal states. These are very specific states that naturally occur if the system is in thermal equilibrium.

The other thing you talk about is true. If you have a system in quantum superposition, to measure it you need to entangle it with some macroscopic system. Macroscopic systems decohere very fast. This concept is well explained in excellent John Preskill's quantum computing lecture notes [1]

Note that the system for which wavefunction collapses is not in equilibrium with the measurements equipment (otherwise it wouldn't be measured), so the claims of the paper do not apply to this case.

[1] https://web.archive.org/web/20231005202201/http://theory.cal...

EvgeniyZh··on Author-paid publication fees corrupt science and should be abandoned
Nobody? The whole premise is that we should somehow start awarding people for working on important questions which we do not do now.

The usual critique of academia is that people are expected to publish a lot (which creates wrong incentives) and that nobody really accesses quality of the publications (which gives opportunities to cheat). At the same time there is huge PhD overproduction, which creates competition.

I don't think the problem is that people are working on wrong problems, the problem is how they work on problem of their choice and why.

I also don't think there is a simple fix to this problem.

EvgeniyZh··on Author-paid publication fees corrupt science and should be abandoned
There are countries where student loans are not a thing and the science there arguably faces the same problem.

PhD is not paid by a student even in US, it's all scholarships.

EvgeniyZh··on Author-paid publication fees corrupt science and should be abandoned
Who decides what's important?
EvgeniyZh··on Japan on edge of EUV lithography chip-making revolution
Canon and Nikon have these already, and yet their market share is pretty small
EvgeniyZh··on Author-paid publication fees corrupt science and should be abandoned
What kind of reform?
EvgeniyZh··on Confessions of a Theoretical Physicist
What do you mean by "how it works"? In my book, being able to predict the outcome of experiment is exactly understating how it works. Anything beyond this is the question of philosophy, not physics.
EvgeniyZh··on The AI Scientist: Towards Automated Open-Ended Scientific Discovery
What you describe is quite different from what the original post describes.

Also I'd say that what you describe is quite different from what many people believe is the problem with research, either academic or industrial.

Academic research has a problem of low-quality outputs, but it's not necessarily "hard to apply" kind of problems. There is no "engineering department" that comes with problems to people in academia, and there is no inherent expectation of the research to be immediately applicable or applicable at all. A lot of high quality research is not immediately applicable and is not engineering-oriented, in fact. And this kind of research is still worth automatization, just like it is worth doing while we can't automatize it. There is of course a feedback between experiment and theory, but I would say it's the thing that's broken in the academia.

As for industrial research, which I guess you referred to in first place, I have less experience with it. But the goal of industrial research is not publishing papers, so it's not in their KPI, and they won't be drinking champaign for long if they are publishing junk papers instead of doing what they're hired for. Note that not any research done in industry is "industrial research" by this classification, and companies do academic research occasionally (Bell Labs or Google Brain for example). So if industrial research lab is created to solve engineering problems and doesn't do it properly, then, well, any sane company will fire them. It doesn't have wrong incentives problem like academic research, because its incentive is to make money for the parent company.

Now back to AI. Assuming theoretical research is easier for AI, the bottleneck is going to be human doing experiments. Obviously, AI output should be better than human output for it to be tested by experimentalists. What I was saying is that even worse, since the volume of AI output is higher, and the usual heuristics (name of author, personal discussions, etc.) won't work for filtering AI output, it should be better than high-quality human work and not just average human work. I'll be happy if this can be achieved, but we are very far from there.

EvgeniyZh··on Are you better than a language model at predicting the next word?
Given the high scores, I guess it was an easy one. I've taken the longer one, and got the following

> You scored 28/100. The best language model, gpt-4, scored 32/100. The unigram model, which just picks the most common word without reading the prompt, scored 28/100.

Assuming complexity averages out on N=100, small test with LLM score above ~5 is "easy"

EvgeniyZh··on The AI Scientist: Towards Automated Open-Ended Scientific Discovery
> If AI churns out 25 scientific models per day, the proper venue isnt publishing them on arxiv or Nature magazine. It’s testing their predictions and putting them up the same way HuggingFace does.

Unless the testing will be done by AI, I doubt human scientist will bother with testing tons of incomprehensible models, unless the accuracy of these models will be exceptionally high

EvgeniyZh··on You got a null result. Will anyone publish it?
The question is how many of them are willing to review and read these publications. Of course as an individual scientist (not me, but someone who does experiments), I'd love to capitalize on my work, even if it is unsuccessful (in the sense of null result), by publishing it. But do I, and scientific community in general, care about null results? I'd say mostly no. Null results, if universally published, would overwhelm already overwhelmed publication system.

If you think it will be helpful to others to know about specific failure, put it in a blogpost or even on arxiv. Or talk about it at conference (for CS, workshop).

Also, if we use publications as a measure of scientists success, and we do, is a scientist with a lot of null results really successful?

EvgeniyZh··on Microsoft cutting crucial link to Gaza, Palestinians say
What's disengenious about? They didn't find evidence of famine, and thus the claim "According to the UN, there is now a famine in Gaza" is factual wrong. The sprinkling of ideology on top of this fact doesn't change it.
EvgeniyZh··on Microsoft cutting crucial link to Gaza, Palestinians say
Quick check: famine review committee was unable to find evidence of famine in Gaza [1]

[1] https://reliefweb.int/attachments/c6421cb7-c936-4145-af54-b7...

EvgeniyZh··on Show HN: Simple script to cripple personalized targeting from Facebook
Reminder to use AdNauseum [1] that will not only hide the ads but also click it, messing with targeting and spending advertisers' money at the same time.

[1] https://adnauseam.io/

EvgeniyZh··on Spending 3 months investigating a 7-year old bug and fixing it in 1 line of code
I recently found a bug whose fix amounted to one-liner. It all started with random CI failure when I was working on adding some new functionality.

I've rerun test locally -- no fail. I've changed the seed to one that was used in failing run -- nothing.

I add a loop to the code to repeat text hundred times -- still nothing. I run the test in bash loop hundred time -- 3 fails. So this already hints on some internal problems. I fixed every possible source of randomness and verified that all the inputs are identical between the runs -- still fails only once in a while. I started building MWE, but the function involved in reproduction is fairly complicated. I'm left with a hundreds lines of Jax code which fails in couple of percent of cases.

I look at the output of the compiler, and it is identical between failing and successful runs. So the problem is in the compiled code. The compiled code is ~1000 lines of HLO (not much better than assembly). Unfortunately HLO tooling is both unfamiliar to me and not well fit to this case (or at least I couldn't figure it out). So I start manually bisecting the code. I'm finally left with ~30 lines of HLO. It fails even less often (1% maybe), but at least it runs fast. It also seems to fail in exactly the same way (i.e., there is single incorrect output that I've between 3 fails). Now that's something maintainers can be hoped to look at.

It turned out that matrices with same content but different layout were deduplicated, leading to, in my case, transposed matrix being replaced by non-transposed one. The hash used for storage did take layout into account so the bug appeared only if two entries ended up in the same bucket (~3% of times). The fix was an obvious one liner [1].

[1] https://github.com/openxla/xla/commit/76e7353599d914546f9b30...

EvgeniyZh··on Cost of self hosting Llama-3 8B-Instruct
1B of tokens for Gemini Flash (which is on par with llama3-70b in my experience or even better sometimes) with 2:1 input-output would cost ~600 bucks (ignoring the fact they offer 1M tokens a day for free now). Ignoring electricity you'd break even in >8 years. You can find llama3-70b for ~same prices if you're interested in the specific model.
EvgeniyZh··on How Meta trains large language models at scale
TPU v5p is ~2 times slower than H100 at larg(ish)-scale training (order of 10k chips) [1]. And they already have v6 [2]. I think it's safe to say that they are fairly close to Nvidia in terms of performance.

[1] https://mlcommons.org/benchmarks/training/

[2] https://cloud.google.com/blog/products/compute/introducing-t...

EvgeniyZh··on Scandal at America's top science fair
Any test checks how good are you at taking this test. I've not seen evidence that at some complexity level it stops correlating with success in university, have you?
EvgeniyZh··on Scandal at America's top science fair
Any system can and will be cheated, that wasn't the point of neither OP's or mine proposals.

Standardized testing is a good predictor. In my country people are admitted based on standardized testing only (country wide subject exams and sat-like test) or results of university-adjacent "preparation courses" and it works fairly well. The affirmative action is realized directly as a bonus points to your scores based on the background, which reduces the effect of ethical gaming.

EvgeniyZh··on Scandal at America's top science fair
Just make tests hard enough so that you wouldn't need lottery
EvgeniyZh··on 93% of paint splatters are valid Perl programs (2019)
The opposite is definitely true
EvgeniyZh··on How to graduate your PhD when you have no hope
> It's not the consequence that makes a problem important, it is that you have a reasonable attack.
EvgeniyZh··on A muon collider could revolutionize particle physics, if it can be built
It's not that simple. There are many smaller scale experiments in HEP, all kind of dark matter detectors among them. Colliders are irreplaceable in a sense that we do not know how to get higher energies without them, but it's unclear for me that just getting higher energies is enough.

Energy scales for new physics can be enormous for all we know, I do not see any reason to expect that new physics hides "around the corner" (in a 1.5 orders of magnitude FCC or muon collider could give us). Muon collider is at least looking in a different place, and it's definitely more interesting than FCC for me, but I'm still not 100% sure it is the right thing to do, given the limited resources.

EvgeniyZh··on Journalist casualties in the Israel-Gaza war
I'll rephrase. What rate do you expect in city fights against the enemy mixed with civilians and actively using them as human shields, assuming that the attacking side makes reasonable effort to minimize civilian casualties.

> intent to minimize civilian casualties

Intent is very hard to argue about. Especially when talking about proportionality which is not rigorously defined. Numbers are easier to argue about. It is reasonable to expect that civilian casualty rate will be higher in cases where there is no intent to minimize civilian casualties compared to cases where it is; comparing numbers with conflict in similar conditions where you believe there was such intent is a good starting point

> enable any sort of progression into a longer holding peace in that area

Getting rid of terrorist government who promised to repeat 7th of October looks like necessary condition for peace.

EvgeniyZh··on Journalist casualties in the Israel-Gaza war
Do you imply that if you're attacked by an army mixed with civilians you're just supposed to surrender and not make a single shot? Luckily people who wrote the Geneva and other conventions understood that it made zero sense.
EvgeniyZh··on Journalist casualties in the Israel-Gaza war
> No recent war has seen anywhere near the deplorable civilian:combatant death ratio that we see here.

Can you give the numbers for comparison? What are acceptable rate in your opinion?

EvgeniyZh··on Journalist casualties in the Israel-Gaza war
> That doesn't mean they are using human shields

Being indistinguishable from civilians (blending in) is exactly using civilian as a shield -- the opponent can't identify you and can't kill everyone who looks like you

> They can still minimize civilian death

Arguably Israel does it with fairly low civilian: combatant death ratio

← PreviousPage 5 of 14Next →