21,863 karma · joined July 14, 2013
Gadgeteer.
Passionate about understanding how things work, and making things work. Hoping to learn about tools and techniques that will make my work more productive / fun.
The distinction between theory and experiment is somewhat arbitrary. Historically, scientists were better at one than the other, with a few rare people being known for both. To make an analogy, there's also no immovable boundary between hardware and software, since most hardware runs on code, but these tend to be different jobs and even departments.
What tends to happen in science is that when a subject is virtualized, e.g., when it can be modeled by a satisfactory theory, scientists move on to new frontiers. Many of the most interesting areas of work are where both theory and experiment can challenge one another.
I've had managers and even engineers ask me in all sincerity why we still need to make measurements when we can just model the data.
I refer to what I think the author is asking for, as "push button reproducibility," i.e., the idea that the results will reproduce themselves at the push of a button, anywhere, at any time in the future. I have a couple of misgivings about this.
First, the whole idea of "open" research predates computer technology. Forcing science to keep up with the latest ideas in software distribution is too much of a burden, when science is already too risky and slow. I had the odd privilege of learning the scientific method from my mom, before there was widespread access to computers. Her version was that a study should be reproducible by a reasonably skilled person. This is a greatly relaxed standard, but is realistic for a discipline that spans decades if not centuries.
I supplied all of the data and code for my thesis research (and a sufficient number of mechanical and electrical drawings). But nobody has Turbo Pascal today, and some of the commercial instruments such as specialized lasers were already obsolete by the time I finished. Also, the experiment was dangerous, and might not pass safety review today. It required about $500k of equipment and a dedicated lab.
Today I have the luxury of saying that if my code fails upon loading a new version of a dependency, the person who discovers that failure is probably skilled enough to fix it, and my work rarely hinges on the idiosyncracies of dependency versions. If it goes into a product, they'll totally rewrite it anyway.
Second, science is still at its core an experimental discipline. Even in physics, there are more experimentalists than theoreticians. Reproducibility means roll up your shirt sleeves and head for the lab. To this day, some processes have not been mechanized, and you still need to spend years developing "lab hands" which not all people succeed at.
I think there's a clue in the fact that the social and medical sciences seem to be the most deeply embroiled in the reproducibility crisis. It's because the quality of results depends on the the quality of measurements, and it's just harder when dealing with living subjects or one-of-a-kind specimens (such as the earth's climate). In fact, not much more than a century ago, it was believed that studying those things was beyond the reach of scientific methodology.
Third, we're not going to stop doing science in areas where it's hard, particularly in medicine, but we're also not going to staff up in areas such as software development, to make science work better. People are suffering from disease right now so there's always an urgency to finding cures, plus an obvious profit motive.
Disclosure: Experimental physicist, developing better measurement equipment.
Disclosure: Old scientist.
I'm 62, and grew up near Detroit. I remember cars developing major rust holes after just a few years. The rusting also affected the inside of the engine compartment, and it did make repairs more difficult. There was an industry of aftermarket rustproofing shops, that were a scam. Every teenager with a car (which was still rare) learned how to patch rust holes with Bondo.
My first car, the frame rusted out and had to be welded back together.
Toyota and Honda put a stop to that. They did a number of things: Better steel alloys, better coatings, painting the bodywork before it could start rusting, eliminating places in the body where water could accumulate, etc.
Today, you rarely see totally rusted out cars on the road.
An analogy is making students write 5-paragraph essays. We don't really believe that the knowledge of how to write a 5-paragraph essay is a real-world ability, and the LLM's can write them all day long. But for some reason we believed that 5-paragraph essays were a good pedagogical tool.
But whether the same heuristics apply to teaching an LLM is anybody's guess. If they learn differently than we do, then feeding them on our learning tools isn't necessarily going to help them.
There's an interpretation that makes the reference answer correct. I didn't realize this until halfway through teaching the math class: Based on the lectures, textbook, and homework problems, you match the "form" of the problem with similar problems in the textbook, and apply the same algorithms to solve it. The students have some vague idea of this, referring to it as finding the "trick," but it's never explicitly explained to them.
I was glad for the interesting puzzles and good grades. But physics really came alive for me in the lab, where mother nature decides the conditions of the problem being solved.
I'm thinking of something like a violin. You can't pick up a violin and judge the perfection of the fitting between the wood surfaces that are glued together. If that work isn't perfect, the instrument will eventually fly apart. But you can look at the "scroll" and "bees sting" which are unforgiving of tiny errors. These kinds of features persist to this day due to traditionalism, but have disappeared from most industrial products.
Today, I don't expect an ornamented house to be of any better quality than a plain house.
My family lives in a MCM house built around 1960. We bought it in a "hot" market where you couldn't be particular about anything. We chose a house based on location and the interior.
I take this seriously, but I think this a problem with the cops. Even regular people hesitate to report problems if they think the police might overreact.
So it is indeed a mixture of reading and improvisation. But this is aside from the choice of notation, which is just a practical matter of finding people who can read it fluently enough to perform from it.
Maybe their needs to be a “music theory for engineers” like we have “physics for poets.”
If programming is an analogy, it's said that once you've learned your first language, you can learn any other language in a jiffy. Also, a lot of higher level learning is "learning how to learn." Maybe it's better to have a standardized "first language" that students can learn in depth, and then let them move into other areas at their own discretion.
I don't have a college degree, but for me, classical music lessons were a great preparation for learning jazz, and I've picked up pop, rock, and folk music on my own. In fact, on the instrument that I play, the double bass, gaining proficiency and experience in jazz without that background is a brutal learning curve.
— Long time subscriber.
Aristocrats were a dime a dozen.
Something else could mean developing a new concept of ownership or a new form of redistribution.