HNHacker News
TopNewBestAskShowJobs

currymj

3,822 karma · joined March 27, 2016

submissionscomments
currymj··on Trump will kill CHIPS Act by gutting NIST employees
i often wonder about this.

because if you look at Big Law, investment banking, or private equity, there are a lot of highly-educated Americans who are willing to work as hard as anyone on earth.

so it's got to be something about TSMC's recruiting pipeline, or perhaps what motivations the Americans who get educated for the skills they need have, rather than specifically American laziness.

currymj··on Penn to reduce graduate admissions, rescind acceptances amid research cuts
most of the general public doesn’t know PhD students get paid stipends.

if they do know that, they don’t realize how tightly each term’s stipend is tied to a specific funding source.

currymj··on New horizons for Julia
those limitations are sometimes real.

However, I have read enough Fortran and C++ written by scientists, to fervently wish that their code had been written in Julia instead. In Julia, I don't think they could manage to write code so debauched. If you just do things the naive way, usually life is good and code is both readable and fast.

The pitfall is type instability and memory allocation, but I think with a few basic tricks, people can learn to avoid this, especially for simpler tasks. (edit: also, if people do mess this up, they notice because their code is slow. whereas people don't notice if their code is fast but incomprehensible and impossible to maintain or extend.)

currymj··on New horizons for Julia
The article describes Julia as "a general-purpose programming language aimed at science, engineering, and related arenas of technical computing".

I think this is about right. If you are going to solve differential equations or run some numerical weather codes or solve an integer program or simulate an electrical circuit, it would be a good idea to consider Julia rather than reaching for Fortran, C++, or MATLAB. It is a much nicer language and still very fast.

My experience is that people who want to use Julia for data science and especially for deep learning, usually find that you can get the job done, but often it's easier to use PyTorch or Jax and just accept the limitations of Python.

currymj··on New horizons for Julia
for problems where Jax works well, in theory there could be a great Julia equivalent, but in practice one has not emerged. instead every 3 or so years, a brilliant solo researcher writes an even better autograd library which is ultimately abandoned.

the language is at its best in other areas. for a few things (differential equations, mathematical optimization) it is actually state of the art.

currymj··on Bookshop.org launches Kindle alternative, sends e-book sales to local bookstores
android e-ink tablets are now a decent user experience and let you read ebooks through the apps. there are many form factors including e-reader like.
currymj··on Wyden Releases Draft Bill to Secure Americans' Communications
if an individual pharma employee makes a donation as an individual, many of the datasets will classify it as “money from the pharmaceutical industry”. worth considering.
currymj··on Diffusion Without Tears
Thanks for sharing this! I tend to agree that it’s easiest to understand this way.

I just find it a frustrating fact about modern machine learning that in fact, the nice SDE interpretation is somehow not the “truth”, it’s just a tool for understanding.

currymj··on Diffusion Without Tears
it makes intuitive sense, but at the same time, you can get good performance replacing the Gaussian noise with deterministic transformations.

so in the end diffusion can’t explain that much about how diffusion models work. disappointingly, so many modern ML things are like this.

currymj··on Does X cause Y? An in-depth evidence review (2021)
even if you hit all the assumptions you need to make Pearl/Rubin causality work, and there is no unobserved factor to cause problems, there is still a philosophical problem.

it all assumes you can divide the world cleanly into variables that can be the nodes of your DAG. The philosopher Nancy Cartwright talks about this a lot, but it’s also a practical problem.

currymj··on Fewer students are enrolling in doctoral degrees
consider Switzerland. PhD salary is around the range of 80k CHF. It's not bad at all.
currymj··on Fewer students are enrolling in doctoral degrees
their collapse didn't have anything to do with computer systems failing though.

they just spent a period of about 10 years making sure they were involved in every major financial scandal due to poor judgment.

currymj··on Thomson Reuters wins first major AI copyright case in the US
Yes, though maybe not explicitly written down.
currymj··on Thomson Reuters wins first major AI copyright case in the US
if you have an MNIST classifier that just takes in images, and spits out a probability of digits 1-9, that wouldn't necessarily be generative, if it is only capable of modeling P(which digit | all pixels).

But many other types of model would give you a joint distribution P(which digit, all pixels), so would be generative. Even if you only used it for classification.

https://en.wikipedia.org/wiki/Generative_model

I guess these days "generative" must mean "it is used to generate outputs that look like the training data".

But until recently, the meaning had to do with the information in the model, not how it's used.

currymj··on Thomson Reuters wins first major AI copyright case in the US
Generative is a technical term, meaning that a system models a full joint probability distribution.

For example, a classifier is a generative model if it models p(example, label) -- which is sufficient to also calculate p(label | example) if you want -- rather than just modeling p(label | example) alone.

Similar example in translation: a generative translation model would model p(french sentence, english sentence) -- implicitly including a language model of p(french) and p(english) in addition to allowing translation p(english | french) and p(french | english). A non-generative translation model would, for instance, only model p(french | english).

I don't exactly understand what this judge meant by "generative", it's presumably not the technical term.

currymj··on Ada crate of the year 2024 announced
i was very careful to use the vague phrasing "features dealing with these problems" and not "Ada has a borrow checker and dependent types".
currymj··on Ada crate of the year 2024 announced
1) memory safety at compiler level, 2) type systems that have dependent types or similar such features, are both things people have gotten interested in again recently, especially because of the excitement around Rust. (you can see this connection from the use of the word "crate" here, which is a Rust term originally.)

Ada is a language that had features dealing with these problems way back in the 1980s, so people have suddenly become interested in revisiting it.

currymj··on Science is a strong-link problem
agree strongly. in the long term, for society, it’s probably true that bad science will just be forgotten.

in the medium term, those “weak links”, whether due to mistakes or fraud, can derail an entire area and prevent the strong link papers from being written.

currymj··on Value-Based Deep RL Scales Predictably
RL is unbelievably finicky, sensitive to hyperparameters, and hard to make work. If you are pursuing a research project and decide to use RL, you are making your life a lot more difficult and stressful.

It's exciting to see any progress in making accurate predictions about what settings will work for RL training. I hope that this research direction can be expanded in scope and that ultimately, people who want to do research in RL can become confident in their training recipes.

I am more excited about that, than about the dream of scaling compute per se.

currymj··on Fully autonomous AI agents should not be developed
it was a pretty defensible term at the time the paper came out, in the context of how LLMs were being trained and used.

in this paper, it's clear that the authors don't think modern LLM-based systems are just stochastic parrots.

currymj··on PlayAI's new Dialog model achieves 3:1 preference in human evals
one issue is that having a good generative model is unavoidable as a component for lots of good, useful tasks. like translation, transcription, etc.

but then of course if you have a generative model, you can use it to generate stuff.

currymj··on Pre-Trained Large Language Models Use Fourier Features for Addition (2024)
pre-trained model would mean training a language model to predict text, then starting from there and training it to add numbers.

training from scratch would be initializing a neural network, and training it to add numbers directly.

currymj··on Subway crime plummets as ridership jumps significantly in congestion pricing era
while the NYC subway is indeed quite safe, it's also undeniably true that there are a lot of stinking smells.
currymj··on How Spotify Killed Lo-Fi Hip Hop
the curated playlists used to be incredibly good.

the initial release of Discover Weekly 10 years ago gave everyone such uncanny recommendations that it was a cultural event, people were talking about it, it was written up in major newspapers. Apple Music had just released a few months before and was a serious threat to Spotify, so Discover Weekly changed the trajectory of the company.

It was also technologically interesting because it was one of the first times that a recommender system incorporated deep learning (on the actual song audio) in addition to collaborative filtering.

It was really amazing! Now instead, they are shoveling whatever garbage costs them the least. Depressing.

currymj··on Jujutsu VCS Introduction and Patterns
it’s great that jj just looks like you use git to other people, so doesn’t inconvenience them.

how does it work the other way? if i’m using jj, and my colleagues are using git, how does it cope? in particular what if they are using git in a really depraved way (like just add all files and always commit everything, etc.)?

currymj··on New York claims a small victory in 'forever war on rats'
there is a push to use garbage containers, and the requirements have partially rolled out, but it's a very controversial political issue.
currymj··on Recent results show that LLMs struggle with compositional tasks
so here's an example: I saw the following amusing math joke on BlueSky, posted a couple days ago.

https://bsky.app/profile/elliotlip.bsky.social/post/3lh5bs7g...

"It's wild that I took a whole-ass class all about differentiable functions from R^2 to itself that satisfy one specific PDE."

output from Claude (one-shot, no retries or trickery) when asked to "explain the joke":

"The joke is playing off the fact that "complex analysis" (the study of differentiable functions satisfying the Cauchy-Riemann equations - a type of PDE) sounds like it should be more complex than it is when described technically. The humor comes from reducing an entire mathematical field to "functions that satisfy one specific PDE," making it sound surprisingly narrow or trivial despite its actual depth and importance."

Wikipedia and lots of math textbooks are in the training data, so it's not so impressive that this model will produce correct statements about math.

But the joke itself was not in the training data (to the best of my knowledge, it's this guy's original joke). And complex analysis was not mentioned in the joke. Yet somehow the text generated is correct with respect to both of those things.

I see things like this quite regularly, which under the "stochastic parrots" story, can't happen.

I've tried to phrase all these sentences very carefully to not claim there is any "agent" or "intelligence" behind the Claude product. There are many explanations for how a language model like this could imitate intelligent dialogue in ways that are somewhat fake and don't generalize -- I think this is what's happening. I also see things break down all the time and the sleight-of-hand fall apart. However, it is not "stochastic parrots" any more.

currymj··on Recent results show that LLMs struggle with compositional tasks
completely agree. people say a classic chess engine is "thinking" when it goes deeper into the search tree, but nobody is confused about that. This colloquial language doesn't bother me so much.

But there really are people who think they are talking to something more than that. Like within a conversation with today's consumer product, they sincerely believe that an actual being is instantiated who has goals and intentions and talks to them.

currymj··on Recent results show that LLMs struggle with compositional tasks
it's no longer an accurate description.

with recent models, while stitching together training data is still most of the story, there's obviously something more going on, especially with regards to code, mathematics, and with the ability to successfully produce dialogue that respects pragmatics. this has nothing to do with a real thinking being, but it's become different from a stochastic parrot.

there are deluded people who think that Claude and ChatGPT are really thinking and feeling. it is admirable to try to argue them out of this position.

but if your argument involves saying things that are not true, and anyone can easily see are not true by interacting with a consumer-grade chatbot for 10 minutes, it will just discredit you, and those people will go on believing they're talking to HAL-9000.

currymj··on Auto-Differentiating Any LLM Workflow: A Farewell to Manual Prompting
i am old enough to remember the opposite: people would try to sell deep learning to the mainstream ML community by pointing out that backprop is just message-passing on a Bayesian network with modified sum/product operations.
← PreviousPage 4 of 34Next →