jkhdigital

I started a PhD in 2020 and I know exactly why you created this app because I tried like half a dozen different tools that didn’t fit. I needed a workflow to

1. collect and prioritize relevant research papers

2. make notes and synthesize ideas across my reading

3. use the notes to assemble draft of original writing

4. seamlessly move my own writings into LaTeX documents along with citation details

and ended up in Obsidian where I basically had to build my own tool anyway. Which I never did, because I just wanted to focus on research without fooling around with tools.

Yes. Not all business models utilize public advertising. For example, MLMs like Amway and LuLaRoe.

Advertising didn’t exist for most of history because mass media didn’t exist. Advertising is part and parcel of mass media.

I hate to paint with such a broad brush, but I’d venture that “creatives” are not primarily motivated by profit. It is almost a truism that money corrupts the creative endeavour.

That most likely explains the slow creep of grade inflation, remedial courses, etc. which has been going on for decades. This article touches on that but mostly describes an entirely different phenomenon.

Exact same experience—first pair from 2008 lasted over three years of heavy use (running on mostly pavement) and each replacement has barely survived one or two years.

Hah! I had a more or less identical experience after getting my first pair in 2008. Went out to the track, jogged a single lap, and that was enough. Calves were absolutely wrecked the next day. Four years later I ran my first marathon (3:54) in them.

17 years later they are still the only shoes I ever wear for running or working out. And I have bigger calves than anyone in my local CrossFit gym

I think your analogy about logic gates vs. CPUs is spot on. Another apt analogy would be missing the forest for the trees—the model may in fact be generating a complete forest, but its output (natural language) is inherently serial so it can only plant one tree at a time. The sequence of distributions that is the proximate driver of token selection is just the final distillation step.

The author mentions being deeply inspired and influenced by Jose Valim; I guess this means (approximately) that KC3 is to C as Elixir is to Erlang?

Why LLM? Isn’t this what AlphaZero is good at? There are many more kinds of useful ML models than LLMs!

This is exactly what I’ve been thinking as I see LLMs being applied to all these complex problem domains. Humans did not conquer the world because our intelligence can solve every problem, we did it by using our intelligence to (1) break down complex problems into small, manageable pieces and (2) designing tools and machines that were exceptionally good at efficiently solving those subproblems.

The other recent example that comes to mind is the paper that explored the reasoning process used by LLMs to answer trivia questions like “Name a national capital whose letters can be rearranged to spell a common greeting in the language of a neighboring country.” (answer is Hanoi by the way)

The LLM responses show that they intuitively grasp the algorithm for answering such a question, but then they basically run the algorithm in their own thoughts (self-talk) which is horrendously inefficient.

Put differently, natural language reasoning is brilliant at turning the messiness of the real world into well-defined abstractions, but as soon as that is done it needs to hand off the task to a machine. For “solved” problems this might be a formally specified machine, but it could also be another class of model such as AlphaZero (along with a proper specification of the problem the “subcontractor” is to handle).

Why would you ever want an LLM that is a perfect calculator? Humans invented calculators for a reason. A good LLM should respond to arithmetic questions by executing a cheap and efficient calculator program instead of wasting cycles on it.

I had the same thought. It reminds me of solving Project Euler problems, where there is often an obvious naive approach which is guaranteed to produce the correct answer but would consume prohibitive memory/compute resources to execute to completion. I suspect the models would perform much better if prompted to formulate a strategy for efficiently solving these challenges rather than solving them directly… which indicates a direction for potential improvement I suppose.

All major technological advancements have eliminated some jobs and created others. In general, producing more with less human input is how society becomes richer. Is it a tragedy that we no longer need telephone switchboard operators, ice cutters, and lamplighters? There were surely countless mini-tragedies as the people who made their livelihoods that way struggled with stagnant wages and unemployment, but the next generation became free to pursue other more productive forms of work and accumulated greater wealth as a result.

Here, let me drop this mic for you

The lab wasn’t just studying coronaviruses. The director had intimate knowledge of gain-of-function techniques, with publications and grant proposals to document this. Some of the research was published during her tenure at the lab, so it can be assumed that the research was performed there.

You're not wrong, but natural language has a lot of stylistic "noise" which can be utilized as a subliminal channel without noticeably degrading the semantic signal.

Why do you need to rerun the LLM? Watermark detection only requires the hash functions (equation (1) from the paper).

This isn't necessarily true, it just depends on the implementation. I can say that because I've published research which embeds steganographic text into the output of GPT-2 and we had to deal with this. Running everything locally was usually fine--the model was deterministic as long as you had the same initial conditions. The problems occurred when trying to run the model on different hardware.

No you've got it right. Watermarks like this are trivial to defeat, which means they are only effective against lazy users like cheating college students and job applicants.

I spent the last five years doing PhD research into steganography, with a particular focus on how to embed messages into LLM outputs. Watermarking is basically one-bit steganography.

The first serious investigations into "secure" steganography were about 30 years ago and it was clearly a dead end even back then. Sure, watermarking might be effective against lazy adversaries--college students, job applicants, etc.--but can be trivially defeated otherwise.

All this time I'd been lamenting my research area as unpopular and boring when I should've been submitting to Nature!

I really don’t know why I stumbled into the comments section on this particular article, but while I’m here I have to commend you on writing perhaps the most thoughtful and eloquent comment I have ever read on HN.

There are a lot of nuances to these situations of third-party involvement and the ruling discusses these at length. If you’re interested in the precise limits of the 4th amendment you should really just read the linked document.

I’m not sure if I am mathematically sophisticated enough to follow along but I’ll try. This chain of thought reminds me of the present state of cryptography, which is built on unproven assumptions about the computational hardness of certain problems. Meanwhile Satoshi Nakamoto hacks together some of these cryptographic components into a novel system for decentralized payments with a few hand-wavy arguments about why it will work and it grows into a $1+ trillion asset class.

About four years ago my parents got a cabin up in the mountains right along the NC-TN border, and we’ve been up there almost every summer since. Just last month we went tubing on the North Toe, so I know exactly where that CSX rail line is. Just wish I had the free time to go and help volunteer!

…did you read the link you just posted? If the person whose words are being presented as evidence is available for cross-examination then (legally) it’s not hearsay. The defendant in a criminal case is always available, so any statement they make out of court is never hearsay.

I don’t think anyone is actually trying to connect with distant cousins for social interaction; rather the shared DNA segments provide hard evidence to confirm distant ancestors. I got into genealogy a couple years ago after thumbing through some old family documents and I’ve been able to confirm 6th cousins (traced via our shared ancestors) using 23andMe data.

Now there are probably some who think genealogy is a silly pastime and maybe it is, but researching my family history helps me feel grounded and more connected to humanity so I derive a lot of benefit from it.

Location: Tokyo, Japan

Remote: Yes, please

Willing to relocate: No

Technologies: Elixir, JS, Lua

Résumé/CV: www.linkedin.com/in/jimhowes

Email: jotakami (at) gmail (dot) com

Midcareer transition after military and consulting experience. MBA + MSCS and about 60% of the way to a PhD in cryptography with published research on private/covert communication. Been writing code since I was a teenager but never professionally.

Because if it were any other way then you wouldn’t exist to sit there and ponder the question. That’s the unsatisfying answer.

I think it makes sense to draw an analogy to evolution—stable arrangements of elementary particles that (somehow) reinforce similar arrangements around them will come to dominate the observable universe.

Profit-maximizing business is the engine of economic growth. Looking over the past couple hundred years, I’d say the outcomes have been nothing short of spectacular.

Sounds like he had a schizophrenic psychotic break. His behavior and especially his conspiratorial comments are classic schizophrenia. (Source: My sister and late aunt are schizophrenic)