The alien space of science: Coherent but cognitively unavailable directions

Therefore, the further you get away from the topic of ML, the less dense our coverage. Frankly, it's a tough to editorial choice to determine which research directions and therefore which proposals to include. ML is quite multi-disciplinary. And people who have a few papers in ML but many in (e.g.) medicine get upset when their profile is sparse. But we can't possibly produce good proposals in medicine due to its tendency to be closed-access.

One part of the explanation is that we focus primarily on AI research, and a substantial chunk of physicists gravitate (heh) towards AI research, so we end up capturing more of their work.

Another part of the explanation could be that (AI +) physics papers are just easier to access, so we end up processing more of those.

That said - we believe the method does need to have sufficient data to model the conceptual density. So it'll not work in closed-science areas.

One other thing I will share: we have been working on a way to enable verified academics with sufficient topical domain expertise to engage in small group discussions that update these documents. You can think of it like Althea (our agent) being an area chair and moderating a discussion to determine how to update pages. The reason we think this will work is that you can have an anonymous discussion with the AI system (and be blunt), but we can weigh your expertise when updating the documents.

edit: clarification: the content itself is fine, in fact I'm curious!

In other words, I experience high-resolution multi-modal qualia all day and it's really noisy and I think a lot of new ideas and concepts come from obsessively exploring a small subset of those details. Maybe that's just me posthoc rationalizing my phd, though.

We are basically already at the point where if what current frontier models can do isn't truly "novel", then the vast majority of humans have never had a novel idea in their lives. My three years from now guess is that if whatever it is that frontier models can do at that point isn't "novel", then no human ever will have had a novel idea.

So this guy made up an imaginary test, provided a completely unjustified result to it- which casually confirms his ideas- and he's so proud he was right! What a good test he invented!

The real question is whether it would happen in a practical timeframe.

The idea is to allocate academic reviewer time based on the estimated quality of a submitted paper. But critically, you should maintain uncertainty bounds in your estimate. It's kind of like an UCB (upper confidence bound) bandit approach but inspired by cascaded speech translation (where you really should not throw away information at any point in your pipeline).

Recommendation: rig up the mcp and have claude or codex trawl the corpus for the kinds of ideas you might care about.

``` claude mcp add --scope user lacuna-research -- uvx lacuna-research-mcp ```

or

``` codex mcp add lacuna-research -- uvx lacuna-research-mcp ```

The one thing I could recommend however is rigging up the mcp (https://github.com/tiptreesystems/lacuna-research-mcp) to your code agent and trawling the database for ideas worth thinking about.

> I find that often a big cluster of papers around a particular topic, none of which seems to move the needle much, is actually an indication that we need a new approach and writing more and more incremental papers just digs deeper into a local minimum.

Yes, I think this can work. We use a similar idea to categorize what papers have been influential or foundational -- look at the ones that have changed the trajectory (in time) of a research direction significantly.

I feel like when it comes to "Alien Spaces" or new approaches, there's no other way than to babble a lot of stuff that doesn't move the needle much but then leads through the implicit to language and communication mechanisms to a spike out of the local minimum. Didn't the ancient guard have hundreds more notes than ended up in their useful collections?

Nowadays you have so much research that nobody really cares to just ramble with attempts and approaches ... and instead skims the work that others certainly tried already ... distancing themselves from the possibility of new approaches ...

But yes, we might be bad at marketing. Internally the term we use more often is "concept elements".

The intuition is that if you take a complex treatment of a topic (like an academic paper), you should be able to break it down into some more abstract self-standing ideas. Then, critically, these units are composable and easily embeddable (i.e. they don't have proper names and numbers).

Here's a couple examples of these idea atoms pulled from our paper https://lacuna.tiptreesystems.com/work/the-alien-space-of-sc...: "Scientific ideation often conflates the logical coherence of a research direction with its likelihood of being discovered by the current community. This oversight makes it difficult to identify research paths that are scientifically sound yet unlikely to be discovered due to the existing distribution of expertise and collaboration networks."

Another example: "A generative model estimates the plausibility of concept combinations by treating conceptual units as tokens and calculating their length-normalized log-likelihood. By training on random orderings of these units, the model learns to identify which components are likely to co-occur in a valid research contribution regardless of their sequence."

I came up with that name, and fair eyebrow. Think of an "atom" as a unit of a working vocabulary (a "token"), not a Platonic form. It's a way of compressing the corpus (paraphrases of the same idea across papers get merged into one symbol, a bit like a tokenizer's vocabulary).

We also talk about some limitations of this framing in page 11 (towards the top).

It cost ballpark $250k

  > Bio is one of our sparsest regions 

It does seem like one of the best uses of AI -- digging through existing literature to make novel connections/ideas.

Curious if you've heard of Don Swanson's "Public Knowledge Problem" paper from the 1980's, which was roughly this line of thought?

I think if you went back in time you could pay some power loom engineers to build you a pretty decent 4-8 bit microcontroller operating at 1-5 Hz (comparable to power looms at the time) and computer-controlled textiles would have paid for the computer revolution 75-100 years ahead of time.