antichronology

This is really interesting - I'm going to be honest I'm not an immunologist so this is my (LLM assisted) understanding of your comment:

The immune system recognizes a sugar as a PAMP, or Pathogen-Associated Molecular Pattern, which is a signature of a potential microbial threat.

This initiates pyroptosis an inflammatory form of programmed cell death causing the cell to burst. This rupture releases tumor antigens and DAMPs (Damage-Associated Molecular Patterns), which are "danger signals" from the dying cell

The release of DAMPs shifts the Tumor Microenvironment (TME) from an immunologically "cold" to a "hot" state, promoting a potent Type I Interferon (IFN-I) response.

The release of DAMPs shifts the Tumor Microenvironment (TME) from an immunologically "cold" to a "hot" state, promoting a potent Type I Interferon (IFN-I) response.

This response recruits Antigen Presenting Cells (APCs), which engulf the newly released tumor antigens.

---

mRNA vaccines are somewhat of a parallel approach where the antigen selection and delivery happens manually. An mRNA vaccine delivers the encoding sequence for specific tumor antigens to drive production and presentation, training the immune system. One of the big challenges of this space is optimal antigen selection from the patient's tumor.

One thing I'm not fully clear on is why only tumor cell react to PAMP instead of healthy cells. Could be a promising approach but molecular biology is pretty tricky and the devil is always in the details.

> finding sequences on NIH

Almost! Yes most of the data is on NIH sub-institutes. For us we take most of the data from NCBI and intelligently pair it together. The training objective of our model takes pairs of sequences (thus the Joint Embedding Architecture) and trains the model to recognize that they are semantically similar but differ in appearance. This is conceptually similar to a lot of the contrastive learning literature from computer vision.

Sounds like a fun side project :)

There are some great tools out there for putting together plasmids for gene therapies where you can plug in different "elements". Promoters UTRs payloads - check out SnapGene I believe they have a free version.

I personally am hopeful that the political headwinds will blow over. When it comes to cancer vaccines it's one of the most exciting new modalities for treating cancer.

1 in 2 Americans are going to get cancer in their lifetime so no matter political affiliation, the need for health will ultimately drive people to invest in the modality.

That would be really cool. Navigating SRA and mining out reasonable $ relevant tasks is a huge bottleneck.

I find it takes a large amount of effort to parse what the authors are doing, whether the data is high quality, and how to pre-process it in a way that makes sense for the task at hand.

Would love to chat more about how you're thinking of evaluating quality of these agents.

Here is the link for benchmarks and their utility: https://youtu.be/JdT78t1Offo?t=1444

"We have internal benchmarks. Yeah. But we don't we don't publish them."

"we have internal benchmarks that the team focuses on and improving and then we also have a bunch of tasks like I think that accelerating our own engineers is like a top top priority for us"

The equivalent for us would be to ultimate looking to improve experimental results. Benchmarks are a good intermediate point but not the ultimate goal

Thanks for engaging. Do you mind elaborating on your stance?

From where we sit - there are people with diseases and mRNA is an effective way to revert them to a healthy state.

I'd be interested to hear more where you're coming from

I watched an interview with one of the co-founders of Anthropic where his point is that although benchmarks saturate they're still an important signal for model development.

We think the situation is similar here - one the challenges is aligning the benchmark with the function of the models. Genomic benchmarks for gLMs and RNA foundation models have been very resistant to staturation.

I think in NLP the problem is that they are victims of their own success where the models can be overfit to particular benchmarks really fast.

In genomics we're a bit behind. A good paper on this is DartEval where they provide levels of complexity https://arxiv.org/abs/2412.05430

in RNA the models work much better than DNA prediction but it's key to have benchmarks to measure progress.

Huge fan of the work! I'm a big fan of papers from Seelig lab :)

We haven't looked into this deeply yet sounds interesting. Do you have any resources where to start looking at this? Feel free to reach out to us

founders@blank.bio

We think that Orthrus can be applied in a bunch of ways to non-coding and coding RNA sequences but it's definitely fair we're a bit more focused on RNA sequences currently instead of non-coding parts of the genome like promoters and intergenic sequences.

For the data - Orthrus is trained on non experimentally collected data so our pre-training dataset is large by biological standards. It adds up to about 45 million unique sequences and assuming 1k tokens per sequence it's about 50b tokens.

We're thinking about this as large pre-training run on a bunch of annotation data from Refseq and Gencode in conjunction with more specialized Orthology datasets that are pooling data across 100s of species.

Then for specific applications we are fine tuning or doing linear probing for experimental prediction. For example we can predict half life using publicly available data collected by the awesome paper from: https://genomebiology.biomedcentral.com/articles/10.1186/s13...

Or translation efficency: https://pubmed.ncbi.nlm.nih.gov/39149337/

Eventually as we ramp up out wet lab data generation we're thinking about what does post-training look like? There is an RL analog here that we can use on these generalizable embeddings to demonstrate "high quality samples".

There are some early attempts at post-training in bio and I think it's a really exciting direction

This is a really interesting direction. There is this big field of Cell Free (cfRNA) cancer detection. We talked to a few people in the field and think that embedding sequences for this direction could be really valuable. One challenge here is that it's hard to set up evaluation tasks since the public data is scarce

> ground truth

Hey yes, the ground truth for our evaluations is measured experimental data. Our models are benchmarked using mRNABench, which aggregates results from high-throughput wet lab experiments.

Our goal, however, is to move beyond predicting existing experimental outcomes. We intend to design novel sequences and validate their function in our own lab. At that stage, the functional success of the RNA we design will become the ground truth.

> peer reviewed?

Both mRNA bench and Orthrus are in submission (at a big ML conference and a big name journal) - unfortunately the academic systems move slow but we're working on getting them out there.

> synthetic mRNA sequences

I think you're asking on generalizing out of distribution to unnatural sequences. There are two ways that we do this: (1) There are these screens called Massively Parallel Reporter Assays (MPRAs) and we eval for example on https://pubmed.ncbi.nlm.nih.gov/31267113/

Here all the sequences are synthetic and randomly designed and we do observe generalization. Ultimately it depends on the problem that we're tackling: some tasks like gene therapy design require endogenous sequences.

(2) The other angle is variant effect prediction (VEP). It can be thought of as a counterfactual prediction problem where you ask the model whether a small change in the input predicts a large change in the output. This is a good example of the study (https://www.biorxiv.org/content/10.1101/2025.02.11.637758v2)

> experimental verification of their predicted properties

all our model evaluations are predictions of experimental results! The datasets we use are collections of wet lab measurements, so the model is constantly benchmarked against ground-truth biology.

The evaluation method involves fitting a linear probe on the model's learned embeddings to predict the experimental signal. This directly tests whether the model's learned representation of an RNA sequence contains a linear combination of features that can predict its measured biological properties.

Thanks for the feedback I understand the caution around pre-prints. We believe a self-supervised learning approach is well-suited for this problem because it allows the model to first learn patterns from millions of unlabeled sequences before being fine-tuned on specific, and often smaller, experimental datasets.

There is a number of different technologies. Some of the big ones are:

- mRNA therapies: These therapies deliver a synthetically created messenger RNA (mRNA) molecule, typically protected within a lipid nanoparticle (LNP), to a patient's cells. The cell's own machinery then uses this mRNA as a temporary blueprint to produce a specific protein.

The big example here is CAR-T therapy from Capstan which just got acquired for 2.1B. Their asset,CPTX2309 , is currently in Phase 1. Previously to do Car-T therapy you had to extract a patient's T-cells and genetically engineer them in a special facility. Now the mRNA gets delivered directly to the patient's t cells which significantly lowers the cost and technical hurdles.

- RNA interferences (RNAi): Used for gene expression knockdown through natural cellular mechanisms for viral detection. The big example here is Alnylam with 5 approved therapies and a number in clinical trials.

- Antisense Oligonucleotides (ASOs): Short single stranded RNA molecules that get delivered directly to the cell and target an existing mRNA. The big win here is Spinraza which is the first approved treatment for Spinal Muscular Atrophy (SMA) which previously didn't have a treatment. The Spinraza clinical trial (ENDEAR) was so effective that they deemed it unethical to continue it because the control arm wasn't receiving the treatment. Prior to Spinraza most patients would pass away prior to two years of age.

I think the apt comparison here is landing on the moon. Although overtly it carries little value, the technologies developed along the way are very valuable.

As the article mentions improved fertilization, more accurate genome editing, and advances in in-vitro gastrulation are the real prize.

To me there is also something profoundly sad about the loss of molecular information with each extinct species. Having a simple hope, perhaps naive, to reverse this process is comforting.

There is a great novel on a related topic: Permutation city by Greg Egan.

The concept is similar where the protagonist loads his consciousness to the digital world. There are a lot of interesting directions explored there with time asynchronicity, the conflict between real world and the digital identities, and the basis of fundamental reality. Highly recommend!

Science Twitter is a really powerful tool for learning about new papers and being able to discuss with authors directly

Where is the proof that human "rational" thinking is anything but curve fitting? I'm not convinced

One of the top scientists of 2018 compiled by Nature was Jess Wade[0]. Her achievement is contribution to wikipedia specifically highlighting contributions of women and POC. When 90% of Wikipedia editors are men their biases must reflect in the content of the website. Although this isn't necessarily insane, this goes along with the idea that most of the content that we derive from the internet comes with an implicit bias of the poster. Since we don't have an intuitive sense of who is posting what on the internet it's a lot more difficult to perceive, and identify those biases.

https://www.nature.com/immersive/d41586-018-07683-5/index.ht...

google maps has a lot of ratings and I find is good at suggesting a list of places from which I can select one. Look into the explore around me feature

Its possible that its better to leave the option more inaccessible to further incentivize yourself from reverting back to color mode. Maybe those few extra steps of converting back to color will stop you from doing so

Has anyone tried changing their computer display to grayscale as well? I tried it for a bit but found that code syntax lacks color. Also graphics in papers all of a sudden are infinitely less informative (elucidating blue from red is really hard in grayscale)

I'm wondering if personally keeping track of it somehow skews the statistics. Maybe the change is significant but overall not that big something like ~10%

I would be interested in cranking through the data myself to see how my phone statistics change vs others. If there was a way to time activities spent in apps...

Probably similar to how a lot of Russian people are unwaveringly support Putin's continuous rule

provide an economic argument for not implementing additional authoritarian policies. Seems that these decisions are incredibly hard to measure. For example how much did Turkey's economy suffer from the previous failed coup?

I wonder if it will have some sort of long term impact on the economy. Is there a way of measuring relative government authoritarianism on economic development of a nation?