celltalk

All of these smaller model paradigm suggests that we need to incorporate pruning into model training. Neat was one of my favorite algorithms of all time. Same thing with BitNet models which keep showing the information you need is not that much for neural networks. And again, it is same with us, we use much less energy than a regular network so there seems to be immense waste of energy training these models.

My intiution tells me the pre-training paradigm will shift immensely in near future because we started to understand that we don’t need all these paramaters since the subnetworks seems to be very robust preserving information in high dimensions. We keep saying curse of dimensionality but it is more like the bliss of dimensionality we keep seeing. Network redundancy still seems to be very high given BitNet is more less comparable to other LLMs.

This basically shows over 50% of the neural net is gibberish! The reason being is that the objective function simply does not include it.

Again my intiution tells me that neural scaling laws are incomplete as they are because they lack the efficiency parameter that needs to be taken into account (or simply left out due to greed of corporate).

And this is what we are seeing as “the wall”.

I am no expert in neural network theory nor in math but I would assume the laws should be something in the vicinity of this formulation/simulation:

https://colab.research.google.com/drive/1xkTMU2v1I-EHFAjoS86...

and encapsulate shannon’s channel’s capacity. I call them generalized scaling laws since it includes what it should include in the first place: entropy.

That’s why the survey. It could mean something totally different for you, and if you would like to share it. It is just there.

Isn’t there? I think current meta is promising AGI and waiting to see what happens.

Well, this was a great answer to the survey!

I’ve created an interactive site showcasing the results of an online survey about AI. My hope is that this site could serve as a snapshot of public sentiment about AI, a sort of historical artifact for our time.

So far, only 15 random people have responded, but I thought it might be interesting to hear perspectives from the HN community as well! If you have a moment, I’d love for you to check it out and share your thoughts.

I’m also open to suggestions on how to improve the site or what else I could add to make it more engaging or insightful.

I haven’t read the paper yet but the news article seemed a bit, meeh.

BCL-2 inhibitors, mainly Venetoclax, is used in cancer therapies quite often which also triggers cell apoptosis and it’s very effective. It was also designed to target B-cell related cancers, but it found to be so effective that FDA approved it to be used in primary cases of Acute Myeloid Leukemia. So, killing cancer with triggerring apoptosis is very well known. I think the novel part might be the two protein, so it is probably more targeted for metabolic activities… but yeah didn’t read the paper yet.

Anyways, for the side effects a major one could be Tumor Lysis Syndrome (TLS). Basically, if you apoptose the cancer cells super fast, the molecules from those cells spread everywhere and it becomes toxic for the patient. This is at least the case for Venetoclax.

I think this is the feeling when you realize what you’re doing does not matter. No big purpose, just a mere immortal. I think you need a bigger purpose bro, like really. Reach out.

How much each hourly update costs? OpenAI pricing table says $15.000/M characters.

Also added a UI for ease of use. You can use in Rstudio with viewer.

I just put a monitor and a survey to the site. Down the road I will have a redesign and maybe create a timeline, so I want people’s opinion on the subject, like where we are and where we will be in two years or so.

am thinking about rede

Hi guys,

I have made a package which basically does the same thing that g-1 or multi-1 does, just not in python but in R. It creates chain of thoughts with the open-source model and gives you a final answer.

Best, O

No, maybe it got lost in translation. I meant, I will open up an account for you for free and you test it. If you like it or hate it, you just tell me. That was the whole point.

Hi everyone,

I am designing a side project to become a finance bro lol. Right now I am in super early phases of the project but I need quality feedback from real users, in this case traders. If you’re interested you could write your email or maybe drop me a message on X (@eonurkara).

Thanks!!

Let's see if he is going to make it or not haha. I bet it will make it to 1M.

Yes, there is a bug somewhere which I could not find so far… it is making me crazy. Yet, I will find it at some point. Other than that, I am still improving the site!

I am trying to get better at trading so I thought it would be nice chance to work on my coding skills. I wondered if financial indicators actually makes money for a given period.

https://indicatorinsights.co

Made it with React, Vite, firebase via Typescript. It’s a really different world for sure.

No comments... That bad, huh? :D

Hahaha this must be wake-up call for me, but I will just close my eyes. I am almost doing the same thing at: https://indicatorinsights.co

It's very quick and dirty, not finished. Still, I am having fun (showcased in about page). Hope not to lose it.

My gut says protein-protein interactions aren’t that useful given how these interactions scale, and on top of that you have post-translational modifications, SNVs etc. It’s a very very hard problem to solve.

Instead, we can focus on gene-gene interactions and go bottom up. There, we don’t need new wetlab techniques that needs to be validated, measure mRNA instead. Plus, an average single cell contains 40 million proteins, yet the number of mRNA molecules are orders of magnitudes less and can be sequenced with high precision.

If you for instance open KEGG database in graph mode, you will see one of the largest manually curated datasets ever. Yet it is still tiny! If you imagine A,T,G,C as alphabet, and genes as special tokens. All we know as humanity is couple of words… and it’s sad.

I think LLMs might be our best bet on these. Given few words they might uncover “new words” we have never thought about. I kinda tried this… but the methodology is still shaky.

https://celvox.co/blog/TCC/index.html

Do cells have a language? With the recent success of large language models and the vast number of curated gene pathways, we visited this fundamental question one more time. We trained a decoder-only transformer model with 120M parameters on >250,000 human pathways, and then asked the model to complete 1000 unseen pathways with 80% masking in zero-shot setting. Remarkably, this simple ready-to-use model was able to complete more than half of the pathways (660/1000) with significant overlap. Our results suggest that even relatively small transformers can capture underlying connection among genes and understand the true nature of cell language.

You’re right in most of your comments however a fine-tuned model will be able to pick up nuances which are missed by a general model. For instance, AVP is used for HSCs, yet not related to flow cytometry at all. If you fine-tune an LLM with experts by your side one time it will be able to give you the grained cell types such as T-cell subsets. Plus, a regression model won’t give you the reasoning behind the given cell type annotation.

Yes, it basically automates the cell type annotation process, plus gives a reasoning for the label.

It still not 100% accurate but it should be useful for baseline annotations.

It should be okay if you edit the base prompt properly.

I don’t have any benchmarking yet, but any help is appreciated. We do have fine-tuned model for anyone interested.

Thank you. Hope it is useful!

No, but the help is appreciated!

If you set the seed and temperature to 0, it is. I did not have any intentions to publish it, but I might think about a 2-pager Bioinformatics paper if I have time.

You all run on 2-bit (DNA) and you all seem pretty stable to me. You run your brain with couple of watts per day and add big numbers without much hassle.

Time for llm videos!