1,061 karma · joined April 11, 2019
Building tools that enable communities to share effectively seems like another additional challenge, and the fact that virtual spaces and digital spaces are dismissed seems like it might prove a major roadblock to connecting and sharing in a larger inclusive community.
Given the interest in leveraging this for doing science it also seems that this is at risk for empowering individual labs while leaving all interfaces to the rest of the larger scientific community dependent on the current utterly broken system of publication.
I can imagine taking the citation tree and using the LLM to compact the hypotheses, results, etc. for each node in the tree and sticking that in the vector database could get you pretty far.
For the literature review piece the key problem is that LLMs are exquisitely bad at working with even the simplest kind of scientific evidence: citations [1, 2]. They will get better, but it is not clear that LLMs can deal effectively with the very sparse kind of evidence that appears in the literature. Also, generating hypotheses isn't exactly the rate limiting step, the bigger issue tends to be when you get people with pet projects/hypotheses in positions of power that dictate funding priorities (e.g. the decades long Alzheimer's Aβ disaster).
For automation and instrumentation of labs the vision is on point and there is interest, and active work, if not large amounts of funding, to bring that vision to reality [3, 4, 5]. However, we simply don't have the tooling needed to be able to express the full complexity of experimental protocols in a way that can be verified. Sure you can write a python script to control a robot, but it is exceptionally difficult to extract the scientific meaning from that.
My PhD work was to develop a formal language for scientific protocols, and I'll be continuing to develop it, but there is still a long way to go.
1. https://doi.org/10.7759/cureus.39238 2. https://doi.org/10.1016/j.mcpdig.2023.05.004 3. https://www.youtube.com/watch?v=_gXiVOmaVSo&t=865s 4. https://doi.org/10.1109/JIOT.2020.2995323 5. https://ccc.ucsf.edu/sites/ccc.ucsf.edu/files/Marshall_W_CCC...
I just reran one of my usual benchmarks and I see 2mins for pypy3 (pypy 7.3.12 python 3.10.12) peak memory usage about 8gigs, 4.8mins for python3.11 (3.11.4) peak memory usage about 3.6gigs (2.4x speedup). On another computer running the exact same workload I see 6.3mins and 19mins (3x speedup) with the same peak memory usage.
I don't have any numbers on the dataset pipelines because I never ran them in production on cpython and went straight to pypy3. It is easy for me to switch between the two implementations in this context so I could run a side by side comparison (with the usual caveat that it would be completely non-rigorous).
I also have some internal notes related to a project that I didn't list because it isn't public, isn't in production, and the benchmarks were collected quite a while ago, but I see a 4x increase in throughput when pulling large amounts of data from a postgresql database from 20mbps on cpython 3.6 to 80mbps on pypy3.
We also use pypy3 to accelerate rdflib parsing and serialization of various RDF formats. See for example [3].
Thanks to you and the whole PyPy team!
1. https://github.com/tgbugs/dockerfiles/blob/6f4ad5d873b7ab267...
2. https://github.com/tgbugs/dockerfiles/blob/6f4ad5d873b7ab267...
3. https://github.com/SciCrunch/sparc-curation/blob/0fdf393e26f...
https://www.timmons.dev/posts/static-executables-with-sbcl.h... https://www.timmons.dev/posts/static-executables-with-sbcl-v...
Some questions and thoughts.
It seems that the result could vary based on how you construct the similarity matrix Z, e.g. is it purely taxonomic? or does it try to account for the ecological roles that a species is playing in the community, etc.
A seeming limitation is that the optimization works only for a fixed set of n species. While it is useful for managing existing communities, it means that there is still a question of whether larger n is strictly better, and leaves open questions of how to deal with transient or migratory members (if the community is spatially bound).
The answer I think, is that it depends on how the similarity matrix is constructed. If every species is fully dissimilar then increasing n is always a good thing. If you use niche space to construct it and new species do not some add or enter new niches so they overlap with others, then they will be close to another species in the matrix and increasing n will not have much impact. On the other hand if you use a purely taxonomic approach then you wind up balancing the number of birds and mammals regardless of niche.
It is not clear to me whether it is possible to construct a similarity matrix that can account for the interaction between n, the carrying capacity of the ecosystem, and the number of available niches (or the ability of species to create new niches). By analogy if you have a stream (sunlight) powering water wheels, how many wheels and how many levels of gears (layers in the ecosystem) can be added, created, and/or sustained? At what point does adding an additional species mean that either two species are forced to be close together in the similarity matrix or both their populations must shrink in size because they must compete for the same energy sources?
Does the model sometimes produce impractical results, e.g. that it is good to have a single member of a sexually reproducing species (this is probably an orthogonal concern and you would want to scale to real population sizes such that the minimum corresponded to the smallest viable a self sustaining population)?
Is there evidence that maximizing diversity using this measure actually produces more robust and stable ecologies?
Obviously you wind up with resolver resolver resolver services if this goes on for too long, but it is one of the few workarounds for the fact that time (and many other things) are not reified in the DNS system.
From my own experience, it is absolutely possible to get good interaction with low latency in Racket, but of course as with any language you have to have enough experience to know how to avoid performance pitfalls.
For some of the gui work I've done I needed to mix threads and semaphores to get things to be really responsive when there could be a long running process triggered by a UI interaction, but in Racket it has been vastly easier and safer than in other languages.
With regard to prototyping GUI's I'd suggest taking a look at https://github.com/mfelleisen/7GUI. https://github.com/Bogdanp/racket-gui-easy could also be a good place to start.
With regard to Racket more generally, I'm probably not the best person to ask since I had a very high friction start where I just banged my head against the wall until things made sense.
Lately I've been using Racket to prototype the interaction design for a game, and I had written some placeholder text. I had put fake interpolation in for {player-name} and things like that, and then I realized that #lang at-exp would let me just ... do that, as described in the original article.
I had to laugh, because I maintain a bunch of Python code, and if Python has batteries included, Racket comes with a retrofittable fusion battery pack.
One other little anecdote, is that I've been porting an old plt-scheme codebase to Racket CS. In the second phase I have started replacing an ancient C++ rendering layer with the racket gui canvas functionality. During the process I moved hundreds of runtime type checks in the C/C++ layer to the Racket contract system. It was as if the Racket team had somehow secretly obtained access to the codebase I was working on and had designed the perfect abstractions to solve the problem.
The list of these kinds of edge cases is long so it probably winds up being easier to try to just run a library to see if it works. I doubt anyone has done a systematic survey.