Ooh, will take a look at this, since it's directly applicable!
875 karma · joined February 20, 2024
Github is https://github.com/smj-edison/
What really caught my attention was this: "So from the start, I took inspiration from C libraries I enjoyed working with and designed the API to have a strong, stable ABI that allows for future additions." What are good examples of this? I feel like I just don't have enough experience with well-designed C libraries to know what to look for.
I've thought a lot about this quote, and the best abstractions I use are absolutely this. I love array indexing as an abstraction, but in the back of my head I have pointer + index * size. I like having the kernel do memory mapping, but I know there's a buddy allocator and TLB cache there.
I don't think LLMs will just magically be able to connect all the dots, because "garbage in, garbage out" still applies. And there's just too many invisible intermediate states, there is literally just not enough information to fully constrain a model. Hence my interest in scaling computational chemistry to get these intermediates :)
1. The first thing you need to do is start with some atomic structure. You can often find pre-existing pieces (through websites like pubchem), but you need to cobble them together. I use chemcraft and gaussview for this, but those are paid options. I've heard good things about the FOSS program Avogadro though.
2. Once you've built the structure of interest, you want to calculate properties of it. Note that we only infrequently simulate the atom's movement, since that's quite expensive. There's a lot of techniques that let us get the most bang for buck, like calculating transition states, or states of lowest energy which we call intermediates, or running something called an IRC, which follows the molecule to its lowest energy point (note we pretty much always think about these things in terms of energy). The traditional programs I use for this are Gaussian and ORCA (ORCA is free but you do have to create an account on their forum). Newer methods include MACE and orbmol, which are pretty much only used for trajectories (e.g. running through time) as far as I know. This newer crop of programs are much more FOSS, but they aren't as rigorous so my lab uses them only as a "rough draft". I pulled in one of my current notebooks if you're curious to try it out: https://github.com/smj-edison/qc-tools/blob/main/examples/ml...
3. Interpreting results/debugging problems. This is currently done in my workflow using a python script to extract values from the job runfiles (oh yeah, did I mention the terrible DX? A lot of this stuff is done by hand-writing a job text file and then using regex to parse the logfile). We do a lot of analysis based on relative energies (this is something called transition state theory of reactions).
I definitely glossed over a lot of details, but this is long enough as is. Definitely let me know if you have questions! If you do want a "hello world" though, I'd definitely check out the notebook I linked to. You'll have to create a file called "start.xyz" with some sort of chemistry program (or use AI to pull one for you), and then you can run the system through time. I would use a system I'm researching right now, but I'm not sure what our lab's policy is on broadcasting things to the world.
1. Biology has a very nasty habit of not cleanly abstracting. It's not like we can treat every protein as a "diffuse mass" without losing a ton of important interactions. This is unlike something like CFD where you can treat a mass of water molecules with certain constants. Think of something like cancer: a small mutation to a DNA strand can cascade into a tumor. You have to deal with the mutation happening at an atomic level while scaling to a whole organism (yes I know there's models that can approximate this, but they can't capture all the interactions).
2. This brings us to modelling at the atomic level. To model a system of atoms with the Schrödinger equation, it naively scales O(6^n) per particle. Note that means every nucleus and electron, so if you have say 10 carbon atoms and 22 hydrogen you're already at 10^127 operations. We can cut corners a bit and end up with something like CCSD which scales O(n^7), or cut more corners and scale O(n^3) for DFT. DFT is pretty much the last method where bond forming and breaking is modelled, so after more corner cutting you lose losing reactions, which are important for understanding why something is happening. Most protein folding simulations (moreso prior to alphafold) run a very crude approximation that models atoms as balls on springs (things like AMBER and CHARMM), which obviously can't handle reactions.
3. If this wasn't bad enough, you also can only move through time at about 1 femtosecond (10^-15), so it takes a ton of simulation steps to get even a nanosecond in. Forget millisecond or second lengths where things start to happen.
So, while the math of the physics exists, computationally it's practically intractible. There's some exciting work with something called Machine Learning Interatomic Potentials (MLIP) which approximate DFT using a neural network which are about 1000x faster than DFT, and iirc scale much more linearly (look at things like MACE and orbmol if you're curious). So we're making progress, but we've gone from so ridiculously impossible to pretty much impossible for large systems.
I have a couple ideas about developing custom microchips to make this faster, which is why I'm pursuing this, but there's still a massive way to go!
The biggest reason I use Zig is it's a very explicit language. The creators made a very intentional decision to avoid too many "high level" designs. This doesn't mean there's no capabilities for abstraction (comptime is great for that), but when you see array indexing, you can think "ptr + index * size with bounds check". There's lots of other things like that where the language does exactly one thing, and that thing is a low level operation.
This is terrible when you want to create high level abstractions that hide details from the programmer, but it's what I need when doing realtime audio synthesis or what I'm doing now which is writing an interpreter. I know exactly what allocates, I know what calls IO and can block (both operations explicitly take in an allocator or IO parameter), no data structures have private fields so I can always poke around at the insides. I know what types of errors a function returns, and creating errors is cheap with Zig's error union design.
So I don't use Zig because I think it's the safer language, I know it has sharp edges because of the number of times I've caused a panic on a poisoned pointer or use-after-free. But because it gives me such a transparent view into what is happening I find it liberating.
In fact I've written a whole interpreter that can recover from OOM by raising a recoverable exception to the user. It's really only because Zig made recovering idiomatic, and I'm not sure I could've done it in another language (maybe Rust but I'd have to rewrite large parts of stdlib to both return an error and take a custom allocator).
This is a really uplifting message. I'm currently in school pursuing a degree in applied math, and I've definitely felt unsettled with the recent advancements with automated proving. So it's nice to know that there's still a place for learning mathematics, and that we can find a new way forward.
Is your goal to reduce global temperatures? If so, it needs to be a sustainable effort. You don't get sustainable effort without economic viability.
If your goal is to dramatically reduce temperatures in the next 5 years, why not dump SiO2 in the atmosphere? That would bring the temperature down pretty fast by reflecting out sunlight in the upper atmosphere.
But if your goal is long-term change, we need an approach that works long-term.
Edit: sorry I forgot to mention your point on coal vs natural gas efficiency, I didn't realize it was not that much gain. I still think it's worth it though, since it's still making progress.
Solar has been making incredible progress, and works in sunny locations, but it's not a general purpose solution. Nuclear is getting there, but it's still too expensive and over regulated. I can't say much about wind as I haven't looked into it enough to say. Both thermal and hydro are location dependent, so good when it works but not a general purpose solution.
So that leaves coal and natural gas. We have a lot of both, but switching over to natural gas to coal by retrofitting existing power generation would make a massive difference imo.
At the end of the day, we want diversity in the power grid, not just one thing for every place, since every place is different/has its own history.
Or were you referring more to all the intermediate allocations that aren't the object heap? V8's zones are interesting in this area, because they're like an arena, except that they're only partially reset when a zone ends, so zones can nest inside each other.
For example, since I allowed for objects to be shared between threads, I decided to use struct of arrays so the reference count, metadata, and value would be stored in separate cache lines. This ended up hurting me because object initialization touched three separate cache lines (obvious in hindsight, but the advice of using SoA failed me here). I also heard that you want to pack your values as tight as possible, so I used a packed string index, but then I ended up with integer division to unpack the string (also a mistake, but again the advice failed me). I used a custom allocator to avoid indirection with lists (list items were allocated directly after the list head), but then I had heap fragmentation and the implementation complexity exploded.
Anyways, I am now happily using two to three levels of indirection in my data structures, large structs, and malloc for individual objects, and it's still been faster in my end to end testing. So maybe this is unique to interpreters, and maybe I could have done it better, but the suggestions don't automatically apply in my experience.
I actually partially disagree with this. What happened to all the excitement about Intelligence Augmentation (IA)? Now it's AI instead of IA. I think there's so much untapped potential for augmenting our intellect with the likes of https://dynamicland.org and https://folk.computer, as well as the work that's been going on in college math education, things like Lean, etc. I think the only reason human math capabilities haven't expanded that much is a failure of our imagination, not our potential.