639 karma · joined January 3, 2020
I regularly use Python (numpy+, pytorch), C, some parts of C++, and now R. I would like to explore J further.
username at gmail dot com. I have the same username on Github.
Python 3.11.0rc1, Django 4.1.7. I think ingenieroariel also has a binary available on the website.
It's a proof-of-concept (because I wrote it just a few hours back), but it shows that we could have the latest version of Python+Django packaged in a single-file executable. Someone could definitely improve upon the build process (like it was done for Python3.6) and add more packages if they wanted.
Making Django work on Py3.11 involved the following steps (see commit history for https://github.com/ahgamut/cpython/tree/cosmo_djngo):
- add django + deps to the Lib/ folder - add sqlite3 and zlib to the build system - run quick tests and small examples to check the build works
I expect there's a bunch of packages like django which can be ported similarly. If anyone's interested, this is a fun area to explore.
(note it's an experimental port, I just put it together).
However, the libc has grown a lot over the last year (Cosmo has a pthreads API now!) -- the updated libc enabled a quick experimental port of Python 3.11.0rc1 back in August (https://github.com/ahgamut/cpython/tree/cosmo_py311), so I would expect that a newer version of PHP can be ported similarly. Let's see....
First let's look at "toolable":
My big issue with Modern C++ these days is not just that it's complex to write, but that it also takes a long time to compile, what with all the template/compiler magic going on in the background. Long compile times break concentration, which I don't like. C++2 is not going to improve compilation times, because it's going to be compiled to C++, and then undergo another compilation step. If I combine C++'s slow compile times, two compilers, and all the dumb bugs that I can create and have to debug+fix, I get ....... (pls wait compiling) ..........
Let's switch context and look at "simpler":
Yay functions look more like JS lambdas or whatever. Fine. Let's look for the pointer-y stuff. Like [1] -- in C or "normal" aka "pre-modern" "C++1" you'd have:
int x = 42;
int *p;
p = &x;
func(*p);
x: int = 42;
p: *int;
p = x&;
func(p*);
What do we have left ... "safer"? sighs
Okay, C is unsafe. Yes, C++ has to carry that baggage. Yes, memory corruption is horrible. Yes, buffer overflow errors lost lots of money, and have caused lots of wailing and gnashing of teeth. Yes, Modern C++ alleviates some of that unsafety with templates and a smart compiler. Yes, Rust has a smarter compiler with better static analysis that helps write safer code. But all this statically analyzed safe code still has to provide C wrappers if it's a shared object, or use the C wrappers provided by whichever kernel it's running on. So somewhere underneath all this safety lurk the familiar monsters, and someone's going to reach for them in the name of performance (remember Actix?). What about all the other issues that you can have unrelated to buffer overflows (log4j)? Will C++2 protect me from my own foolishness? Perhaps that's why its so complicated -- all my C++2 code is safe, because I will write no C++2 code.
And another thing: why not just do the Raku or Kotlin thing and call it by a different name, and have it be a separate language compiling to C++? Why "C++2"? Calling it C++2 vs C++ makes me think of Python 3 vs Python 2, and oh man that is not a good connection to hint at when claiming full backwards compatibility. Python 3 also got quite complicated compared to Python 2, so that doesn't give me hope either.
Now that I think about it, shouldn't this be called C+=2?
TL;DR:
- C++2 is not going to be more toolable because ultimately you're still compiling template-heavy C++
- C++2 is not going to be more simpler because unusual pointer syntax, confusion when mixing C++1, template magicks still have to exist
- C++2 is not safer because you can/have to still write or interface with C/C++ and the footguns therein.
- C++2 should not be called C++2, pls let's find a Raku/Kotlin.
Let C++ be the complexity monster it is. That's the tradeoff we make when we have to write it. If safety is the issue, can't we just use a sandbox or something? Is C++2 really the better option?
[1]: https://github.com/hsutter/cppfront/blob/main/regression-tes...
TLS will probably happen first -- Cosmopolitan already has mbedTLS for redbean, so a modified `_ssl.c` should get us most of the way there.
Recent Python versions ... what version is Python on right now? 3.10? (3.11 apparently) I picked 3.6 at the time because Cosmopolitan Libc did not have threads then (it does now, pthreads API is being filled as of this writing), and also because it wouldn't have too much churn. Perhaps after 3.11 gets stable I'll port it to use Cosmopolitan Libc.
Like after I wrote this blog post, I wrote a small Python app using Flask/SQLite/D3.js, put everything in a single file, just copied it onto computers with Debian/Fedora/FreeBSD/Win10, and now I just access the info I need on those systems via the browser. Providing a browser GUI makes it easier for my colleagues as well, some of whom don't use terminals at all :)
For C extensions (where I've stubbed my toes quite a bit with PyInstaller), I think a nice goal with APE Python would be to have a website like Christoph Gohlke's (https://www.lfd.uci.edu/~gohlke/pythonlibs/), where you just select the packages you would like, and you download them compiled as a single-file executable you can use anywhere. This is an interesting problem, and there's been some work towards solving it, let's see if we can have an elegant solution.
Toys are still fun to play with :)
The flask demo is unnaturally slow because my old computer is slow (running python tests in the background). plus I was using `_dummy_thread` Python3.6's pure-python threads implementation.
Edit: thank you @dang! That was super fast!
Agreed.
> Why do you think Python isn't good for comparing program structure?
From recent experience I prefer control flow analysis, or something that results in a graph structure. As you said, that's harder with dynamic languages. I also think some Python features (english-like syntax, f-strings, division converts int to float, whitespace indentation, loops vs generator expressions) make structural comparisons messy, but that may just be bias.
The ideal language would be one with minimal syntax, where we can target a decent range of programs, and obtain as much info as possible about program structure without actually running the program. I've come across LISP-without-macros in Dreamcoder (https://arxiv.org/abs/2006.08381), BF++ (https://arxiv.org/abs/2101.09571), and a couple of others which I can't remember right now. I think the APL family (APL/J/K) would interesting because fewer characters to generate, but each character has a lot of meaning.
Right now I'm looking at flow-based programming (FBP) for this: In FBP the control flow is explicit - the program code describes a directed acyclic graph (DAG), so comparing program structure becomes straightforward (subgraph isomorphism with some heuristics). I'm writing a toy FBP language that draws images (https://github.com/mayahq/flatland), with which I aim to test what these models understand.
My point is, I would like to verify this claim with different metrics, because we probably have different interpretations of the word "meaningful".
AlphaCode measures similarity between programs via longest common substrings. That's better than nothing, but that would mean two programs that differ only in variable naming would not be considered similar. If two programs differed only in the names of the variables/functions, I would consider that copying.
I think there are better comparisons of structural similarity: compare the ASTs, or the bytecode/assembly code generated, the control flow graphs, or perform SSA and compare the blocks generated. Each of these might have weaknesses as well, but they won't be as obvious as variable renaming, and so we'd get a better idea of what AlphaCode is copying, and therefore a better idea of its full capabilities.
I expect AlphaCode performs well on Python because the training data is dominated by Python, but Python isn't ideal for comparing program structure. I wonder which programming language (given enough training data) would be best suited for language model generation and analysis.
Exactly! It's great that Copilot can generate correct code for a given question, but we cannot gauge its full capability unless we try it on a range of different questions, especially ones that are not found in the training data.
I mentioned this in the other AlphaCode post: It would be nice to know how "unusual" a given question is. Maybe an exact replica exists in the training data, or a solution exists but in a different programming language, or a solution can be constructed by combining two samples from the training data.
Quantifying the "unusual-ness" of a question will make it easier to gauge the capability of models like AlphaCode. I wrote a simple metric that uses nearest neighbors (https://arxiv.org/abs/2109.12075). There are also other tools to do this: conformal predictors, which are used in classification methods, and the RETRO transformer (https://arxiv.org/pdf/2112.04426.pdf) has a calculation for the effect of "dataset leakage".
1) It's an imprecise target: believers can always hype and skeptics can always downplay improvements. Humans can do lots of different things somewhat well at the same time, so a machine beating human-level performance in one field (like identifying digits) says little about other fields (like identifying code vulnerabilities).
2) ELO ratings, or similar metrics are measurements of skill, and can be brute-forced to some extent, equivalent to grinding up levels in a video game. Brute-forcing a solution is "bad", but how do we know a new method is "better/more elegant/more efficient"? For algorithms we have Big-O notation, so we know (brute force < bubble sort < quick sort), perhaps there is an analogue for machine learning.
I would like performance comparisons that focus on quantities unique to machines. I don't compare the addition of computer processors with reference to human addition, so why not treat machine intelligence similarly?
There are many interesting quantities with which we can compare ML models. Energy usage is a popular metric, but we can also compare the structure of the network, the code used, the hardware, the amount of training data, the amount of training time, and the similarity between training and test data. I think a combination of these would be useful to look at every time a new model arrives.
Zero, which is why if a trained network could do it, that would be "impressive" to me, given my personal biases.
>. If you could get a machine that takes in all of Github and can solve "any" DP problem you describe in natural language with a couple of examples, that is AI above and beyond what many humans can do, which is "awesome" no matter how you put it.
I agree with you that such a machine would be awesome, and AlphaCode is certainly a great step closer towards that ideal. However, I would like to have a number measures the "awesomeness" of the machine (not elo rating because that depends on a human reference), so I will have something as a benchmark to refer to when the next improvement arrives.
Yes, and I would like to know how similar the dataset(s) were. Suppose the models were trained only on greedy algorithms and then I provided a dynamic programming problem in the test set, (how) would the model solve it?
> And yet, many humans who participate in these contests are unable to do so (although I guess the issue here is that Github is not properly indexed and searchable for humans?).
Indeed, so we don't know what "difficult" means for <human+indexed Github>, and hence we cannot compare it to <model trained on Github>.
My point is, whenever I see a new achievement of deep learning, I have no frame of reference (apart from my personal biases) of how "trivial" or "awesome" it is. I would like to have a quantity that measures this - I call it generalization difficulty.
Otherwise the datasets and models just keep getting larger, and we have no idea of the full capability of these models.
IIUC, AlphaCode was trained on Github code to solve competitive programming challenges on Codeforces, some of which are "difficult for a human to do". Suppose AlphaCode was trained on Github code that contains the entire set of solutions on Codeforces, is it actually doing anything "difficult"? I don't believe it would be difficult for a human to solve problems on Codeforces when given access to the entirety of Github (indexed and efficiently searchable).
The general question I have been trying to understand is this: is the ML model doing something that we can quantify as "difficult to do (given this particular training set)"? I would like to compute a number that measures how difficult it is for a model to do task X given a large training set Y. If the X is part of the training set, the difficulty should be zero. If X is obtained only by combining elements in the training, maybe it is harder to do. My efforts to answer this question: https://arxiv.org/abs/2109.12075
In recent literature, the RETRO Transformer (https://arxiv.org/pdf/2112.04426.pdf) talks about "quantifying dataset leakage", which is related to what I mentioned in the above paragraph. If many training samples are also in the test set, what is the model actually learning?
Until deep learning methods provide a measurement of "difficulty", it will be difficult to gauge the prowess of any new model that appears on the scene.
As someone who has never messed around with Libc-level programming, it was surprisingly straightforward (and exciting!) to compile Lua all the way.
Cosmopolitan is incredible, thank you so much.
<./ ([&}. - }:) xs