variadix

They’re unknown because they don’t exist. There will simply be skills that haven’t yet been automated, but nothing will remain out of reach. The idea of having a career in a single field is probably dead, unless that field is minimum wage laborer.

I’m in a similar boat, though I’ve found some solace in that AI development (for me at least) has also made other kinds of work more enjoyable. Previously, since I enjoyed programming, problem solving, and learning more than the end product, I tried to work on projects where I would learn as much as possible which had interesting problems to solve that would make me a better software engineer.

Now I can work on much larger and much more challenging projects where I care more about the end result than learning little details.

Something has certainly been lost, but I’ve at least gained something else in return. I can also do use an empirical approach now, like testing whether a big change would be beneficial without having to map it out and try to estimate the effect. This has made what I try to work on more research-like than engineering-like; I’m trying to figure out if something is possible rather than how to best implement something I know is possible.

I still worry that soon there won’t be any way to contribute economically, that the machine will eventually be better at everything, but this is a much broader problem than just software or even engineering.

This is making the same mistake: looking at where AI is now and not at the rate of progress it’s made. If you have a thinking machine that is more capable than every human on Earth and thinks 100x faster, how long do you think it will take for it to design humanoid robots.

Does paying someone on Fiverr to draw something for you make you an artist?

Indeed. I think this will ultimately require government intervention and regulation to keep it from destabilizing and monopolizing the entire economy. They’ll have to make AI more like a public utility than a private SaaS subscription.

You can’t necessarily do that in C either, it depends on if your object is in a container that relies on pointer stability. Intrusive linked lists are common in C and they get broken by this, for example.

It remains to be seen, but a reasonable argument against this is found in the progression of chess AI engines. After Kasparov was beaten there was still a period where the best humans working with machines were still better than the machines on their own. Eventually (around 2020 I think?) this was no longer the case, having even the best humans on Earth in the loop was a net negative. There are reasonable arguments against this, but I've found them mostly unconvincing.

The wealthy bring the capital required. The problem is fundamentally the commoditization of intellectual labor. It will simply be a game of whoever spends the most wins.

Who cares what it is if most of us suffer as a result?

Almost certainly the collapse of democracy as a political system and the end of nation states. Also a collapse of the human economy and the rise of the AI economy.

Other people have made this point but the short of it is: for the last few hundred years having more humans on your side meant you would probably win a violent conflict with a party with fewer people, ergo majority rule was the mostly peaceful equilibrium. Powerful technology, specifically AI-enabled drones, robots, etc. changes this dynamic to whoever has the most capital: manufacturing capacity, energy access, compute access, etc. This changes the political equilibrium to dictatorship by capital holders. As the value of human labor decreases due to AI and automation, humans will be priced out of the economy and AI agents will transact with each other. There will essentially be a parallel economy with AIs transacting to build e.g. data centers and solar arrays while humans live in squalor because their labor is effectively worthless relative to AI and robots.

This isn’t inevitable but without major changes wrt. AI regulation and capital accumulation it seems like the most likely result. Even with better governance it still seems likely governments will be incentivized to ignore their citizens to accrue more power by funneling money into the AI economy.

None of the sentences in your comment are even close to correct.

Probably meant albedo

Sounds a lot like “who is going to pick the cotton?” to me.

There is an optimal chess program and there is an optimal AI algorithm. The former is merely practically uncomputable, the latter is theoretically uncomputable.

I think it’s possible AI job disruption (or the threat thereof) could lead to a new labor movement capable of uniting people across the isle. It remains to be seen, but I think if the threat of a “permanent underclass” isn’t enough to get people to compromise and cooperate, nothing will.

At least for me, I think half the value in building to learn was that the knowledge and skills acquired in the process, especially cursory skills and knowledge, might be useful in the future, even if there was no obvious path to application at the time.

I built several projects at home, many involving learning e.g. graphics programming and rendering, that would never be useful in my professional work, but which were intrinsically interesting and enabled me to build other, more useful projects later on. It also gave me greater confidence in my abilities as an engineer, and cursory skills I learned in the process did help in my professional work.

Now it feels like what’s the point. The machines can or will be able to build anything I could want, useful or not, faster and with less frustration. I probably won’t be able to be employed as an engineer long enough to build a career on said skills. And I can’t mentally justify not spending that time with friends and family, when the expected return is basically zero.

I still find math, science, and engineering interesting and intrinsically rewarding, but in a closer sense to how one might feel about playing video games. The information is or will eventually be useless, so it isn’t worth spending a significant amount of time on.

On-prem AI is another solution. I’m surprised more companies aren’t leaning in this direction due to security/IP concerns. My employer is planning to spend several million on local AI hardware.

In the field of pure mathematics this might be true, but it has implications regardless for applied math, engineering, and physics.

I agree it is tautological in some sense. It is more precise to say compression requires prediction, and rather than sampling (as in a generative model), we use the probability distribution to efficiently encode observations. A better predictive model of the space you want to compress gives you a better compressor.

To your second point, it depends on what you mean by “futures you care for”. In my view, if you optimize your compressor for a set of particular inputs at the expense of all other inputs, you are explicitly making a judgement about what future data you expect to see (and in some sense what data you care about). If the space of data you want to compress well includes all truthful, valid English sentences, then in the limit you need a model of the world to make accurate predictions to compress well.

If you want further evidence of the predictive ability of compression, Marcus Hutter has a talk [0] where he presents a few papers that used a compressor (gzip) to approximate the Kolmogorov complexity of a string. Using this computable approximation you can construct the algorithmic information distance between two string by compressing the concatenation of two strings (pieces of text, DNA sequences, etc.) and measuring how well they compress together vs. their independent compressed sizes. The idea is that the Kolmogorov complexity of the concatenation of two similar strings is lower than the complexity of two dissimilar strings, since you can learn the structure of the data from a portion of the data, and use the learned predictive model to better compress other parts of the input data. Using this idea, the paper’s authors were able to perfectly reconstruct phylogenetic trees and linguistic family trees _simply by measuring the compressed size of concatenations_ of DNA sequences and a piece of text available translated to many languages.

[0]: https://youtu.be/8Q2G2OGHm1c?t=2483

I’m not exactly sure what distinction you are trying to make, but a compressor that generalizes from currently observed day to potential future data _is_ a better compressor, as it will have better predictions as it observes more of the source data distribution. Maybe you are making a point about whether compressors can “learn” after compressing the input data, but this also isn’t a fundamental property of compressors. Also compressors do have to deal with non-stationary data sources, so it also isn’t correct to say a compressor only models stationary distributions.

This is a lot less surprising when you learn how non-LZ compressors work, that is, by modeling a probability distribution and using those probabilities to encode information in the minimum number of bits required to transmit the data. A less obvious conclusion is that LZ compressors do this to implicitly, the length of each symbol they could emit (literal or match, etc.) can be converted to the probability distribution the LZ compressor induces, since the number of bits to encode the symbol is related to its probability by the information content.

Really cool project. I wonder what the future of LLM inference will look like. The Talaas demo is promising, but using an ASIC with weights in ROM means you can’t update the model (weights or architecture) without replacing the entire chip. SRAM isn’t dense enough to store model weights, but DRAM has bandwidth issues unless you use HBM which is expensive. Maybe novel memory technologies are the future (there are a number of emerging technologies in R&D), but they likely require breakthroughs to become commercially viable. Systolic arrays could work, one could imagine architectures where routing (architecture) is fixed but weights are programmable, or architectures where the weights and routing are programmable but the compute units are fixed function (coarse grained architecturally reprogrammable), or maybe the weights are in ROM but can be hot swapped easily with into fixed compute elements. Definitely an interesting and emerging field.

Once you have a mask of the positions you want to compress, you can generate a shuffle index vector from that mask to place the desired elements in the low part of the vector. You can expand the mask into nibble-sized indices using pext/pdep and some magic constants, then expand those nibble-sized indices into a vector of indices to use as the shuffle indices.

I think this is one approach to AI safety and interpretability that could work, but would require labs to slow down to figure out how to extract circuits/algorithms out of trained LLMs rather than deploying the opaque artifact.

I’m starting to think the probability of RSI within 12 months is more like 99%

I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which would more likely require months/years, but algorithmic progress would likely saturate quickly. I guess it depends on how much you think further AI progress depends on hardware vs. software.

RTFM

Even worse, people will discuss/diagnose/troubleshoot with their LLM of choice, some of which will be valuable, and the only way to get access to said information will be through the LLM provider’s paywall. I think we are entering an information dark age, for many reasons, that will likely only get worse from here on.

Perhaps, but I think what people mean by intelligence is something that learns and adapts. If LLMs couldn’t do in context learning I don’t think people would think of them as AI, more as a kind of queryable database via natural language. There are other algorithms that learn and adapt, but in much more narrow circumstances that most people won’t obviously interact with.