dontwearitout

Chapter 1 of "The Ministry of the Future" describes a fictional wet bulb event. It's grisly and horrific and I highly recommend you read this chapter, it changed my view on climate change.

https://books.rockslide.ca/read/780/epub#epubcfi(/6/14!/4/2/...

This is the "hard problem of consciousness". It's more important than ever as machines begin to act more like humans, but my takeaway is we have no idea. https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

How many tokens/s do you get on a 3090? With the extra tokens for the internal monologue, is it still performant enough for smooth VSCode integration?

Dissociatives would work, like dextromethorphan or ketamine. Psychedelics are a subset of hallucinogens.

I've never found it compelling that blinding isn't possible for drug-naive patients. There are other drugs that could induce a "trip" and an unfamiliar individual could mistake for a minor psychedelic experience.

The energy of neutrons isn't really analogous to the energy of atoms in chemical reactions, but absolutely affects the reaction dynamics. The "cross section", or interaction probability, is a strong function of neutron speed. In bombs the neutrons don't get a chance to slow down, but in reactors you try to reduce them to "room temperature" speeds using collisions with light, inert nuclei (the moderator). Here's a diagram showing the interaction probability vs. neutron speed for a few isotopes of uranium: https://tinyurl.com/u235-cross-section

They certainly can, see https://en.wikipedia.org/wiki/Resonance_escape_probability. The all-important number in nuclear reactions is "k", the average number of child neutrons a single neutron will produce. The neutron population follows an equation something like N = exp((k - 1) * t). For k<1, you get exponential decay, and for k>1, you get exponential growth (until everything becomes a plasma and k changes). Check out https://en.wikipedia.org/wiki/Six_factor_formula.

Claude notably does not use RLHF, but uses RLAIF, using a LLM to generate the preferences based a "constitution" instead of human preferences. It's remarkable that it can bootstrap itself up to such high quality. See https://arxiv.org/pdf/2212.08073 for more.

Are dendritic sub-compartments necessary to explicitly model, or does this work just imply that biological neurons are complicated and are better modeled as a multi-layered artificial network, rather than a single simple computational unit?

Similarly, do you think that spiking networks are important, or just a specific mechanism used in the brain to transmit information, which dense (or sparse) vectors of floats do in artificial neural networks?

Your "blip" anecdote is super interesting. I know that exact sensation of disorientation (but without the migraines thankfully), and I'm curious if this has been studied in depth anywhere.

I don't know if you're out of the loop but the AI scene has dramatically changed in the past few years. We are solidly out of the AI winter.

I haven't heard the term "shallow basin hypothesis" but I know what it refers to, these two papers spring to mind for me:

1) Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs https://arxiv.org/abs/1802.10026

2) Visualizing the Loss Landscape of Neural Nets https://arxiv.org/abs/1712.09913

There's also a very interesting body of work on merging trained models, such as by interpolating between points in weight space, which relates to the concept of "basins" of similar solutions. Skim the intro of this if you're interested in learning more: https://arxiv.org/abs/2211.08403

I don't think a dot product between high dimensional vectors is considered a convolution? I'm familiar with convolution between continuous functions, and with kernels in neural networks providing invariance. I'd love to learn more if you have any links that expand on your statement.

Lots of interesting work out there; time will tell!

The transformer module is currently dominating ML, and is widely used in text, vision, audio, and video models. It was introduced in 2017 and shows no real signs of being displaced. It has no convolutions.

https://en.wikipedia.org/wiki/Transformer_(deep_learning_arc... http://jalammar.github.io/illustrated-transformer/

Does this include tools to evaluate for performance on out-of-distribution and adversarial images?

Their point was the insane efficiency, not the low consumption. Modern LLMs are many orders of magnitude less energy efficient; despite all our progress computational biology makes our silicon looks primitive in some metrics.

Yep, transformers showed up in 2017, nearly 7 years ago, and they still wear the crown. Maybe some new architecture will come to dominate eventually, but I would love a low cost PCIe board that could run 80B transformer models today.

Diffusion models could actually be implemented with transformers, hypothetically. Their training and inference is what makes diffusion models unique, not the model architecture.

There's an ocean of difference between optimizing for a single wakeword and the class of models that are taking off today. I'm excited for more on-board processing, because it will mean less dependency on the cloud.

This only goes back to 1913, but says $1 in 1913 is worth $30.65 today. That's pretty rough. https://www.minneapolisfed.org/about-us/monetary-policy/infl...

Does anyone know of any photo management projects that include integration with recent ML models for classification and captioning?

It'd be awesome to have a system that could automatically generate (model versioned) CLIP or SAM embeddings as metadata for your whole library, for downstream plugins to work with (for deduplication, facial recognition, semantic search, text search, etc.)

I think the point being made is that without grid scale storage, wind and solar are not full solutions and not comparable to nuclear energy. Grid scale storage capacity will come, but it's not here yet.

I had to look this up - apparently it's around 0C if you do this! https://mynasadata.larc.nasa.gov/sites/default/files/2018-12...

An aside, but do you find the quality improvements in GPT-4 are necessary for your daily use, or is 3.5 80% as good?

> Definitely a good time to go long on uranium prices in any case.

Is there an easy way for a casual investor to do this?

But it's naive to think there's no effect - some level of chemophobia is warranted IMO, because we do not understand all the complex second order effects to these powerful compounds. Case in point: the article we're discussing.

I firmly believe science will illuminate the true risks, but it may unfortunately take decades. Time will tell, until then I try to eat organic when I can.

There is Zoox

It's great for AI. Today's large language models are very hungry for RAM. Consumer electronics have been rocking just a few GB of RAM for the last decade because there was no real usecase for 100+GB, but that has changed since GPT3 came along. We'll likely see large language and image models integrated into all sorts of software stacks.

Is it an either-or thing? The Nature link I responded to above proposes that human ancestors were coastal dwellers for a few 10's of thousands of years to survive a climate catastrophe. How long would it take for this to start shaping physiology and behavior? This isn't my area of expertise, it's more fanciful fascination than opinion.