The Santa Fe Time Series Prediction and Analysis Competition

Gradient Boosted Trees: This is a machine learning technique used for predictive modeling. It involves creating a series of decision trees, where each subsequent tree corrects the errors made by the previous ones. Think of it like a team of experts working together, where each expert specializes in fixing the mistakes made by the previous experts. This iterative process leads to a strong predictive model.

Ensembling: Ensembling is a strategy in which multiple individual models are combined to create a more robust and accurate prediction. It's like taking the opinions of multiple experts to make a better decision. In the context of gradient boosted trees, ensembling involves combining the predictions from multiple trees to get a more accurate overall prediction.

Secondary Ensembling Scheme: This refers to an additional layer of combining models to further improve accuracy. It's like having a group of experts (the gradient boosted trees) collaborate and then having another group of experts (the secondary ensembling scheme) collaborate based on the first group's insights.

So, "gradient boosted trees fed into some secondary ensembling scheme" means using a combination of decision trees that correct each other's errors, and then taking the outputs of these trees and further improving their predictions through another layer of combining models. This approach often leads to highly accurate predictions in forecasting tasks.

It's funny you say that. Over the past few months I've ordered copies of almost every book on analog computing that Amazon has, planning to do a deep dive into the "current" state of that, with an eye towards figuring out if there's some "there, there" to explore. I say "current" in scare quotes because there aren't a lot of recent books on the topic, given that analog computing mostly died out decades ago. There does seem to be a bit of a limited resurgence of interest recently though.

Transformers IMO are basically pointing towards time series predictions as being the root of epistemological grounding for "intelligence"

I've also been spending some time studying neuroscience and the literature on "spiking neural networks" based on a vague hunch that that and the analog stuff might tie together into something interesting.

All of that said, I do think that it makes sense to think about hybrid analog/digital systems since digital has so many advantages, especially in terms of flexibility and programmability. The question then, is something like "what's the best architecture that lets you best take advantage of both analog and digital styles of computation"? I won't pretend to say I know the answer to that, but that's a path I'm trying to explore a bit.