Hacker News Reader

Comment by shas3 | Hacker News Reader

Novel paper. But it seems like a lot of the excitement is because of the fusion of two buzz-words, one from the 80s and 90s and another from the 2010s. So, the output is a bunch of stuff coming out of a kinda simple dynamical system. Chaotic for sure, but still simple. Deep learning (and more generally, recurrent neural nets, LSTMs, and derivatives thereof) has been shown capable of learning much more complex nonlinear systems, including human perception. Given this, I think the OP paper is a low-hanging fruit. It was only a matter of time before someone figured out a way to learn specific chaotic nonlinear dynamical systems. Nice work nonetheless.

View on HN

In the example they give they predict the future of a simulation. If you have a perfect simulation, with perfect observation, why not just run the simulation forwards? Well, the goal is to apply this to the real world, where you possibly have only an approximate model and observations - which are noisy and imprecise. So, instead, try predicting the future based on noisy observations. Due to exponential divergence, it seems unlikely this would work. Looking through the paper, it looks like they do not analyze the performance under noisy observation - they just analyze their ability to estimate the Lyapunov exponents under noise, which is much easier.

So the real world application (in terms of forward forecasting) seems like it's limited to cases where chaotic divergence between simulation and real world is due to simulation model errors rather than observation error - the latter is still a fundamental limit. Otherwise, this demonstrates that DNNs can be trained to solve diff eqs well, which is fairly well-trodden work.

As a result, claiming that this can enhance ex. weather prediction is highly misleading at this point, as nothing has demonstrated any prediction performance improvements under noisy observation, which is really the fundamental (and practical) limit in chaotic systems. While it still may be possible to use DNNs to do that (by ex. learning how to most accurately estimate the components of state with the largest Lyapunov constants) I don't think this work demonstrates the feasibility of that yet. The language in the paper is more reserved, in that they primarily claim that they can use the DNN to learn a dynamic model even with observation noise and then that model can be used to estimate Lyapunov constants accurately. That's much more reasonable.

That's like saying, "it was only a matter of time someone figured out time traveling", when it happens. The whole point of the investigation was about chaotic systems, regardless of whether they're complex or not.

Nope, given our current theories time travel is practically impossible. On the other hand we know that chaotic systems are functions and that matrices can approximate any function.

> matrices can approximate any function

I'm not sure what point you're trying to make. Matrices only perform linear transformations, so matrices only approximate functions linearly, which in general, is a terrible approximation globally.

Especially, if you have complex systems where discretization and linearization aren't computationally achievable and/or numerically accurate ... like predicting global weather patterns or even very small experiments. I think about the phrase: All models are wrong; some models are useful.

ITYMeant "A multilayer perceptron is a universal function approximator.

I was pretty sure someone was going to focus on the example I gave instead of the idea :), very predictable. Chaotic systems are not just functions, that's why there's an entire discipline that studies them. There are many ways to approximate functions that have been developed during a long time and all have had trouble approximating chaotic functions.

If you’re going to be that pedantic, I’ll see your pedantry and raise you. Time travel is not only possible, but routine, just in the forward direction only. It’s only travel into the past which is probably impossible, and to be even more pedantic and abstract, impossible only in our observable spacetime geometry.

So is with the chaotic systems: we know for sure that approximating them is just a matter of boundary conditions: that is - the more you know about the starting points, the better you approximate the real world.

I mean, there might be a case where we will finally get to the point where we are just "good enough" at measuring the boundary conditions to predict weather for the next year, but it has got nothing to do with obtaining the knowledge about chaotic systems themselves.

I'm not sure if human perception is a chaotic system.

Chaos is defined as "small change in input -> large change in output".

Perception is actually the opposite, with small input changes (changes in light, different angles,...) leading to fundamentally unchanged perception ("It's a tree").

First, sensitivity to initial conditions is a necessary but not sufficient property for chaos. Some sort of folding/mixing is also necessary, which can be gauranteed by a bounded state space.

Second, it's definitely true that information processing systems, if they are to be reliable enough to be useful, are not going to be chaotic throughout their state space. They need to return the same output given a certain input. But I'd imagine there are also at least a few noisy regions.

I always thought that a perception like "it's a tree" that remains stable could possibly be an attractor in a chaotic system. If you look at the trajectory of a particle around an attractor, its position is very unpredictable after a while, but which attractor it is orbiting is not so random or unstable. Thinking of the visual of the Lorenz attractor.

You need also ergodicity and mixing, as jessriedel states, meaning respectively that you explore every area of your phase space with equal probability, and that two trajectories that begin arbtrarily close together do in fact diverge instead of sticking together, at infinite time

Reply on news.ycombinator.com