Comment by effie | Hacker News Reader

Comment by effie | Hacker News Reader

Parent Full thread

Probability is useful also when we have complete knowledge of the state of a model involving many particles (e.g. gas), but can't reliably integrate equations of motions.

This lack of skill is the same towards the future as towards the past, at least in the context of reversible theories of motion like classical mechanics or evolution governed purely by Schroedinger's equation.

Flip a coin, do not look at it, and make a short video of the coin from above when it is at the top position. This video provides us with details of state of the coin.

Assuming a fair coin and a strong enough flip, from this state of things we can't predict the resulting face, and also can't predict the starting face on the thumb. This problem is symmetric towards the future and towards the past.

The same argument holds for knowledge of one microstate of gas at single time instant. Past and future are equally undetermined, and all probabilities are the same towards the future as towards the past.

To get 2nd law, we need to assume the asymmetry in addition, and it's best if it is explicit. E.g a process connecting two equilibrium macrostates, where the two states aren't defined in a symmetric way. For example, as Jaynes does, we define B as the reliable result of autonomous evolution after manual intervention in A. This makes A,B asymmetric, because it is not stated in the assumptions (and usually not considered possible) that also A is the reliable result of autonomous evolution after intervention in B. This asymmetry is what allows us to derive 2nd law.

See e.g. this Jaynes' paper, Section IV The 2nd Law. https://github.com/lawrennd/jaynes/blob/main/gibbs-vs-boltzm...

It is interesting that this derives asymmetric 2nd law from symmetric (reversible) underlying microscopic model. The asymmetry comes in exclusively due to the assumption of B being the reliable result of intervention in A.

> when we have complete knowledge of the state of a model involving many particles (e.g. gas)

We cannot have complete knowledge of the state of a gas. We cannot even have complete knowledge of the state of a single particle because the positions and velocities of classical particles are real numbers but our knowledge is constrained to be finite, so there will always be some measurement error, and so chaos theory applies. (And physical reality is actually quantum, so we have the uncertainty principle at work as well.)

> This problem is symmetric towards the future and towards the past.

Only because you've carefully crafted this example to have this property; it is not true in general. You have constrained your knowledge of the present to exclude that which is needed to reliably extrapolate into the past. But it is possible to change your scenario and collect data that allows reliable extrapolation into the past, for example, by looking at the coin before it is flipped and remembering (or writing down) what you saw. But for the future this is not possible. That is the essential asymmetry.

> See e.g. this Jaynes' paper

I have a lot of respect for Jaynes so I'll take a look but it may be a while before I have the time to give it the attention it deserves.

One thing I see right away that he gets right (well, mostly): in section 6 he observes that there is no such thing as "the entropy" of a system, which is correct. Entropy can only be measured relative to a state of incomplete knowledge. But calling it "anthropomorphic" is not right. It's not a function of human ignorance specifically, it's a function of ignorance in general.

> ...it is not true in general.

Agreed. But it is true for your original argument applied to a non-equilibrium state at any single time instant, while not assuming any more on the past or the future states.

If we adopt further assumptions about the past states, but not about the future states, then we agree this problem of prediction/retrodiction is asymmetric between past and future and this may be the reason why the multiplicity argument does not have to result in past entropy being higher. In particular, there may be other facts on record about the past states and that would override/modify the multiplicity argument for the past.

> it is possible to change your scenario and collect data that allows reliable extrapolation into the past, for example, by looking at the coin before it is flipped and remembering (or writing down) what you saw. But for the future this is not possible. That is the essential asymmetry.

Yes, in any realistic scenario involving prediction and retrodiction at any time, we have some data on the past states (in some lucky cases, enough data for having certainty), but never data on the future states (we may have predictions, but those are not of the same quality as data). I agree.

But in addition, you seem to argue that this asymmetry of availability of data is an essential fact to derivations of the 2nd law.

Consider a scenario where we have an isolated system that is a mixture of miscible liquids that exists for ages but we have data on its state only for the last 2 years, and that data says its entropy was steadily rising the whole time.

Now, given this data, was entropy increasing before the collection of data started, or was it decreasing? For short enough times around the edges, we can extrapolate the entropy curve based on collected data, but for long enough times, we can't. Using the multiplicity argument, including the data, we get that entropy of very distant past states was much larger than entropy of the first recorded state. Which contradicts 2nd law.

Effect of that "essential asymmetry" has a time span that is not universal, but limited to times where collection of data was happening. Outside it vanishes, and using only that data and the multiplicity argument, the distant past becomes as cloudy(highly entropic) as the distant future is.

> it is true for your original argument

No, it isn't. But we need to take this one step at a time. (And, I should also point out, this is not my argument. It's Boltzmann's and Shannon's.)

> further assumptions about the past states

They are not assumptions, they are observations. There is actual evidence that it is possible to remember the past but not the future. (I can't believe I'm actually having to persuade you of that. This is a pretty basic fact about reality.)

> in some lucky cases, enough data for having certainty

Nope, you can never be certain. Even the best measurements have error bounds, and our brains and recording devices have finite storage capacity. Our knowledge of the past can never be perfect. But -- and this is the important part -- our knowledge of the past can be vastly better than our knowledge of the future.

> you seem to argue that this asymmetry of availability of data is an essential fact to derivations of the 2nd law

I don't know if it's essential. I'm just saying that 1) it is a fact supported by evidence, not an assumption and 2) the second law follows from it (plus a few other things). It might be possible to derive the second law without it, I don't know, and I don't care because the actual fact of the matter is that this asymmetry between past and future does exist in point of actual fact. I don't care about hypothetical universes that behave differently from the one I inhabit.

> Using the multiplicity argument

Sorry, what is "the multiplicity argument"?

> Effect of that "essential asymmetry" has a time span that is not universal, but limited to times where collection of data was happening.

No, it doesn't matter whether data is actually being collected. What matters is that it is possible to collect data (about the past) because the past actually happened. Whether or not anyone actually bothered to make a record is irrelevant. If a tree falls in the forest, it actually falls even if no one makes a video of it.

> There is actual evidence that it is possible to remember the past but not the future. (I can't believe I'm actually having to persuade you of that. This is a pretty basic fact about reality.)

You certainly don't have to waste time convincing me of that, but it seems you should try to explain why this general asymmetry of concept pairs record-past, record-future is relevant to your original argument. We're at this for days and you did not write any coherent argument where this evidence (which I don't contest) would play obvious role.

When I talk about some statement as an assumption, this does not mean I contest its factual evidence or validity. I am interested in the structure of your argument, and all assumptions need to be illuminated, even if they are based in factual evidence.

> what is "the multiplicity argument"?

It's your original argument: given the system is evolving without external influences, and we know its macrostate and its entropy at some time, then most likely the future macrostates will be such that they do not have lower entropy, but higher or same, and this is so because there is immensely more microstates corresponding to macrostates of higher entropy (macrostates of high multiplicity) than there is microstates corresponding to macrostates with lower entropy (macrostates of low multiplicity).

I don't see how asymmetry of memories or records made plays role in this argument. I can see how they play role in my example with collecting data for 2years. But then my argument stands: the data becomes more irrelevant the farther into the past we retrodict, and for very distant past it can be discarded, and the multiplicity argument wins, and fails to be consistent with 2nd law.

To me, this is due to fact one cannot derive 2nd law in general, as probable behaviour for all processes. There is as many phase space trajectories increasing entropy as there is decreasing it. One has to select those macroprocesses that we consistently observe to have the same result (Jaynes ' states A -> B) and then one can derived 2nd law for those processes, only.

> Whether or not anyone actually bothered to make a record is irrelevant.

It is relevant in that it changes data based on which predictions (or retrodictions) are made. I'm sure in science, different data means possibly different predictions.

Try to formulate your whole argument (which I understand as being based on the multiplicity argument), including how "it is possible to remember the past" makes its application to distant past which nobody remembers invalid, while keeping its application to distant future valid.

> the data becomes more irrelevant the farther into the past we retrodict, and for very distant past it can be discarded

No. The universe has kept a record of the very distant past. We can directly observe the past all the way back to the cosmic microwave background, which gets us >99% of the way to the big bang.

So yes, I agree with you that it is possible to have a universe where the second law does not apply. It just happens that the universe we live in is not such a universe, not because of any assumptions, but because of what we can directly observe today, and how we can explain those observations in terms of a past that actually existed.

So, we started with your plausible argument why 2nd law holds, then I point out deficiency in it, then we talk past each other, then you say "we can explain those observations in terms of a past that actually existed". In other words, the original argument is not really sufficient, something else is needed. When I push for this to be added and illuminated, you're content with vague remarks on observations of the past. I'm not convinced by your posts at all, there is no convincing argument about illuminating 2nd law (not even in terms of past records) there. Let's agree to disagree.

As you wish. But I think you simply forgot (or maybe chose to ignore) something I said very early on:

"The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery."

> "The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery."

And my answer to this is the same. Unsubstantiated assumption that universe started out, and that its entropy at this start was low. In addition, it is not at all clear how this assumption helps us to explain that an isolated system could not have higher entropy in the past long before we observed it.

> Unsubstantiated assumption that universe started out, and...

You have elided something very important here. I did not say that the universe started out, full stop. I said it started out in a low-entropy state. It is the "in a low entropy state" part that matters, not the "started out" part. If you don't like the idea that the universe "started out" then you can replace this with "the universe was, at some point in the past (specifically, about 13 billion years ago) in a low-entropy state. We don't know how it got there, or what, if anything, happened before that."

And this is not an "unsubstantiated assumption." There is evidence that the universe was in a low-entropy state in the past. In fact, the evidence is overwhelming. Disputing it is akin to disputing that the earth is round.

> it is not at all clear how this assumption helps us to explain that an isolated system could not have higher entropy in the past long before we observed it.

That's because it isn't true. An isolated system can have a high-entropy past, e.g. https://en.wikipedia.org/wiki/Spin_echo

But any system that we actually observe is not isolated, and that is part of the reason why we never observe decreasing entropy despite the fact that decreasing entropy is not only possible, it actually does occur under the right circumstances.

In fact, we can invoke the anthropic principle to explain why we do not observe decreasing entropy: creating a record of the past is not possible in a universe where entropy decreases. There are, by definition, fewer low-entropy states than high-entropy states, and so transitioning from a high entropy state to a low entropy state requires the destruction of information by the pigeonhole principle. And so the reason such universes are not observed is that such universes cannot contain records of the past, and so they cannot contain observers.

That Jaynes paper is very cool. I'm surprised that Wolfram doesn't mention it in his overview of the literature on the second law.

Reply on news.ycombinator.com