Hacker News Reader

Comment by lisper | Hacker News Reader

I am ever more impressed with Wolfram's ability to obfuscate and self-aggrandize.

The second law is very easy to understand when presented as Boltzmann formulated it, and Shannon later generalized it: the entropy of a system is the (log of the) number of states that system could possibly be in given a set of values we can actually measure. In a gas, for example, we can measure its volume, temperature, and pressure, but that still leaves a lot of possible actual underlying configurations of the gas particles. The more possible configurations there are, the greater the entropy. And this explains the second law: given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.

That's really all there is to it on a conceptual level.

View on HN

Well you're giving the statistical mechanics explanation (or information theory), which some thermodynamicists don't appreciate so much, but... I totally agree. Even in person we would joke that his ego filled an entire room, and that's before he got famous. Thank you for reading it so I didn't have to.

> some thermodynamicists don't appreciate so much

Why?

You do not need statistical physics to define entropy. In fact, entropy as a concept was introduced long before statistical physics was a thing. Being able to do thermodynamics without invoking anything related to statistical physics is very valuable tool in mathematical physics, as it leads to definitions and proofs that rely on way fewer assumptions. People should not claim they understand entropy if they understand only the statistical physics way of introducing it. The deep value of the concept of entropy comes from understanding how the two completely unrelated definitions (in thermodynamics and in statistical physics) are equivalent.

Edit to add: In statistical physics you rely on the idea of equivalent microstates. In thermodynamics you do not introduce anything about the microscopic structure of the system under study (an extremely valuable way to generalize your results to cases not covered by typical stat physics assumptions), rather you introduce the zeroth law of thermodynamics as an axiom.

> entropy as a concept was introduced long before statistical physics was a thing

That's true, but the historical development doesn't always make for the best pedagogy. Quantum mechanics, to take my favorite example, is vastly simpler than it is commonly made out to be [1]. It just took about 80 years for this to be understood.

The development of thermodynamics followed a similar path because thermodynamics predated the widespread acceptance of the atomic theory. Indeed, statistical mechanics played a major role in getting the atomic theory accepted. But it does not follow that the thermodynamic treatment is superior in any way. I challenge you to explain to me what entropy actually is in fewer words than I used without reference to atoms or microstates. I'll bet you can't even explain what "temperature" is under those constraints, and that is much simpler.

---

[1] See https://en.wikipedia.org/wiki/Quantum_Computing_Since_Democr... chapter 9

You are right about pedagogy. But "understanding" requires much more than giving one simple definition without discussing the consequences of the definition.

Your (statistical) definition of entropy is perfectly reasonable and fairly intuitive and it should be taught first (as you alluded twice and as it is relatively common these days). But "understanding" entropy involves understanding multiple facets of it and why it is useful, not just knowing the easier definitions. Hence, it seem unreasonable to claim "understanding" entropy without having familiarity with the independent thermodynamic definition of entropy and *why are they equivalent*. That would be like saying one understands programming language theory while being familiar only with object oriented programming.

Using atoms and microstates to explain entropy makes the explanation shorter but it also makes it *more* complex because it now depends on the definition of *atoms and microstates*. The thermodynamics explanation of entropy does not actually require more words, but crucially, it requires fewer concepts (heat, useful work, and heat transfer).

Concerning the development of thermodynamics: defining entropy *without* statistical physics requires *fewer* assumptions about the physical system. Statistical physics did not make thermodynamics simpler, nor is it necessary to explain thermodynamics. To your example about quantum mechanics: this is different because statistical physics is not some better more-fundamental explanation of thermodynamics. The two are separate sets of axioms with equivalent explanatory power.

Concerning the definition of temperature: are you aware that there is an implicit assumption about temperature scales when defined in the statistical physics way? That is much more explicit in the thermodynamic treatment. And it is why I brought up the zeroth law of thermodynamics: it is where temperature is defined. If you think temperature is easier to define in the statistical physics setting, you are simply hiding a ton of assumptions under the rug, which is not a very good way to build a mathematical theory.

I am curious what do you claim is a simple statistical explanation of temperature? If it is something about average energy, that is a *wrong* explanation of temperature, valid in only a few special circumstances.

As to actually giving you the thermodynamic definition of temperature: The joke is that it is whatever the thermometer says it is. The rigorous version of that joke is: According to the zeroth law of thermodynamics, if bodies A and B are in thermal equilibrium (no heat transfer) and bodies B and C are in thermal equilibrium, then A and C are also in thermal equilibrium. Use this equivalence property to tag all bodies that belong to the same equivalence class. This tag is your measure of temperature. It is an arbitrary (ordered) scale. If you want to, you can then use the second law of thermodynamics to pick a neater canonical scale.

Finally, the definition of entropy: enumerate the (macroscopic) variables describing your system and draw them as a coordinate system; compute the value of ΔQ/T as you travel through some path in that coordinate system; the second law of thermodynamics says that value is zero on a closed path, thus we have a potential function; name that potential function entropy.

Mathematically, in many aspects that is both a simpler and a more powerful definition. One could hardly claim to understand entropy if they do not understand the enormity of the previous paragraph and its relationship to the independent statistical definition.

TLDR: "Understanding" entropy is about understanding why these vastly different definitions (thermodynamic and statistical) are equivalent, not about being able to recite one or the other. Especially given that statistical physics is not more fundamental than thermodynamics (unlike special relativity which is indeed more fundamental than Newtonian gravity)

> But "understanding" requires much more than giving one simple definition without discussing the consequences of the definition.

But I did discuss the consequences: my definition (well, Boltzmann and Shannon's) leads directly to the second law, and in a way that is much more general and intuitive than the thermodynamic approach. What, for example, is the entropy of a cryptographic key? On the thermodynamic approach that question doesn't even make sense to ask. It's a category error because the thermodynamic definition relies on the concept of temperature and cryptographic keys don't have temperatures.

> Using atoms and microstates to explain entropy makes the explanation shorter but it also makes it more complex because it now depends on the definition of atoms and microstates.

Complexity is in the eye of the beholder. The classical billiard-ball model of atoms is perfectly adequate to explain the second law -- both the thermodynamic and information theoretic version -- and most people have no trouble wrapping their brains around the idea of "lots of tiny billiard balls wiggling around".

> I am curious what do you claim is a simple statistical explanation of temperature?

Higher/lower temperatures mean that the billiard balls are, on average, moving faster/slower relative to each other, all else being equal. And yes, I know that is not entirely accurate, but neither is the idea that atoms are billiard balls in the first place. This model is perfectly adequate as a first-order approximation. It allows you to describe the qualitative features of a heat engine purely in terms of the Newtonian mechanics of colliding billiard balls, which most people have a pretty good intuition for. No, it's not 100% accurate in the thermodynamic case, but the benefit is that it is obvious how to apply the concept of entropy defined in statistical terms to computation and information. I think that's a worthwhile tradeoff. Knowing how to calculate the efficiency of a heat engine is not very useful to most people in today's world. On the other hand, knowing how to estimate the entropy of a password or cryptographic key is very handy.

> The joke is that it is whatever the thermometer says it is.

Yes. But that joke conceals a deep truth: actually describing (let alone defining) thermodynamic temperature without reference to atoms, and without resorting to the thermometer joke, is very, very hard.

(It is analogous to trying to explain quantum mechanics without reference to entanglement, which is how it was done for decades. The problem with that approach is that it leaves you waving your hands about what a "measurement" is, because measurements are entanglements. Once you understand that simple fact, all of the mysteries of QM simply evaporate.)

> if bodies A and B are in thermal equilibrium (no heat transfer)

And how can I tell if A and B are in "thermal equilibrium" without resorting to the joke?

(BTW, your definition as you've given it is actually wrong. I can have two systems that are different temperatures but with no heat transfer between them simply by separating them with a perfect insulator. So now you have to incorporate that into your definition somehow, which means you have to define "insulator", and I don't see how you're going to do that without going around in circles.)

> It is an arbitrary (ordered) scale.

No, it's not. If it were arbitrary, the concept of specific heat would be meaningless.

Ultimately you need to make a connection between heat and motion. That is, after all, the thermodynamic project. You can convert heat into motion, and you can convert motion into heat, but the latter is much, much easier than the former. Why?

The statistical answer is: because heat is motion. It's a particular kind of motion. But if you deny yourself a referent to the things that are moving in the case of heat (atoms) then you deny yourself access to this simple straightforward explanation of the fundamental question of thermodynamics.

> "Understanding" entropy is about understanding why these vastly different definitions (thermodynamic and statistical) are equivalent

I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems. I've never seen that done successfully.

There's a reason that the atomic theory is a thing.

I mostly agree with the first half of what you wrote, especially in terms of pedagogy, but when you delved in the nitty-gritty in the second half, there are a lot of mistakes that would impede understanding "advanced" concepts in physics. I believe that is at the root of our disagreement - you are perfectly right about early pedagogy, but understanding (which for me means "capability of advanced applications") requires more. Disdain for the thermodynamic definition limits ones toolbox. (Edit: "disdain" is a bad choice of words)

My main claim is: building the notion of entropy without relying on atomic theory is crucial if you want to apply it to "interesting" things like quantum mechanics or black hole physics. Admittedly that does not contradict your earlier posts, but it does significantly contradict the philosophy of your last post.

An assortment of issues I have with the second half of what you wrote as related to our discussion of what "understanding entropy" means:

- Passwords and cryptographic keys do not have entropy. What we call password entropy is a useful heuristic, but it differs from the rigorous information theory notion of entropy (or the thermodynamic one). I claim that it is important to understand that difference as it leads to understanding the limitations of the heuristic.

- The definition I have given is not wrong, it is one of the most standard ways scientist have been defining temperature for the last century. "Perfect insulators do not exist" is an extremely important theorem that you "derive" in theoretical physics and use for a vast array of incredibly important thought experiments on which much of our understanding of nature is build. Not taking seriously the thermodynamic definition of entropy thus limits your power to explain Nature.

Edit: "perfect insulators do not exist" is not a statement about how difficult it is to solve some engineering problem. It is an absolute statement with certainty equal to that of "√2 is irrational" or "you can not solve the halting problem".

- Temperature is indeed an arbitrary scale, again something that has been established in pretty much any textbook on statistical physics or thermodynamics. At this point I really think it is worthwhile to play the authority card: you are claiming multiple things that contradict standard physics textbooks on which the last hundred years of science are based.

- The heat engine efficiency is indeed not something most people care about day to day, but most people do not care about entanglement either. Moreover, I do not care about the heat engine as a question of engineering. However, thought experiments that involve heat engines are crucially important for the derivation of many no-go theorems (including in quantum mechanics).

- This is incredibly important: heat is NOT motion. This is incredibly limiting worldview that is insufficient for anything but the simplest of toy problems. Very useful early pedagogical tool, but it has to be discarded early on if you want your student to grow.

Sure, it is reasonable to say that it is a bad pedagogy to start with the thermodynamic definition. I teach graduate classes both in physics and in quantum information science (theoretical CS) and I actually follow your way of presenting entropy in the first few lectures. But the atomic theory interpretation of entropy is actually limiting when it comes to advanced physics. Really, the only thing I disagree about with you is your insistence that one does not need the thermodynamic definition in order to "understand" entropy. Please excuse my use of the word "insistence" if that is not the case - the limitation of text-based conversation are creeping in.

> I agree, but I think it's much easier to start with the statistical definition and show how the thermodynamic one follows (because that is straightforward) than to start with the thermodynamic definition and try to extend it to non-thermodynamical systems.

100% yes

> I've never seen that done successfully.

Most of the graduate classes I have taken have done that relatively well, but I would agree with you that undergrad classes need more work in the US. For what is worth, the French Class Prepa curriculum on this topic is absolutely outstanding. I particularly like classes/textbooks that take the differential geometry approach to thermodynamics. As long as the textbook treats stat physics and thermodynamics separately, it is probably a textbook that does it well.

I'm not really relying on atomic theory per se, I'm simply relying on the notion of macroscopic observables that encompass an ensemble of microstates. It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition.

Now, it's possible that my insistence on this is a reflection of my ignorance. I'm not a physicist, I'm a computer scientist. But there are a few things that I know (or think I know) that are at odds with what you are telling me. For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary. You get to pick two points on the scale, but the remaining points are then determined for you by actual physics. There is a whole field of study called calorimetry which relies on this. Your profile says you are a physicist so you must be aware of these things.

I'm not pointing this out to challenge your bona fides, only your pedagogy. You say, for example, "Heat is not motion" and that this is "incredibly important", and that "perfect insulators do not exist is a theorem", but then you don't provide any explanation or references. You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive.

BTW...

> Passwords and cryptographic keys do not have entropy.

You are mistaken. Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic".

(Imagine how annoying it would be if I stopped there and said nothing further.)

Entropy is only a coherent concept relative to a state of (generally incomplete) knowledge. It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas in cryptography it is determined by withholding information (i.e. keeping secrets). If I flip a coin, and I peek at it but I don't show you, that coin has zero entropy relative to my knowledge, but one bit of entropy relative to yours. The only difference between that and thermodynamics is that in the latter there are O(10^23) coins and it is impossible for anyone to peek at them.

My password has zero entropy relative to my knowledge, but (one hopes) a lot of entropy relative to an adversary's knowledge. Indeed, the amount of entropy in my password (measured in bits) is precisely equal to the base-2 logarithm of the number of possible passwords that an adversary would have to try in order to guess it.

> It just so happens that in thermodynamics those microstates are the positions and velocities of particles, but that that is a reflection of the underlying physics. It's not inherent to the definition.

Yes, but the rather complicated notion of microstate (or probability or knowledge or information) is not necessary in the thermodynamic definition, which is why it is valuable.

> For example, there is this physical phenomenon called specific heat, which makes temperature scales non-arbitrary.

If you start by introducing the existence of specific heat (which is a pretty good way to introduce concepts related to temperature both in the statistical and in the thermodynamic case) then you indeed need to choose a canonical temperature scale. Kinda how introducing exponentiation through its derivative provides a canonical choice for the basis of the logarithm. But it is important to appreciate the fact that this canonical choice is somewhat arbitrary: it just makes the equations look more "natural", nothing more.

> I'm not pointing this out to challenge your bona fides, only your pedagogy.

But I never challenged your assertions on pedagogy. I repeatedly and enthusiastically agreed with them in all my posts. I just strongly disagree with your definition of the word "understand".

> You may be right about these things, but you haven't actually defended or supported them. You've just put them out there as bald assertions. I don't see how you expect that to be constructive.

Part of it is that I never claimed that understanding entropy is easy. I multiple times referred to the axioms in question by name, but did not claim there is a simple explanation that would let one understand what entropy is. The wiki pages on the topic are quite instructive though:

- https://en.wikipedia.org/wiki/Zeroth_law_of_thermodynamics#F...

- https://en.wikipedia.org/wiki/Thermodynamic_temperature

- Most importantly, the various statements of the second law. I claim fully understanding entropy requires understanding why these various statements are equivalent: https://en.wikipedia.org/wiki/Second_law_of_thermodynamics#V...

> Information-theoretic entropy is a generalization of thermodynamic entropy, not a "heuristic".

No, they are different concepts, neither one is generalization of the other. That is the point I am making all along.

"Password entropy", the way it is commonly used and how I assumed you used it, as in "here is my password, calculate its entropy" is not (information theoretic) entropy, it is just a useful heuristic figure of merit. You can not have entropy without having some probability distribution as you already said in your clarification. And a password is just a single microstate: there is no such thing as entropy of a single microstate, but again, it seems we are in agreement about that.

> It just so happens that in thermodynamics the incompleteness of our knowledge is determined by physics whereas.

This should say "in statistical physics". Thermodynamic entropy has (superficially) nothing to do with knowledge or information.

Few more fun references if you are into extreme mathematical rigor:

- https://web.ist.utl.pt/berberan/data/68.pdf

- https://en.wikipedia.org/wiki/Constantin_Carath%C3%A9odory#T...

Edit to add: There is a neat "contradiction" between the notions of thermodynamic entropy and statistical entropy. Namely, the notion of ergodicity. Thermodynamic entropy does not let you understand ergodicity or the fact that entropy is permitted to spontaneously drop. Information theoretical approaches do permit you to talk about entropy dropping on its own. While on the topic of pedagogy, discussing this clash is probably quite fruitful. But it is a bit outside my area of expertise. The following is probably a good place from which to start picking terms to google https://math.stackexchange.com/questions/433854/relationship...

This just occurred to me:

> Perfect insulators do not exist

Then it is not possible for any systems ever to be in thermal equilibrium before the heat death of the universe. So your definition of temperature is based on something that you yourself have said is physically impossible to achieve.

You are kinda right :D There is always a tension between rigor and practicality in theoretical physics (and frankly, in math -- just read how giants like Poincare felt about rigor or how Euler worked with infinite series in horrifically handwavy fashion).

This level of hierarchical abstraction is pretty much the only tool we have though, and it seems it works better than whatever else we have tried. Axiomatic thermodynamics (the last couple of links in my previous post) does answer this particular problem though, it is just that no-one uses it.

This is not about rigor vs practicality, this is about logical coherence. You cannot base your definition of a physical quantity on a situation that is physically unachievable even in principle and expect to be taken seriously.

> no-one uses it

Gee, I wonder why.

Here is how it's actually done:

https://www.nist.gov/si-redefinition/kelvin-introduction

It really does not feel like you are taking this conversation up with intellectual honesty. For starters, rigor and logical coherence are the same thing. Second, the links I provided go into the rigor/logical coherence and already discuss the information from the link you shared in much more details. You can not at the same time complain about pedagogy (providing a handwavy explanation of entropy as the one that has better pedagogy) and rigor (providing a winding explanation that deals with all the intricacies but requires a lot of reading). Just get of your high horse and actually read through the things I shared. It is becoming silly to argue about entropy when you disregard half of the literature on the topic on a hunch.

It is truly silly to share a pop-sci blurb from NIST in this situation. It certainly provides a good initial intuition, but it skips all the rigor and logical coherency you pretend to care about (and which, again, is discussed in what I shared).

> rigor and logical coherence are the same thing

Well, you are the one who introduced the term "rigor" into the discussion, but you didn't define it so I had to infer what you meant, and the only clue I had to go on is that, whatever you meant, it was at odds with "practicality". So I assumed that "rigor" meant something like, "thoroughly and explicitly attending to all of the details of an argument, defining all the terms, making all the assumptions and logical reasoning steps explicit" or something like that. So sweeping some of the details of rigor under the rug in service of "practicality" was an acceptable transgression.

On the other hand, I am the one who introduced the term "logical coherence" so I'm the one who gets to say what it means: a logically coherent argument is one which is not based on any transparent contradictions or logical fallacies. On this view, rigor and logical coherence are orthogonal. You can have a rigorous argument that is not logically coherent (here is an example: https://math.hmc.edu/funfacts/one-equals-zero/), and you can have a logically coherent argument that is not rigorous. You can even have an argument that is both rigorous and logically coherent but is nonetheless wrong. But a logically incoherent argument cannot be correct. Logical coherence is necessary but not sufficient for correctness.

It doesn't really matter. What matters is that "rigor", on your view, is something that can be casually sacrificed on the alter of "practicality". Logical coherence is not. If you sacrifice logical coherence you abandon all pretense of hewing to logic and reason, and your position is no longer worthy of serious consideration or rebuttal beyond the observation that it is logically incoherent.

What makes your position logically incoherent is that you are taking a position on what it takes to truly "understand" the second law, but you are unable even to define the word "temperature". Your first attempt was literally a joke, and your second was based on a logical contradiction. And BTW, when I pointed this out, your response began with "You are kinda right :D"

No, I'm not "kinda right". I am right, full stop, no smiley. Your stated position includes both P (there are no perfect insulators) and NOT P (it is possible for two systems to be in thermal equilibrium with each other). And this is not an inference, you were absolutely adamant about both of these things:

> [P]erfect insulators do not exist" is not a statement about how difficult it is to solve some engineering problem. It is an absolute statement with certainty equal to that of "√2 is irrational" or "you can not solve the halting problem".

> According to the zeroth law of thermodynamics, if bodies A and B are in thermal equilibrium (no heat transfer) and bodies B and C are in thermal equilibrium, then A and C are also in thermal equilibrium. Use this equivalence property to tag all bodies that belong to the same equivalence class. This tag is your measure of temperature. It is an arbitrary (ordered) scale.

I don't need to know anything else to know that what you are saying is wrong, just as I don't need to know the details of a perpetual motion machine design to know that it won't work. (Though it certainly doesn't hurt to further observe that what you are saying is at odds with the SI definition of the Kelvin, to say nothing of the fact that I can accurately predict how much electricity it will take to boil a cup of water.)

BTW...

> It is truly silly to share a pop-sci blurb from NIST in this situation.

That citation was more for the benefit of lurkers than it was for you. You have clearly buried any desire to seek the truth under layers of defensiveness that are far beyond my ability to penetrate.

NIST is quite literally in the business of defining physical quantities in ways that allows them to be accurately measured, and also of communicating to the public how their work is done. If you think it is "silly" to cite one of their official publications in the context of a discussion of the pedagogy of the second law, well, all I can say is that I feel sorry for your students. I really urge you to try to develop a little more empathy for your audience, because if you don't you are going to do a tremendous amount of damage.

Did you even bother reading any of the references you asked for or googled any of the terms/axioms/theorems I quoted? Half of the things you claim I said are quite willful misquotes of what I wrote. Look, I tried to read what you wrote charitably (hence the smileys and multiple mentions of the limitations of online comments as a medium). Without you doing the same, I have trouble imagining how we can agree on the truth. Again, check the actual rigorous logically coherent textbook references I shared, not a popsci page (which is indeed from a reputable source, but popsci nonetheless). By the way, that popsci page defines just the unit of Kelvin, not the concept of temperature - if you deny there is a difference between these two notions, there is hardly a chance this conversation would go anywhere.

> Half of the things you claim I said are quite willful misquotes of what I wrote.

All of the things I claim you said are verbatim copies of things you wrote, copied and pasted directly from your comments, with one exception: I changed a lower case p to an upper case P, but even there I indicated the edit with [brackets] as is customary. No reasonable person could consider that a "willful misquote".

> check the actual rigorous logically coherent textbook references I shared

What textbook references? You linked to one paper by Pogliani and Berberan-Santos, and one mathexchange article. All of your other links were to Wikipedia.

BTW, I just browsed through the P&B-S paper, and it is also logically incoherent.

> ... his famous second axiom, that constitutes the real novelty of his work ... “In the neighborhood of any equilibrium state of a system (of any number of thermodynamic coordinates), there exists states that are inaccessible by reversible adiabatic processes”. ... It can be noticed that in the axioms and definitions of the new method there is no mention of heat, temperature or entropy whatsoever

But that is not true. The axiom is framed in terms of "adiabatic processes" and "equilibrium state", both of which depend on the concept of heat.

I'm sorry, but if this is the best you have, then the whole field of axiomatic thermodynamics is utterly bankrupt.

> Concerning the definition of temperature: are you aware that there is an implicit assumption about temperature scales when defined in the statistical physics way?

What’s wrong with the usual arguments for the identification of beta as being (proportional to) the inverse of the thermodynamic temperature? You mean that it’s based on an arbitrary choice of constant to identify the entropies?

Mostly yes (but we can probably debate semantics). It is not at all obvious that the inverse beta should be proportional to what we conventionally call temperature. One needs to prove that.

Once we define the entropy its integrating factor is what we conventionally call temperature. I agree that correspondence between things is not obvious - but once defined it seems to work and I don’t know if we could have more proof than that. I also agree that thermodynamics is worth studying in itself - and otherwise the attempt to reproduce it using statistical mechanics doesn’t make a lot of sense.

> Today I would have more strongly made the rather Feynmanesque point that if you have a theory that says everything we observe today is an exception to your theory, then the theory you have isn’t terribly useful.

I've found this quote amusing and I believe it explains to some extent problems with the Second Law.

> The second law is very easy to understand when presented as Boltzmann formulated it

When I read prominent scientists I often become impressed by their creative ability to not understand "simple" ideas and to spend years in attempts to understand them. It seems for me that this ability and willingness to think is the main driver of a scientific progress.

Einstein for example rejected a very simple idea of time, and reinvented it to create the theory of relativity. How on Earth someone with a brain can not understand time? But scientists can and they find this useful. I think they have special secret training on incomprehension.

To apply your mind to a problem you need first to find a problem to attack, and this is the most tricky part of science. If you do not see problems with existing theories you are out of luck, you have nothing to attack. So your first reaction to any idea presented to you better be "I do not understand" then "it is very easy and even obvious".

> problems with the Second Law

Like what?

> Einstein for example rejected a very simple idea of time, and reinvented it to create the theory of relativity.

But that _really_ isn't what he did! He started from the seemingly obvious notion of linear time. Added the _observation_ of the finite speed of light. And _noticed_ that the "obvious" notion of linear time leads to contradictions (basically saying "at the same time" has different meanings for different observers). And from that he _had_ to go out and construct relativity in order to reconcile these contradictions. Does it matter to anyone? In practice? Probably not. As long as you do not get close to the speed of light (either in velocity or in mass. Yes.. that statement makes sense as is) relativity really is indistinguishable from our notion of linear time. This is _necessary_ for relativity to be a useful theory.

But it is _also_ a massive shift in how we think about the world. That's the danger with "intuitive" understandings. They can only ever be based on experiences and those do not lend themselves to extrapolations. The 2nd law of thermodynamics is one of those things. It seems completely obvious if learn about it the first time. Then you go a bit deeper, start picking it a part, thinking about the second and third order consequences of it. Then you argue a bit about whether the probabilistic formulations are worth anything in the first place, then you try to reconcile it with information theory. At this point you get into the whole quantum mess and ask yourself how this "law" can hold in all these circumstance, even if it doesn't seem to have any reason to mean anything anymore.

To repeat even more tersely, the fraction of information you have about a system can only go spontaneously down, never up. Knowing less is free, learning is at cost.

That is brilliant. Is that your idea, or did you get it from somewhere?

Mine.

Cool. I would like to use this idea and credit you but you have no identity info in your profile. Shall I just credit this to your handle, IIAOPSW?

Do me a favor, credit it to Euler.

It is the notion behind "Completely Positive Trace Preserving" maps. It is used a lot in quantum information and quantum computing. It is present by other names in other situations.

That’s not the fundamental formulation which does not require statistical mechanics. Anybody saying entropy is very easy to understand doesn’t understand it.

> That’s not the fundamental formulation

Why not? By whose standard is a different formulation more "fundamental"?

It is more about the fact that there are two "equally fundamental" explanations, and saying there is only one is a great disservice to the curious mind: the most fascinating and deep insights about entropy come from understanding how the two definitions are equivalent.

> The more possible configurations there are, the greater the entropy. And this explains the second law: given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.

This is an argument for observing high entropy macrostates at any time, assuming all microstates are equally probable. But not for evolution in time that makes entropy higher in the future, or at least non-smaller in the future. You assume the closed system evolves its microstate in time "by pure chance", i.e. any microstate is equally probable as the next microstate, which ignores many constraints there are due to physics laws like conservation of phase volume or energy. This probabilistic argument could be applied to past just as well as the future. But observations show entropy of an isolated system is lower in the past than in the present. So your argument does not really derive all aspects of 2nd law.

> But not for evolution in time that makes entropy higher in the future, or at least non-smaller in the future.

Well, obviously the system is subject to the constraints of its dynamics. It's not just picking a new microstate at random at each instant in time. It can only evolve to states that are accessible from its current state according to its dynamics. But given that set of accessible microstates, you are more likely to get a macrostate corresponding to a larger number of microstates than a smaller number.

The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery.

> But given that set of accessible microstates, you are more likely to get a macrostate corresponding to a larger number of microstates than a smaller number.

This is due to mixing in phase space, which is plausible for many particle systems. However, it works in both directions of the parameter t.

The only way to derive 2nd law including its time asymmetry (which is the major point of 2nd law) is to restrict considerations to those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state. This means we consider only subset of all microscopically possible processes. E.g. Jaynes selects those macroscopic processes that bring one macrostate A, via some irreversible evolution, to another macrostate B, repeatedly. Then, assuming this reliability of the resulting state, he derives non-decrease of entropy from maximum entropy principle and Hamiltonian statistical physics. If we didn't assume this restriction and allowed all mechanically possible processes, entropy could both increase and decrease.

> The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery.

This often repeated idea is in vein with Clausius' ostentatious statements on energy and entropy of the Universe, which have been out of place and plagued discussions of thermodynamics with needless mystical assumptions ever since he wrote them. There is no evidence for them, as there are no reliable measurements of entropy of the Universe. There is no need for them, as there are no good reasons to believe Universe has defined entropy at all. Thermodynamics, including 2nd law, is about finite sized systems free of strong gravity effects. It does not extrapolate easily beyond compact systems like planets/stars to ever bigger systems. There is no Universe in it anywhere.

2nd law that we see in daily life or a lab is not a statement about entropy of Universe being lower in the past, nor does it hang on such an assumption. It is (in one of its variants) the statement that a finite isolated system can change its macrostate to some other final macrostate, but this final macrostate is of higher or same entropy, so in real process entropy can't decrease.

> The only way to derive 2nd law including its time asymmetry...

I don't think that's true, though I'm not sure I understand what you mean by "those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state". (The same as what? The same in what sense?)

The reason you can't unscramble an egg is not because there is anything fundamentally irreversible in the laws of physics, it's because the time-evolution of the scrambling egg is governed not only by the motion of the whisk but also by the random thermalized motion of the constituent particles. To unscramble the egg you would need not only to precisely reverse the motion of the whisk, but also all of the thermal velocities of the constituent particles. The latter might be possible in principle (though not in practice) in a classical world, but throw in quantum mechanics and it's not possible even in principle. So the second law happens simply because the dynamics of non-trivial systems (like eggs) are chaotic, and quantum mechanics always throws in some randomness at the lowest levels.

And yes, quantum mechanics is also time-reversible. If you want to quibble about that, read this first:

https://blog.rongarret.info/2014/10/parallel-universes-and-a...

> There is no evidence for them

If you believe that the 2nd law is universal (and I see no reason to doubt it) then the inescapable conclusion is that the further back you go, the lower the entropy of the universe was, reaching a global minimum at inception.

> I'm not sure I understand what you mean by "those microscopic processes that are compatible with macroscopic processes that systematically lead to the same final state". (The same as what? The same in what sense?)

In the formulation of 2nd law for adiabatically isolated systems, there is the initial equilibrium macrostate A, and the different equilibrium macrostate B that the system gets into eventually after the intervention, where either some constraint is released, or some macroscopic work is done. 2nd law states that if the same intervention in A results in B reliably, then S_B >= S_A.

This statement can be derived from mechanics and maximum entropy principle only under the assumption that B is a reliable result of intervention in A (always or with close to 1 probability). This is more than mechanics or logic can provide, and it means we have a restriction on the allowed set of microscopic processes considered.

It is not possible to derive 2nd law from mechanics or logic for all microscopic processes. Some microscopic processes bring the state B to state A, and those break 2nd law.

> the initial equilibrium macrostate A, and the different equilibrium macrostate B that the system gets into eventually

[emphasis added]

Your use of "the" here, with the implication that there are two unique microstates A and B, is problematic. There is no such thing as "the" equilibrium microstate. A system at thermodynamic equilibrium is not static.

(In fact, I think it is fair to say that the whole point of thermodynamics is to sweep vast numbers of the physical degrees of freedom of a system under the rug so that you can treat a system at equilibrium as if it were static even though it really isn't. It is frankly astonishing that this is even possible at all without losing any fidelity in terms of measurable outcomes.)

> Some microscopic processes bring the state B to state A, and those break 2nd law.

Yes, that's true. The second law is probabilistic. It is "possible" to break the second law in the same sense that it is "possible" for a baseball to quantum-mechanically tunnel through a catcher's mitt. You can do the math to figure the odds in both cases. In fact, actually going through this process is an informative exercise. It isn't difficult.

There are some events whose probabilities are so low that the chance of them occurring anywhere in the universe before heat death is barely distinguishable from zero. The odds of actually observing such an event here on earth are vastly smaller still. Both macroscopic tunneling and 2nd law violations are events of this sort. Neither is categorically impossible, but (to put it mildly) it's pretty safe to bet against them nonetheless.

You have missed my point. The formulation of 2nd law I'm talking about is stating something about all pairs of two MACROstates A,B, not MICROstates, such that B reliably succeeds A after intervention. This pair of MACROstates is not assumed unique, there are hugely many, but formulating the argument uses a single pair. (I'm not a native speaker so don't get distracted by possibly incorrect use of the definite article).

The problem with your handwavy argument about macrostates with higher multiplicity is not due to 2nd law being a probabilistic claim. The problem is that it is not clear from that argument alone why it applies to future states, but it does not apply to the past states. To address that, another assumption is needed, such as the asymmetric relation between my macrostates A,B.

Sorry, my mistake, I somehow misread "macro" as "micro". More than once. But it was obviously an "a" all along because it's an "a" in my quote. I need to start paying closer attention.

So, having now re-read what you wrote upstream more carefully, can you elaborate on what you mean by "intervention"?

> The problem is that it is not clear from that argument alone why it applies to future states, but it does not apply to the past states.

I don't understand this. What do you mean that it "does not apply to past states"?

> another assumption is needed, such as the asymmetric relation between my macrostates A,B

Well, yeah, there is an asymmetric relationship: one precedes the other, and not the other way around. But that's not an "assumption", that is just how you defined A and B.

I really don't see the problem you are trying to describe.

By intervention I mean some external agent causing the system to leave the initial equilibrium macrostate A, which would not happen spontaneously. E.g. a wall is removed, or a piston is quickly pushed, to make the system evolve towards new equilibrium macrostate.

> given a set of measurement values, the more states there are which produce those values, the more likely we are, all else being equal, to observe those values. And so a closed system will evolve by pure chance into a state where the entropy is maximized simply because you are more likely to observe values corresponding to greater numbers of underlying states.

This argument is that future macrostates of an isolated system passing through non-equilibrium states (due to some intervention to the system that was in an equilibrium state before) should be almost always of higher entropy, because those have immensely higher multiplicity than macrostates with lower entropy. There is hugely more microstates that are compatible with higher entropy macrostate, than microstates that are compatible with lower entropy macrostate.

However notice the word "future". We can formulate the same argument using "past": past macrostates of an isolated system passing through non-equilibrium states should be almost always of higher entropy, because those have immensely higher multiplicity than macrostates with lower entropy. There is hugely more microstates that are compatible with higher entropy macrostate, than microstates that are compatible with lower entropy macrostate.

Now, why does the argument give correct answer when applied to future macrostates but not when applied to past macrostates? We need additional assumption about the non-equilibrium process that makes the argument invalid for the past.

> By intervention I mean some external agent causing the system to leave the initial equilibrium macrostate

OK, that's what I thought.

> We need additional assumption about the non-equilibrium process that makes the argument invalid for the past.

Ah.

No, you don't need any additional assumptions. All you need is to observe that the present constrains the past differently from the future. There are fewer pasts compatible with a given present than there are futures compatible with that same present.

You can't just say that there are many high-entropy microstates, and so you are as likely to encounter one in the past as you are in the future, because any candidate past state has to be able to evolve into the present state. And if the present state is a low-entropy state, the precursor for that is much more likely to be an even lower-entropy state. Yes, there are high-entropy microstates that will evolve into low-entropy ones, but the fraction of those states chosen from among all high-entropy microstates is indistinguishable from zero, and so the odds of a high-entropy microstate in general being the precursor of a present low-entropy state is likewise indistinguishable from zero.

By way of very stark contrast, the successor states of a low-entropy state (or any state for that matter) is much more likely to be a high entropy state than a low-entropy one for the exact same reason: the present constrains the past more rigidly than it does the future.

> you don't need any additional assumptions

What you wrote after that is such an assumption. You can't derive asymmetry from nothing, logic or mechanics. The only way to have it, is to assume it in addition to those.

> What you wrote after that is such an assumption.

Not quite. There is an underlying assumption, but that's not it. I can demonstrate the asymmetry between the past and future. For example:

Suppose I flip a fair coin, and I make a video of the coin flip. Let's say the coin comes up heads. I can predict with near 100% certainty (modulo shenanigans) that if I watch the video I will see a coin being flipped and coming up heads. By way of very stark contrast, I cannot predict the outcome of actually flipping the coin again with better than 50% odds.

I claim that this asymmetry in my prophetic abilities (100% ability to predict what will be seen on the video vs 50% for re-flipping the actual coin) is a result of the asymmetry between the past and future.

Note that this is not a circular argument; this demonstration would work equally well in a universe where it is possible to unscramble an egg. All it requires is for there be some experiment whose outcome I can record but not predict, even with arbitrarily advanced technology. The existence of such an experiment is an assumption, but it is a very weak assumption, much weaker than the second law itself. Indeed, it is a necessary assumption for doing thermodynamics at all, indeed, for doing anything. If I can reliably predict the outcome of any experiment, that means I know everything that is going to happen before it happens, including my own actions.

So yes, I have to assume that I cannot be omniscient. Or maybe a better word here would be "hope" because being omniscient would render my entire existence meaningless.

> I claim that this asymmetry in my prophetic abilities (100% ability to predict what will be seen on the video vs 50% for re-flipping the actual coin) is a result of the asymmetry between the past and future.

No, this and other similar memory-induced asymmetries have much more prosaic reason. It is more successful to predict systems we have enough knowledge and control of (video playback) than those we don't (coin flip).

Predicting and retrodicting systems we understand and have memory of is successful, and predicting and retrodicting systems we don't understand and don't have memory of is not. This has nothing to do with universal asymmetry of past vs future.

Your memory of what has happened (e.g. some system had lower entropy in past) does not explain why your original argument does not apply now to retrodiction of past macrostates; the memory only contradicts the result of that argument.

> If I can reliably predict the outcome of any experiment, that means I know everything that is going to happen before it happens, including my own actions.

That would in no way prevent validity of 2nd law.

> It is more successful to predict systems we have enough knowledge and control of (video playback) than those we don't (coin flip).

Nope. Consider this variation on the experiment: I flip a coin, I make a video recording, but I do not look at the coin, I only reveal it to the video camera.

Now I cannot predict the result of watching the video despite the fact that my "knowledge and control" of my video recorder is exactly the same as before.

And, BTW, my ability to make a reliable prediction in the first case is in no way contingent on my knowledge of how video recorders work, nor is it contingent on my having control of the camera. I could be completely ignorant of the inner workings of video recorders, and the camera could be completely outside of my control, and I could still make an accurate prediction, but only if I saw the coin after if was flipped.

> That would in no way prevent validity of 2nd law.

I never said otherwise, but that is neither here nor there.

When you do not look at the coin result that gets recorded, your knowledge of the state of the recorder (the record inside) is not "exactly the same as before". You lack information on the state of the recorder that you would have if you looked.

Yes, knowledge of details of how the recorder works is indeed largely irrelevant, but that was obvious. The point was that being successful with predictions or retrodictions of some system is correlated with having enough information on its state and the laws it obeys. This difference of success for predicting playback vs predicting result of a future experiment sensitive to unknown conditions is due to sufficient information vs insufficient information. I don't see anything in your responses that shows some universal asymmetry between past and future that would explain why your original argument does not work for past macrostates, but does work for future ones.

> You lack information on the state of the recorder that you would have if you looked.

No, I lack information about the state of the coin. All of my observations and knowledge of the operation of the recorder are exactly the same as before. I can still see the camera, see that it is pointing at the location of the coin (even though the coin itself is hidden from me). Nothing has changed in that regard.

> I don't see anything in your responses that shows some universal asymmetry between past and future

Maybe it would be clearer if I change the scenario slightly: instead of one camera, I'll use N>>1 cameras, all pointing at the same coin. I flip the coin in view of all the N cameras, but hidden from my view, and then I put the coin in my pocket without ever looking at it.

At this point I can make no reliable predictions. But if I wach the playback on one of the N cameras, I can then reliably predict the outcome of viewing the playback on any and all of the other cameras. But under no circumstances can I ever accurately predict the outcome of flipping the coin again.

Why? Because there is a fundamental asymmetry between the past and future. It is possible to make reliable records of the past, and if you make multiple records of the same event, all of those records will agree on what the outcome was (that's what "reliable record" means). By way of very stark contrast, it is not possible to make reliable records of the future.

> No, I lack information about the state of the coin.

And that implies you lack information on the record, which is decisive for predicting successfully its playback.

> But under no circumstances can I ever accurately predict the outcome of flipping the coin again.

Because you lack the necessary information, because you assume those experiments are that way. If you set them up so that you don't look at the coin flip result, but in the next coin flip you measure the initial conditions and the flipping process accurately enough, the conclusion would be the opposite - you would accurately predict the new coin flip, and would not be able to predict the playback.

> Why? Because there is a fundamental asymmetry between the past and future.

Not at all. The asymmetry is due to how you defined constraints on the experiments.

> It is possible to make reliable records of the past, ... it is not possible to make reliable records of the future.

Record is, by definition, a pattern that has been created in the past, and this is likely to correlate more with the past states than with the future states. This is a property of the concept, not of some universal asymmetry between past and future. We may introduce analogous concept of a future record, which will be created in the future, and which will correlate with the future states.

We can talk instead about prediction and retrodiction. And then this is more of an easy/hard thing, not possible/impossible thing, and the distribution of success on this axis depends on details. Predicting Moon's position on sky 2 years into future is easier than retrodicting it 20 000 years into past.

None of this explains why your original argument does not apply to past macrostates.

> None of this explains why your original argument does not apply to past macrostates.

Yeah, that's gonna be a long row to hoe if I can't even get you to acknowledge that there's a difference between the past and the future independent of the second law itself.

Maybe I should just ask: do you think there is any difference between the past and future (independent of the 2nd law)? If so, what is it?

But in the meantime:

> that implies you lack information on the record

This turns on what it means to have information "about" something. All of the data I have first hand access to comes to me through my senses, mainly my eyesight. When I say a coin is heads-up it's because I can see the coin, i.e. there are photons emitted from the coin (presumably) that end up on my retina and create electrical impulses that go to my brain and leave me with the subjective sensation of seeing a heads-up coin. There are all kinds of background assumptions about how light works, how coins work, how my retina works, and how my brain works, but for the most part one chooses to sweep those under the rug and accept as a working hypothesis that the reason I have the subjective sensation of seeing a heads-up coin is that there is in point of actual fact a coin "out there" and that it is, again in point of actual fact, heads-up.

But the camera is a very different story. I get direct input from the camera in the same way that I get direct input from the coin. I can see the camera just as I can see the coin. But unlike the coin, the camera's state with respect to what it has recorded is not directly apparent to me. That state is part of the internal state of the camera. It is hidden from me. I cannot directly observe it simply by looking passively at the camera the way I can with the coin while my visual cortex of my brain does the heavy lifting. I cannot even tell that there is hidden internal state based on anything that I can see about the camera while it is recording.

Making a prediction about what will happen during "playback" (indeed, even predicting that there will be something interesting to observe at all!) requires a much more complicated model of the world and chain of inference than it took for me to conclude that the coin was heads-up.

I introduced multiple cameras so that the structure of the example eliminated all that complexity and everything was symmetric. All the cameras are the same, and my knowledge of them is all the same -- until I start looking at the playback, at which point things start to diverge. When I look at the playback on one camera, I now have direct information about the (previously hidden) state of that camera, but not of any of the others.

So here again the question is: why should (direct) information about the state of one camera tell me anything about the state of any of the other cameras? It is because of a very complex chain of inference, part of which includes the hypothesis that there are, in point of actual fact, events which have occurred in the past (and which now cannot be changed) and that there are things in the present (like cameras) whose states correspond to those past events, and for that reason correlate with each other in the present.

I've lived a long time and I've engaged in a lot of philosophical discussions, but you are the first person with whom I've ever had to go into that level of detail on such a foundational matter. Not that there's anything wrong with that, you're keeping me on my toes. But my point is just that if you're going to insist that I be excruciatingly precise about something this elementary, it's going to be a very long time before we can come back to talking about thermodynamics.

Is it a fair summary of your rebuttal to say that its not permissible to speak of the probability distribution of past events, because the past just is what it is?

No. Probability is something that arises from incomplete knowledge, not the actual state of affairs (except in quantum mechanics). In a classical universe both the past and future are fully determined by the present. But our knowledge of the present is limited, and that is why we have to introduce probabilities. Because it is possible to have incomplete knowledge of the present (and hence incomplete knowledge of the past), it makes sense to speak of the probability distribution of past events. If I flip a coin but don't look at it, then I can fairly say that the probability of heads (with respect to my knowledge state) is 0.5 despite the fact that the coin definitely, in point of actual fact, either landed heads up or it did not.

Probability is useful also when we have complete knowledge of the state of a model involving many particles (e.g. gas), but can't reliably integrate equations of motions.

This lack of skill is the same towards the future as towards the past, at least in the context of reversible theories of motion like classical mechanics or evolution governed purely by Schroedinger's equation.

Flip a coin, do not look at it, and make a short video of the coin from above when it is at the top position. This video provides us with details of state of the coin.

Assuming a fair coin and a strong enough flip, from this state of things we can't predict the resulting face, and also can't predict the starting face on the thumb. This problem is symmetric towards the future and towards the past.

The same argument holds for knowledge of one microstate of gas at single time instant. Past and future are equally undetermined, and all probabilities are the same towards the future as towards the past.

To get 2nd law, we need to assume the asymmetry in addition, and it's best if it is explicit. E.g a process connecting two equilibrium macrostates, where the two states aren't defined in a symmetric way. For example, as Jaynes does, we define B as the reliable result of autonomous evolution after manual intervention in A. This makes A,B asymmetric, because it is not stated in the assumptions (and usually not considered possible) that also A is the reliable result of autonomous evolution after intervention in B. This asymmetry is what allows us to derive 2nd law.

See e.g. this Jaynes' paper, Section IV The 2nd Law. https://github.com/lawrennd/jaynes/blob/main/gibbs-vs-boltzm...

It is interesting that this derives asymmetric 2nd law from symmetric (reversible) underlying microscopic model. The asymmetry comes in exclusively due to the assumption of B being the reliable result of intervention in A.

> when we have complete knowledge of the state of a model involving many particles (e.g. gas)

We cannot have complete knowledge of the state of a gas. We cannot even have complete knowledge of the state of a single particle because the positions and velocities of classical particles are real numbers but our knowledge is constrained to be finite, so there will always be some measurement error, and so chaos theory applies. (And physical reality is actually quantum, so we have the uncertainty principle at work as well.)

> This problem is symmetric towards the future and towards the past.

Only because you've carefully crafted this example to have this property; it is not true in general. You have constrained your knowledge of the present to exclude that which is needed to reliably extrapolate into the past. But it is possible to change your scenario and collect data that allows reliable extrapolation into the past, for example, by looking at the coin before it is flipped and remembering (or writing down) what you saw. But for the future this is not possible. That is the essential asymmetry.

> See e.g. this Jaynes' paper

I have a lot of respect for Jaynes so I'll take a look but it may be a while before I have the time to give it the attention it deserves.

One thing I see right away that he gets right (well, mostly): in section 6 he observes that there is no such thing as "the entropy" of a system, which is correct. Entropy can only be measured relative to a state of incomplete knowledge. But calling it "anthropomorphic" is not right. It's not a function of human ignorance specifically, it's a function of ignorance in general.

> ...it is not true in general.

Agreed. But it is true for your original argument applied to a non-equilibrium state at any single time instant, while not assuming any more on the past or the future states.

If we adopt further assumptions about the past states, but not about the future states, then we agree this problem of prediction/retrodiction is asymmetric between past and future and this may be the reason why the multiplicity argument does not have to result in past entropy being higher. In particular, there may be other facts on record about the past states and that would override/modify the multiplicity argument for the past.

> it is possible to change your scenario and collect data that allows reliable extrapolation into the past, for example, by looking at the coin before it is flipped and remembering (or writing down) what you saw. But for the future this is not possible. That is the essential asymmetry.

Yes, in any realistic scenario involving prediction and retrodiction at any time, we have some data on the past states (in some lucky cases, enough data for having certainty), but never data on the future states (we may have predictions, but those are not of the same quality as data). I agree.

But in addition, you seem to argue that this asymmetry of availability of data is an essential fact to derivations of the 2nd law.

Consider a scenario where we have an isolated system that is a mixture of miscible liquids that exists for ages but we have data on its state only for the last 2 years, and that data says its entropy was steadily rising the whole time.

Now, given this data, was entropy increasing before the collection of data started, or was it decreasing? For short enough times around the edges, we can extrapolate the entropy curve based on collected data, but for long enough times, we can't. Using the multiplicity argument, including the data, we get that entropy of very distant past states was much larger than entropy of the first recorded state. Which contradicts 2nd law.

Effect of that "essential asymmetry" has a time span that is not universal, but limited to times where collection of data was happening. Outside it vanishes, and using only that data and the multiplicity argument, the distant past becomes as cloudy(highly entropic) as the distant future is.

> it is true for your original argument

No, it isn't. But we need to take this one step at a time. (And, I should also point out, this is not my argument. It's Boltzmann's and Shannon's.)

> further assumptions about the past states

They are not assumptions, they are observations. There is actual evidence that it is possible to remember the past but not the future. (I can't believe I'm actually having to persuade you of that. This is a pretty basic fact about reality.)

> in some lucky cases, enough data for having certainty

Nope, you can never be certain. Even the best measurements have error bounds, and our brains and recording devices have finite storage capacity. Our knowledge of the past can never be perfect. But -- and this is the important part -- our knowledge of the past can be vastly better than our knowledge of the future.

> you seem to argue that this asymmetry of availability of data is an essential fact to derivations of the 2nd law

I don't know if it's essential. I'm just saying that 1) it is a fact supported by evidence, not an assumption and 2) the second law follows from it (plus a few other things). It might be possible to derive the second law without it, I don't know, and I don't care because the actual fact of the matter is that this asymmetry between past and future does exist in point of actual fact. I don't care about hypothetical universes that behave differently from the one I inhabit.

> Using the multiplicity argument

Sorry, what is "the multiplicity argument"?

> Effect of that "essential asymmetry" has a time span that is not universal, but limited to times where collection of data was happening.

No, it doesn't matter whether data is actually being collected. What matters is that it is possible to collect data (about the past) because the past actually happened. Whether or not anyone actually bothered to make a record is irrelevant. If a tree falls in the forest, it actually falls even if no one makes a video of it.

> There is actual evidence that it is possible to remember the past but not the future. (I can't believe I'm actually having to persuade you of that. This is a pretty basic fact about reality.)

You certainly don't have to waste time convincing me of that, but it seems you should try to explain why this general asymmetry of concept pairs record-past, record-future is relevant to your original argument. We're at this for days and you did not write any coherent argument where this evidence (which I don't contest) would play obvious role.

When I talk about some statement as an assumption, this does not mean I contest its factual evidence or validity. I am interested in the structure of your argument, and all assumptions need to be illuminated, even if they are based in factual evidence.

> what is "the multiplicity argument"?

It's your original argument: given the system is evolving without external influences, and we know its macrostate and its entropy at some time, then most likely the future macrostates will be such that they do not have lower entropy, but higher or same, and this is so because there is immensely more microstates corresponding to macrostates of higher entropy (macrostates of high multiplicity) than there is microstates corresponding to macrostates with lower entropy (macrostates of low multiplicity).

I don't see how asymmetry of memories or records made plays role in this argument. I can see how they play role in my example with collecting data for 2years. But then my argument stands: the data becomes more irrelevant the farther into the past we retrodict, and for very distant past it can be discarded, and the multiplicity argument wins, and fails to be consistent with 2nd law.

To me, this is due to fact one cannot derive 2nd law in general, as probable behaviour for all processes. There is as many phase space trajectories increasing entropy as there is decreasing it. One has to select those macroprocesses that we consistently observe to have the same result (Jaynes ' states A -> B) and then one can derived 2nd law for those processes, only.

> Whether or not anyone actually bothered to make a record is irrelevant.

It is relevant in that it changes data based on which predictions (or retrodictions) are made. I'm sure in science, different data means possibly different predictions.

Try to formulate your whole argument (which I understand as being based on the multiplicity argument), including how "it is possible to remember the past" makes its application to distant past which nobody remembers invalid, while keeping its application to distant future valid.

> the data becomes more irrelevant the farther into the past we retrodict, and for very distant past it can be discarded

No. The universe has kept a record of the very distant past. We can directly observe the past all the way back to the cosmic microwave background, which gets us >99% of the way to the big bang.

So yes, I agree with you that it is possible to have a universe where the second law does not apply. It just happens that the universe we live in is not such a universe, not because of any assumptions, but because of what we can directly observe today, and how we can explain those observations in terms of a past that actually existed.

So, we started with your plausible argument why 2nd law holds, then I point out deficiency in it, then we talk past each other, then you say "we can explain those observations in terms of a past that actually existed". In other words, the original argument is not really sufficient, something else is needed. When I push for this to be added and illuminated, you're content with vague remarks on observations of the past. I'm not convinced by your posts at all, there is no convincing argument about illuminating 2nd law (not even in terms of past records) there. Let's agree to disagree.

As you wish. But I think you simply forgot (or maybe chose to ignore) something I said very early on:

"The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery."

> "The reason entropy was lower in the past is that the universe started out in a low-entropy state. The reason for that is still a mystery."

And my answer to this is the same. Unsubstantiated assumption that universe started out, and that its entropy at this start was low. In addition, it is not at all clear how this assumption helps us to explain that an isolated system could not have higher entropy in the past long before we observed it.

> Unsubstantiated assumption that universe started out, and...

You have elided something very important here. I did not say that the universe started out, full stop. I said it started out in a low-entropy state. It is the "in a low entropy state" part that matters, not the "started out" part. If you don't like the idea that the universe "started out" then you can replace this with "the universe was, at some point in the past (specifically, about 13 billion years ago) in a low-entropy state. We don't know how it got there, or what, if anything, happened before that."

And this is not an "unsubstantiated assumption." There is evidence that the universe was in a low-entropy state in the past. In fact, the evidence is overwhelming. Disputing it is akin to disputing that the earth is round.

> it is not at all clear how this assumption helps us to explain that an isolated system could not have higher entropy in the past long before we observed it.

That's because it isn't true. An isolated system can have a high-entropy past, e.g. https://en.wikipedia.org/wiki/Spin_echo

But any system that we actually observe is not isolated, and that is part of the reason why we never observe decreasing entropy despite the fact that decreasing entropy is not only possible, it actually does occur under the right circumstances.

In fact, we can invoke the anthropic principle to explain why we do not observe decreasing entropy: creating a record of the past is not possible in a universe where entropy decreases. There are, by definition, fewer low-entropy states than high-entropy states, and so transitioning from a high entropy state to a low entropy state requires the destruction of information by the pigeonhole principle. And so the reason such universes are not observed is that such universes cannot contain records of the past, and so they cannot contain observers.

That Jaynes paper is very cool. I'm surprised that Wolfram doesn't mention it in his overview of the literature on the second law.

The question of what are the states to count and how is not obvious.

“It cannot be emphasized strongly enough that W is not the measure of the set C of all states […] compatible with the external macroscopic constraints; rather, it is the dimension of the subset of those states that can be realized in the greatest number of ways.” [Entropy and the Time Evolution of Macroscopic Systems, Walter T. Grandy Jr.]

That's interesting, I didn't know that. I always thought that W in k ln(W) was the number of possible states. Thanks for bringing this to my attention!

I wonder, though, does this actually matter to anyone but a physicist? Consider the common just-so story about electromagnetic waves: a changing electric field produces a magnetic field, and a changing magnetic field produces an electric field. Therefore, once you start to wiggle an electron around, the changing field become self-sustaining and you end up with a wave.

This story is wrong because it would lead you to conclude that the E and B fields are 90 degrees out of phase when in fact they are in phase, and understanding why that is requires going into quite a bit more detail. But does that actually matter to anyone other than a physicist or an antenna engineer?

> I always thought that W in k ln(W) was the number of possible states.

That formula - Boltzmann's entropy - is only applicable when all the microstates are equiprobable. The question of what does it mean in general is subtle.

If we have a gas in a container in thermal equilibrium with a heatbath, for example, the energy of the microstates compatible with those constraints is not fixed and their probability depends on the energy. The energy and the pressure fluctuate.

> I wonder, though, does this actually matter to anyone but a physicist?

The people most concerned with the question of entropy on a conceptual level tend to be physicists - and philosophers. https://deepblue.lib.umich.edu/bitstream/handle/2027.42/4341...

Reply on news.ycombinator.com