Energy tells you how difficult it is to put a bunch of particles (atoms, for example) in a given state. Entropy is how many of such states are possible. Ordered states (like crystals) have lower entropies than disordered states (like liquids), because there are fewer ways of arranging atoms and still get a crystal compared to the combination of possible positions for each atom in a liquid. In systems that are isolated (i.e. they cannot exchange energy with anything else), entropy cannot decrease on average. In non-isolated systems, anything goes and entropy can go up or down locally all the time.
We don’t measure it directly. What we can do usually is measure how much something heats when we give it energy, and work from there to deduce any entropy change using equations from thermodynamics or statistical Physics. It is impossible to measure an absolute entropy in general (in the same way as it is impossible to measure an absolute energy in general).
There are lots of caveats, asterisks, and cases that look like exceptions; that’s a quick and dirty description.
Mind if I ask follow-up question, what is a "state"?
Doesn't the entropy change when I change my definition of the state? If I go to the extreme and there is only one kind of state, an actual arrangement of particles, fields etc., the entropy would be the same of each and everything (one possible arrangement per state).
Does that make entropy an entirely subjective measure?
Sure. Actually there are two main things we call states. The first one (macroscopic state, or macrostate) is what we think as characteristic properties of a bit of matter, e.g. "1 kg of liquid water at 300 K under atmospheric pressure" compared to "1 kg of solid water at 250 K under atmospheric pressure".
The second one (microscopic states, or microstate) is the way the particles that constitute this bit of matter are arranged. The naming is a bit unfortunate and it can get technical quite quickly, but the distinction between macroscopic and microscopic states is crucial.
So, a more precise version of my previous post would be something like that. In the liquid example, there are many, many ways of distributing the H2O molecules that would result in the same macroscopic description. This means that there are many microscopic states that are consistent with the macroscopic state we observe. And, looking at a glass of water, we cannot say where the molecules are.
On the other hand, in a perfect ice crystal, the positions of all the atoms constituting al the H2O molecules are uniquely determined by the crystal structure. So, looking at a perfect ice cube we can say where every molecule is. It gets a bit more complicated in reality because no crystal is ever perfect. There are defects that introduce some disorder, so there are more than one microscopic state, but much fewer than in the liquid.
Entropy is larger for things that have more microscopic states consistent with their macroscopic state. You can also see here a hint of the link with information entropy in CS, if you think about the number of microscopic states as our knowledge of the molecules' positions.
> Doesn't the entropy change when I change my definition of the state? If I go to the extreme and there is only one kind of state, an actual arrangement of particles, fields etc., the entropy would be the same of each and everything (one possible arrangement per state).
It does sound very subjective. The conventional naming is a bit unfortunate and a consequence of the historical roots of statistical Physics in 19th-century thermodynamics. After all, there is no clear boundary between macroscopic and microscopic.
If this bothers you, you can say that "macroscopic" refers to the thing you are looking at as a whole, and "microscopic" to its constituents. This framework works as long as the stuff you are studying is made up of smaller things. We commonly use atoms and molecules in examples because it is somewhat intuitive, but you could consider an atom itself as a macroscopic system and its quarks as its constituents. Or the universe and its galaxies clusters.
The caveat is that the mathematical formalism is exact in the limit where the number of constituents is infinite, but might break down if their number is too small. That's why the distinction between macroscopic and microscopic is helpful.
> Does that make entropy an entirely subjective measure?
In a way, a bit. You can define the constituents seemingly arbitrarily (like considering molecules, or atoms as separate entities or not, adding electrons, etc). Adding more details gives a more accurate answer, but at some point it becomes irrelevant. So it is actually less subjective than relative.
To keep (ab)using the water example, a molecule is made up of 3 atoms, each one having a position (3 positions, each one being a 3-dimensional vector, so 9 parameters in total). But when grouped in a molecule these positions are not independent, and need to be consistent with the O-H bond length and the H-O-H angle. A molecule is characterised by a position (3-d vector), an orientation along the axes of the reference frame (3 other parameters), an angle and two bond lengths (again 9 parameters in total). So there is no more information if you describe the ice cube as a collection of atoms than as a collection of molecules, even though the choice seems arbitrary.
Take a deck of cards, say, a pre-ordered one (technically, you know what microstate it's in) - so let's say that its "disorder" is "zero", and your entropy about it is zero.
Now shuffle it face down really well, now both its "disorder" and your entropy about it is pretty high (it has a specific microstate that you don't know, and the ensemble of all potential microstates form a macrostate).
Now look at this shuffled deck face up again - it's "disorder" is still high, but now your entropy about it is zero again, because you know what microstate it's in.
In what details are they different ?
http://www.av8n.com/physics/thermo/entropy.html
EDIT : Perhaps more relevant:
http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel...
EDIT2 : A specific example where both the Thermodynamic entropy and the Shannon entropy can be experimentally seen to be equivalent :
http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag
EDIT3 : Ah, found the chapter discussing this specific point :
http://www.av8n.com/physics/thermo/entropy-more.html#sec-s-i...
The state of each card. If you define a macrostate in such a way as it can have only one microstate, then yes, entropy is 0. It is also completely artificial.
> EDIT : Perhaps more relevant: > http://www.av8n.com/physics/thermo/s-relevance.html#ch-s-rel...
I am not impressed with that website in general, but in this instance I don't see anything wrong with that section. Note in particular:
> Very roughly speaking, the items higher on the list can be assigned to the “information theory” camp, while the items lower on the list can be assigned to the “thermodynamics” camp. However, there is tremendous overlap between the two camps.
There is indeed some overlap (I have personally worked for a couple of years on applying information theory to calculate entropy in glass-forming materials), but not enough that you can just apply random concepts from one field to the other. Basically, your interpretation is that "we know a lot about the system, therefore entropy is low", whilst in a physical system it's the other way around: "entropy is low, therefore we know a lot about it". Knowing something does not change the state of the system you are observing.
> EDIT2 : A specific example where both the Thermodynamic entropy and the Shannon entropy can be experimentally seen to be equivalent : > http://www.av8n.com/physics/thermo/expt-basis.html#sec-demag
Entropy goes to zero when spins align, but it does not mean that entropy is non-zero until we check that the spins are aligned. Entropy itself is whatever it is even before we bother calculating it, and we can estimate it different ways. Sure, we can use Shannon's formula in some cases, which is really just Boltzmann's formula with different units.
Entropy is a thermodynamical property of a bit of stuff, regardless of what we know about it. The conservation of Gibbs free energy does not suddenly break down because we stop (or start) looking.
Obviously, a deck of cards can be in a lot of different microstates, I'm not sure what do you mean ? Consider that "macrostate" is another name for "distribution" ?
> There is indeed some overlap (I have personally worked for a couple of years on applying information theory to calculate entropy in glass-forming materials), but not enough that you can just apply random concepts from one field to the other. Basically, your interpretation is that "we know a lot about the system, therefore entropy is low", whilst in a physical system it's the other way around: "entropy is low, therefore we know a lot about it". Knowing something does not change the state of the system you are observing.
This is why having two different terms "entropy" and "lack of information" that correspond to the same underlying physical concept is problematic : it causes this kind of confusion.
Knowing a lot about a system is equivalent to having low entropy about it.
> Entropy itself is whatever it is even before we bother calculating it, and we can estimate it different ways. [...] Entropy is a thermodynamical property of a bit of stuff, regardless of what we know about it. The conservation of Gibbs free energy does not suddenly break down because we stop (or start) looking.
"Looking" (aka "measurement" or "observation") is not a "free" operation. To go back to the example of the deck of cards, the deck would have needed to be prepared beforehand to be "ordered", and the action of "looking" at it and to check in what order each card is in is not "free" either. In the same way, neither constructing an aligned spin system nor checking its alignment are "free".
As another example, consider the Szilard engine, which allows us to "extract" energy "from" information :
https://www.pnas.org/content/111/38/13786
(Semi-)Classical entropy is defined in terms of probability, it's a functional that depends on a specific probability distribution (aka macrostate) : it's the expected value (aka weighted average) over the surprisals (aka information contents) of all the possible outcomes (aka microstates).
(There's also a generalization of it for the situations where we have entangled microstates and quantum mechanics really come into play.)
As far as we know, this model works for all the situations where the two nuclear forces and/or relativistic effects can be neglected.
http://www.av8n.com/physics/thermo/entropy.html#sec-s-not-kn...
(Yes, it is indeed context-dependent.)