Thermodynamics · Chapter 3

Entropy and the Second Law

Film a drop of food coloring spreading through water, play the film backward, and anyone can tell it’s backward. Yet the laws that move each molecule work just as well in reverse. So why does the world have a direction? Here you’ll run a gas backward and watch it un-spread, then spoil the trick by nudging one atom by a billionth. You’ll count arrangements, watch heat flow from hot to cold by pure chance, and play Maxwell’s demon to see why even a clever demon can’t beat the second law.

Atoms

Live numbers

Missions

    What you’re seeing

    The atoms. Each glowing dot is an atom. They push each other away when they get close and bounce off the walls, and that’s all. Their colors only show where each one started (blue near the bottom, pink near the top), so you can follow the mixing. Nothing pulls them to the right. When the divider opens, they spread because there’s room.

    The entropy meter. Split the box into a left and a right half and count the atoms in each. If n of the N atoms are on the left, the number of ways to pick which ones is W = N!/(n!(N−n)!). The meter shows ln W, the entropy in units of Boltzmann’s constant. With everything on the left, W = 1 and the entropy is zero. Half and half is the most arrangements, so the entropy is highest.

    Reverse time. This button flips the velocity of every atom at once. Each atom then retraces its path exactly, like a film run backward, and the crowd un-spreads into the left half. The laws of motion allow it. The entropy really goes down.

    Reverse + nudge. The same flip, but first one atom (circled in pink) is moved by one billionth of the box’s height. The outlines show where the atoms would be without the nudge. Watch the pink line on the graph: the gap between the two runs grows about tenfold with every collision an atom makes, until the two runs have nothing in common.

    The second law

    Heat goes from hot things to cold things, never the other way on its own. A hot pan cools in a cold kitchen; the kitchen never warms the pan above itself. That everyday fact is one form of the second law of thermodynamics. Physicists state it in two classic ways, which turn out to say the same thing:

    Clausius statement

    Heat never flows by itself from a colder body to a hotter one. A refrigerator can move heat from cold to hot, but only with help from outside: work done by its motor.

    Kelvin–Planck statement

    No engine working in a cycle can take heat from a single source and turn all of it into work. Some heat must always be dumped into something colder. (That’s why power plants need cooling towers; see Chapter 4.)

    Entropy statement

    The total entropy of an isolated system never decreases. It grows in every real process and stays the same only in ideal, reversible ones.

    If either of the first two statements could be broken, you could use that to break the other. Both say the same thing as the third: the entropy of the universe goes up.

    ΔS = Q / Tchange in entropy (J/K) = heat added reversibly (J) ÷ absolute temperature (K)

    This is how Rudolf Clausius defined entropy in the 1860s, from heat and temperature. Adding heat to something cold raises its entropy more than adding the same heat to something hot, because you divide by a smaller T. That one fact explains why heat flows from hot to cold.

    Worked example: heat flowing downhill

    1,000 J of heat leaks from a hot block at 400 K into a cool block at 300 K. Both blocks are big, so their temperatures barely change.

    1. The hot block loses entropy: ΔShot = −1,000 J ÷ 400 K = −2.50 J/K.
    2. The cool block gains entropy: ΔScold = +1,000 J ÷ 300 K = +3.33 J/K.
    3. Total: ΔS = +0.83 J/K. Positive, so this happens.

    Run it the other way, with heat flowing from cold to hot, and the total would be −0.83 J/K. That never happens on its own.

    Counting arrangements

    Ludwig Boltzmann saw what entropy really counts. A macrostate is what you can see: “60 atoms on the left, 60 on the right”. A microstate is the full detail: exactly which atoms are where. Most macrostates can be made in many ways. Entropy measures how many:

    S = k ln Wentropy = Boltzmann’s constant (k = 1.380649 × 10⁻²³ J/K, exact) × natural log of the number of microstates

    Flip 100 coins. There is only one way for all of them to land heads, but about 1029 ways to get 50 heads and 50 tails. Every single arrangement is equally likely, so “about half” wins by a landslide. The chance of all 100 on one particular side is 2−100 ≈ 7.9 × 10−31. Atoms in a box work the same way, with “left” and “right” instead of heads and tails. Try it here:

    Worked example: waiting for a miracle

    Suppose a machine tossed 100 coins a billion times every second, for the whole age of the universe, about 13.8 billion years, or 4.4 × 1017 seconds.

    1. Number of tosses: 109 per second × 4.4 × 1017 s = 4.4 × 1026.
    2. Chance per toss that all 100 land on one chosen side: 2−100 ≈ 7.9 × 10−31.
    3. Expected number of times it happens: 4.4 × 1026 × 7.9 × 10−31 ≈ 0.0003.

    Probably never. And a teaspoon of air holds about 1020 molecules, not 100. That’s why the second law looks like an iron rule even though, deep down, it is about chances.

    Worked example: a gas doubles its volume

    One mole of gas (N = 6.022 × 1023 molecules) rushes from one flask into two, like the divider opening in the simulation. Each molecule now has twice the room, so the number of arrangements is multiplied by 2 for every molecule: Wafter/Wbefore = 2N.

    ΔS = k ln 2N = N k ln 2 = R ln 2 = 8.314 J/(mol·K) × 0.693 ≈ 5.76 J/K.

    The same answer comes from ΔS = Q/T: to double the volume slowly at constant temperature, a gas must absorb heat Q = nRT ln 2, so ΔS = nR ln 2. Clausius’s heat-based entropy and Boltzmann’s counting agree.

    Heat flow as probability

    Why does heat flow from hot to cold? Here is the answer from counting, using the simplest model of a solid, the Einstein solid: a set of oscillators (think of atoms vibrating) that each hold energy in whole packets, called quanta. Two solids touch. At every step, one quantum hops from a random atom to another random atom, in either solid. Nothing pushes the energy anywhere. Watch what happens anyway:

    Start with all 100 quanta in solid A. The energy drains into B until each solid has about the same energy per oscillator: about 60 quanta in A’s 300 oscillators and 40 in B’s 200. Then it jiggles around that split forever. There are about 1033 times more ways to share the energy 60–40 than to keep it all in A, so that’s where the system spends its time. In a real object with 1023 atoms, the peak is so sharp that you never see the energy wander back.

    This is what temperature means. Whichever solid gains more arrangements per quantum it receives is the colder one, and energy flows into it. In symbols, 1/T = ΔS/ΔU: the temperature tells you how much entropy a bit of added energy buys. When the two solids reach the same temperature, moving a quantum either way doesn’t change the total number of arrangements, and the flow stops on average. This approach to temperature comes from Thomas Moore and Daniel Schroeder’s 1997 paper and Schroeder’s textbook (see Sources).

    Try this: pick the tiny solids, 3 + 3 oscillators with 6 quanta. There are only 462 ways to arrange everything, and 100 of them have the quanta split 3–3. Press “All energy in A” and watch: the energy comes back to A again and again, about 6% of the time. With 300 + 200 oscillators, the chance of finding all 100 quanta back in A is about 10−20. Small systems break the “law” all the time. Big ones never do.

    The arrow of time

    Newton’s laws don’t care which way time runs. Film two billiard balls colliding, play it backward, and the backward film obeys the same laws. The main simulation proves it: flip every velocity and the gas un-spreads, all the way back. So the laws of motion can’t be where time’s arrow comes from.

    The arrow comes from two things together. First, a starting point with very low entropy: all the atoms on one side. Second, counting: there are vastly more ways to be spread out than bunched up, so almost every path from a bunched start leads to spread out. To un-spread, every atom must move in exactly the right way at exactly the right time. The flip set that up perfectly. The nudge showed how fragile it is: one atom moved by a billionth of the box hits its neighbors at slightly different angles, they hit theirs, and the error multiplies at each collision. This is chaos. After about ten collisions per atom, roughly 3 seconds here, the billionth has grown to the size of the box. In a real gas, no one could ever set every velocity that precisely.

    Try this: let 120 atoms spread for just 1 second, then press “Reverse + nudge”. They probably still get home, because there was no time for the error to grow. Then try 10 seconds. The longer the history, the more impossible it is to undo.

    For the whole universe, the low-entropy start was the early universe itself. Every egg that breaks, every cup of coffee that cools, and every memory you make runs downhill from that.

    Maxwell’s demon

    In a letter to his friend Peter Guthrie Tait in December 1867, James Clerk Maxwell imagined a tiny being guarding a small door between two boxes of gas. It watches the molecules. When a fast one comes from the left, it opens the door; when a slow one comes from the right, it opens the door. Otherwise the door stays shut. Without doing any work, the being sorts fast from slow, so the right side gets hot and the left side cold: heat flowing from cold to hot, against the second law. William Thomson (Lord Kelvin) later named it a “demon”. Now it’s your turn.

    Information ledger (in units of k)

    Let fast (red) molecules from the left through to the right, and slow (blue) ones from the right through to the left. With patience, you can make one side 40 K hotter than the other. Did you just beat the second law?

    The twist. To decide, the demon has to measure each molecule that comes to the door and remember the result, at least long enough to work the door. That’s at least one bit of information per molecule. A notebook, a brain or a computer memory is finite, so sooner or later the demon must erase old records to make room. In 1961 Rolf Landauer at IBM showed that erasing one bit of information always releases at least

    E ≥ k T ln 2at room temperature, 300 K: 1.380649 × 10⁻²³ J/K × 300 K × 0.693 ≈ 2.9 × 10⁻²¹ J per bit

    of heat into the surroundings, raising their entropy by at least k ln 2 per bit. The ledger keeps score. The sorting lowers the gas’s entropy, but the notebook holds one bit for every molecule that reached the door, and erasing them pays it all back with interest. The total never drops below zero. Charles Bennett showed in 1982 that this erasure is exactly where every demon pays. In 2012, Antoine Bérut and colleagues at the École Normale Supérieure de Lyon in France, working with theorists in Germany, measured it. They stored one bit in which of two laser traps held a tiny silica bead, erased it, and found that the heat released came close to Landauer’s limit when they erased slowly, but never went below it.

    Worked example: the price of forgetting

    A computer at room temperature (300 K) erases a gigabyte, 8 × 109 bits. Landauer’s minimum heat is 8 × 109 × 2.9 × 10−21 J ≈ 2.3 × 10−11 J. Real chips release far more than this minimum, but no chip can ever beat it.

    Common mistake: “Entropy just means disorder”

    “Disorder” is a handy picture, but it can mislead. Entropy counts microstates: the number of ways a system can be arranged at the atomic level while looking the same from outside. A messy bedroom doesn’t have more entropy than a tidy one in any way that matters, since rearranging socks hardly changes the number of ways its 1027 or so atoms can share energy. And some things that look more orderly have more entropy. When a supercooled liquid suddenly freezes into neat crystals, it releases heat, and that heat raises the entropy of the surroundings by more than the crystals lose. Think “number of ways”, not “messiness”.

    Common mistake: “Living things break the second law”

    A seed grows into an oak, putting carbon dioxide and water into the precise order of wood. Doesn’t that lower entropy? Inside the tree, yes. But the second law is about the total, and a tree isn’t isolated. It takes in sunlight, which carries a lot of energy and little entropy, and it gives off the same energy as heat at a much lower temperature. By ΔS = Q/T, heat released at a low temperature carries more entropy. Every living thing, you included, keeps its own order by exporting more entropy than it saves: you take in food and give off heat and waste. The total always goes up.

    A little history

    The word. Rudolf Clausius coined “entropy” in 1865, from the Greek τροπή (tropē), “a turning” or “transformation”. He wanted a word that sounded like “energy” because the two ideas are so closely linked. In his words, he proposed to call S the entropy of a body, after the Greek word for transformation.

    Boltzmann’s counting. In 1877 Ludwig Boltzmann connected entropy with the number of ways a gas can be arranged. He died in 1906 and is buried in Vienna’s Central Cemetery, where his tombstone carries the formula S = k log W (here “log” means the natural logarithm, ln). The neat short form, with the constant k, was actually written first by Max Planck in 1900–1901; Planck also named k after Boltzmann.

    The demon. Maxwell described the demon in an 1867 letter to Tait and later in his book Theory of Heat. Leo Szilard argued in 1929 that measuring costs entropy. Landauer (1961) and Bennett (1982) showed the real cost is in erasing memory, and Bérut and colleagues measured it in 2012.

    Reversing a gas. The trick this page uses, stepping atoms on a whole-number grid so the computer can undo every step exactly, comes from Dominique Levesque and Loup Verlet (1993). Verlet is the same physicist whose name is on the Verlet method of 1967, which molecular simulations have used ever since.

    Where you see it

    • Milk in coffee mixes and never unmixes. So does the gas in Gas in a Box: try its “Two gases” scene and pull out the divider.
    • A refrigerator moves heat from cold to hot, allowed by the Clausius statement only because a motor does work, and the back of the fridge dumps more heat into the kitchen than it removed from the food. That’s Chapter 4.
    • Car engines and power plants must throw away heat. A car’s radiator and tailpipe and a power plant’s cooling tower are the second law made visible. See How Cars Work: The Engine.
    • Computers warm up partly because erasing and overwriting bits has a minimum heat cost, though today’s chips are still far above Landauer’s limit.
    • Ice in a drink melts because heat flows from the warm drink into the cold ice, raising the total entropy. The reverse, a drink that warms up while its ice grows, is never seen.

    How the simulation works

    The gas. The atoms push each other with a smooth, soft force: the energy of a pair is U = ε(1 − r²/d²)³ when they are closer than d, and zero otherwise. The walls push the same way. Each time step uses the leapfrog (velocity Verlet) method: change each velocity by the force, then move each atom by its velocity. This method is time-reversible: running a step with flipped velocities undoes it.

    The whole-number trick. On a computer, ordinary decimals get rounded at every step, and those rounding errors (about 1 part in 1016) act like tiny nudges. Chaos grows them. In our tests with ordinary decimals, a gas run for 1,000 steps came back fine, but after 3,000 steps only 74 of 120 atoms found their way home. So here, as Levesque and Verlet suggested, positions live on a grid 109 steps tall, velocities and forces are rounded to whole grid steps, and every pair force is rounded once and applied equal and opposite. Rounding depends only on positions, so a reversed run makes exactly the same roundings, and the reversal is perfect, bit for bit.

    What’s simplified. It’s a two-dimensional gas of soft disks, not real atoms; the entropy meter counts only left versus right, not positions and speeds in full detail. The Einstein solids are tiny, and their quanta move by a simple random rule. The demon’s molecules don’t collide with each other, and each side’s temperature and entropy are computed as if that side had settled down. Every molecule that reaches the door costs the demon one bit, whether it lets the molecule through or not.

    How it was checked (Node.js tests, run before the page was built):

    • Exact reversal: 120 atoms run for 2,000, 10,000 and 40,000 steps, reversed and run back: every position and velocity returned to its starting value with an error of exactly 0 grid steps, and all 120 atoms were back on the left.
    • Nudged reversal (5 random starts each): after 300 steps of spreading, all came back; after 2,000 steps, 67 to 119 of 120 came back; after 4,000 steps, only 62 to 68 did. The gap from the perfect run grew from 10−9 to 10−1 of the box in about 1,000 steps (2.8 s of movie time), while each atom made about 9 collisions.
    • Energy: over 20,000 steps the total energy stayed within 0.6% (rounding to whole numbers adds tiny noise); the forces between atoms are exactly equal and opposite, so they never change the total momentum.
    • Equilibrium: averaged over 60,000 steps, 61.0 of 120 atoms were on the left, with a spread of 4.9 (independent atoms would give 5.5).
    • Counting: 2−100 = 7.889 × 10−31; W(50 of 100) = 1.009 × 1029; the probabilities add to 1 within 10−12.
    • Einstein solids: for 3 + 3 oscillators and 6 quanta, 462 arrangements in all, 100 of them with a 3–3 split; 400,000 random moves gave probabilities within 0.001 of the exact ones. For 300 + 200 and 100 quanta, the most likely qA is 60, Ω(60)/Ω(0) ≈ 1033.4, and the long-run average was 59.98 against an exact 60.00, matching Moore and Schroeder.
    • Demon: starting with both sides at exactly 300 K, a robot demon that sorts every molecule made 106 K and 514 K, lowering the gas entropy by 11.6 k while recording 349 bits (242 k). A greedy demon that opens the door only when that lowers the entropy squeezed every molecule into one side, saving 27.7 k, but recorded 558 bits (387 k). The total never fell below zero in any test.

    Check yourself

    1. Ten coins are tossed. What is the chance that all ten land on the left side of the table?

      Show the answer

      Each coin has a chance of ½, so ten together have (½)10 = 1/1,024. Out of the 1,024 equally likely arrangements, only one has all ten on the left.

    2. In the simulation, the gas un-spreads after “Reverse time”, but not after “Reverse + nudge”. Why?

      Show the answer

      The motion is chaotic. Each collision magnifies a small change in angle, so the nudged run drifts away from the perfect reversal exponentially fast. Newton’s laws work fine backward; the nudged run just never finds the one special path home.

    3. 600 J of heat flows from a body at 600 K into a body at 300 K. What is the total entropy change?

      Show the answer

      The hot body loses 600/600 = 1 J/K; the cold body gains 600/300 = 2 J/K. Total: +1 J/K, so this flow happens on its own.

    4. Two Einstein solids touch. Why does energy flow from the one with more quanta per oscillator to the other?

      Show the answer

      No force is involved: each quantum hops at random. There are just overwhelmingly more arrangements near the even split, so that’s where the random wandering ends up.

    5. What finally saves the second law from Maxwell’s demon?

      Show the answer

      Landauer’s principle: forgetting has a minimum cost. The entropy the demon must dump into its surroundings when it clears its notebook is always at least as large as the entropy it removed from the gas.

    6. A growing plant builds ordered structures from simple molecules. How does that fit with the second law?

      Show the answer

      The second law is about the total. A plant uses sunlight and gives off heat at a low temperature, exporting more entropy than it saves.

    Sources

    Saved on this device only.
    @teddyrookbook GitHub