balls and bag

One of the books with a valued place on the shelves of the CarnotCycle library is Introduction to Statistical Thermodynamics by Malcolm Dole (1903-1990) who had a long and distinguished career at Northwestern Tech, and who incidentally was involved in the operations of the K-25 plant at Oak Ridge, TN. The Dole Effect, which led to carbon replacing oxygen as the reference standard for atomic weights, is named for him.

But back to his book. In the chapter on the math of probability, Professor Dole illustrates the multiplication and addition rules with this sampling problem:

“A bag contains 2 white balls and 3 black balls, all balls being indistinguishable except for their colors. Three balls are drawn from the bag; what is the probability that the third ball is white?”

OK, there are two ways to solve this problem. One way is to list the sequences in which three balls can be drawn ending with a white ball on the third draw. Letting W represent the drawing of a white ball and B represent the drawing of a black ball, we have

W – B – W
B – W – W
B – B – W

Next we apply the multiplication rule to calculate the probabilities of the three sequences

W – B – W
2/5 × 3/4 × 1/2 = 6/60
B – W – W
3/5 × 2/4 × 1/3 = 6/60
B – B – W
3/5 × 2/4 × 2/3 = 12/60

Finally we apply the addition rule to get the answer: 6/60 + 6/60 + 12/60 = 24/60 = 2/5. The probability that the third ball is white is 2/5. Problem solved. Now we can go look at Facebook.

Hey but hang on, you said there were two ways. What’s the other way?

The other way is difficult to put into words but can be described as the illusion of sampling, in the sense of appearing simply to take balls out of the bag. If you read the question again, you will notice that the colors of the first two balls drawn from the bag are not mentioned. Does this matter? Yes it does, because unless this information is supplied, the sampling procedure cannot be said to exclude putting the first and second balls back in the bag before drawing the third.

So the second way is to recognize this and calculate on the basis of sampling with replacement. Under these circumstances, at each draw there are always 2 white balls and 3 black balls in the bag and the probability of drawing a white ball is always 2/5. It’s a lot quicker to get the answer this way.

But wait a minute, the probability calculations above did not involve replacing any balls, so how come the answer turned out as 2/5?

The best way to see this is to visualize the entire set of sequences of drawing all five balls. Using the combinatorial math for permutation with repetition we see there are 5!/2!3! = 10 sequences in total. For simplicity, the diagram below shows only the white balls in each sequence.

 1   2   3   4   5 

What you are looking at here is the probability distribution of the two white balls in the absence of any information concerning the colors of balls drawn from the bag, which according to the principle established above is equivalent to sampling with replacement. Hence there must be four white balls in each column, representing the 4/10 = 2/5 probability of a white ball in that position. This is what the probability calculations given above are actually computing for row 2 (W-B-W), row 5 (B-W-W) and rows 8 and 9 (B-B-W) in column 3.

– – – –

A playing card analogy

Another way to illustrate the illusion of sampling is to replace the bag and the 2 white and 3 black balls with a pack of 5 playing cards of which 2 are red suits and 3 are black suits. Two cards are drawn from the pack and placed face down on the table. The question is now asked, what is the probability that the third card drawn is red? The answer is immediately 2/5 since the suits of the first two cards have not been revealed – under these circumstances all cards have the same 2/5 probability of being red. Dealing cards face down is therefore equivalent to sampling with replacement. It is not until the cards have been looked at that they can be considered by the person(s) doing the looking as a sample taken from the pack.

– – – –

P Mander April 2021

For a physicist, Albert Einstein (1879-1955) took a remarkable interest in physical chemistry. His doctoral thesis, submitted in 1905, was concerned with determining the dimensions of molecules. And his famous paper from the same year on Brownian motion has at its core the molecular-kinetic theory, a cornerstone of physical chemistry. In both these works, and incidentally in his equally famous paper on the photoelectric effect, Einstein is noticeably occupied with the determination of Elementarquanta (fundamental atomic constants), principal among them being the Avogadro number N. Indeed, following the publication of his epoch-making paper on Special Relativity, he went back twice to his thesis, in 1906 and 1911, to revise his estimate of this number. Not only that, but in between these revisits, he published a paper in 1907 describing yet another method of determining N which will constitute the principal content of this blogpost. But first, a few words about Amedeo Avogadro (1776-1856) and the number that is named for him.

– – – –

Avogadro’s hypothesis

In 1811 Amedeo Avogadro, an Italian of noble birth who studied ecclesiastical law before teaching science at a school in Vercelli in northern Italy, sent a paper to the Journal de Physique, de Chemie et d’Histoire naturelle. In it, he said this:

It was the first statement of what became known as Avogadro’s hypothesis, that equal volumes of all gases [at the same temperature and pressure] contain the same number of molecules. In one sentence, Avogadro took science a crucial step forward. He reconciled the atomic theory (1803-1806) of John Dalton and the gas volume studies (1808) of Joseph Gay-Lussac through the inspired idea that the elemental substances in Gay-Lussac’s experiments existed as divisible polyatomic units i.e. molecules and not as single atoms upon which Dalton insisted.

Different gases have different densities, and by relating these densities to the lightest known gas, hydrogen, the concept of molecular weight and the gram-molecule or mole was developed. A mole of any gas has the same volume (22.4 liters at 1 atmosphere pressure and a temperature of 273K) and therefore contains the same number of molecules. So what is this number? The answer is 6.022 x 1023, a fundamental constant fittingly known as the Avogadro number.

– – – –

Einstein’s forgotten idea

Einstein’s preoccupation with determining this number reflected his belief in the atomic view of matter and the validity of molecular-kinetic theory, which was not universally recognized at the turn of the 20th century. By presenting methods for the determination of the Avogadro number through experiments on observable phenomena, Einstein built a case for the real existence of molecules which came to fruition in 1908 with Jean Perrin’s famous work based on Einstein’s Brownian motion paper of 1905.

Perhaps this explains why Einstein’s interim paper of 1907, suggesting yet another route to determining the Avogadro number, seems to have been passed over – the reality of molecules had been demonstrated and there was no need for further proof.

Even so, it would be interesting to know whether any determination of the Avogadro number was ever conducted on the basis of the 1907 paper. And if not, whether anyone might be interested in putting Einstein’s forgotten idea to the test. Below I have posted both the original text of the paper and an English translation.

– – – –

Über die Gültigkeitsgrenze des Satzes vom thermodynamischen Gleichgewicht und über die Möglichkeit einer neuen Bestimmung der Elementarquanta; von A. Einstein

(Annalen der Physik 22 (1907): 569-572)

Der Zustand eines physikalischen Systems sei im Sinne der Thermodynamik bestimmt durch die Parameter λ, µ etc. (z. B. Anzeige eines Thermometers, Lange oder Volumen eines Körpers, Substanzmenge einer gewissen Art in einer Phase. Ist das System mit anderen Systemen nicht in Wechselwirkung, was wir annehmen, so wird nach der Thermodynamik Gleichgewicht bei bestimmten Werten λ0, µ0 etc. der Parameter statthaben, fur welche Werte die Entropie S des Systems ein Maximum ist. Nach der molekularen Theorie der Wärme jedoch ist dies nicht genau, sondern nur angenähert richtig; nach dieser Theorie besitzt der Parameter λ auch bei Temperaturgleichgewicht keinen konstanten Wert, sondern einen unregelmäßig schwankenden, der sich von λ0 allerdings nur äußerst selten beträchtlich entfernt.

Die theoretische Untersuchung des statistischen Gesetzes, welchem diese Schwankungen unterworfen sind, scheint auf den ersten Blick bestimmte Festsetzungen in betreff des anzuwendenden molekularen Bildes zu erfordern. Dies ist jedoch nicht der Fall. Es genügt vielmehr im wesentlichen, die bekannte Boltzmannsche Beziehung anzuwenden, welche die Entropie S mit der statistischen Wahrscheinlichkeit eines Zustandes verbindet. Diese Beziehung lautet bekanntlich

wobei R die Konstante der Gasgleichung und N die Anzahl der Moleküle in einem Grammäquivalent bedeutet.

Wir fassen einen Zustand des Systems ins Auge, in welchem der Parameter λ den von λ0 sehr wenig abweichenden Wert λ0 + ɛ besitzt. Um den Parameter λ auf umkehrbarem Wege vom Werte λ0 zum Werte λ bei konstanter Energie E zu bringen, wird man eine Arbeit A dem System zufuhren und die entsprechende Wärmemenge dem System entziehen müssen. Nach thermodynamischen Beziehungen ist:

oder, da die betrachtete Änderung unendlich klein und ʃ dE = 0 ist:

Andererseits ist aber nach dem Zusammenhang zwischen Entropie und Zustandswahrscheinlichkeit:

Aus den beiden letzten Gleichungen folgt:


Dies Resultat insolviert eine gewisse Ungenauigkeit, indem man ja eigentlich nicht von der Wahrscheinlichkeit eines Zustandes, sondern nur von der Wahrscheinlichkeit eines Zustands-gebietes reden kann. Schreiben wir statt der gefundenen Gleichung

so ist das letztere Gesetz ein exaktes. Die Willkur, welche darin liegt, daß wir das Differential von λ und nicht das Differential irgendeiner Funktion von λ in die Gleichung eingesetzt haben, wird auf unser Resultat nicht von Einfluß sein.

Wir setzen nun λ = λ0 + ɛ und beschränken uns auf den Fall, daß A nach positiven Potenzen von ɛ entwickelbar ist, und daß nur das erste nicht verschwindende Glied dieser Entwickelung zum Werte des Exponenten merklich beiträgt bei solchen Werten von ɛ, fur welche die Exponentialfunktion noch merklich von Null verschieden ist. Wir setzen also A = αɛ2 und erhalten:

Es gilt also in diesem Falle fur die Abweichungen ɛ das Gesetz der zufälligen Fehler. Für den Mittelwert der Arbeit A erhält man den Wert:

Das Quadrat der Schwankung ɛ eines Parameters λ ist also im Mittel so groß, daß die äußere Arbeit A, welche man bei strenger Gültigkeit der Thermodynamik anwenden müßte, um den Parameter λ bei konstanter Energie des Systems von λ0 auf zu verändern, gleich ½RT/N ist (also gleich dem dritten Teil der mittleren kinetischen Energie eines Atoms).

Führt man fur R und N die Zahlenwerte ein, so erhalt man angenähert:

– – – –

Wir wollen nun das gefundene Resultat auf einen kurz geschlossenen Kondensator von der (elektrostatisch gemessenen) Kapazität c anwenden. Ist die  Spannung (elektrostatisch), welche der Kondensator im Mittel infolge der molekularen Unordnung annimmt, so ist

Wir nehmen an, der Kondensator sei ein Luftkondensator und er bestehe aus zwei ineinandergeschobenen Plattensystemen von je 30 Platten. Jede Platte habe von den benachbarten des anderen Systems im Mittel den Abstand 1 mm. Die Größe der Platten sei 100 cm2. Die Kapazität c ist dann ca. 5000. Für gewöhnliche Temperatur erhält man dann

In Volt gemessen erhält man

Denkt man sich die beiden Plattensysteme relativ zueinander beweglich, so daß sie vollständig auseinander geschoben werden können, so kann man erzielen, daß die Kapazität nach dem Auseinanderschieben von der Größenordnung 10 ist.

Nennt man π die Potentialdifferenz, welche durch das Auseinanderschieben aus p entsteht, so hat man

Schließt man also den Kondensator bei zusammengeschobenen Plattensystemen kurz, und schiebt man dann, nachdem die Verbindung unterbrochen ist, die Plattensysteme auseinander, so erhält man zwischen den Plattensystemen Spannungsdifferenzen von der Größenordnung eines halben Millivolt.

Es scheint mir nicht ausgeschlossen zu sein, daß diese Spannungsdifferenzen der Messung zuganglich sind. Falls man nämlich Metallteile elektrisch verbinden und trennen kann, ohne daß hierbei noch andere unregelmäßige Potentialdifferenzen von gleicher Größenordnung wie die soeben berechneten auftreten, so muß man durch Kombination des obigen Plattenkondensators mit einem Multiplikator zum Ziele gelangen können. Es wäre dann ein der Brownschen Bewegung verwandtes Phänomen auf dem Gebiete der Elektrizität gegeben, daß zur Ermittelung der Größe N benutzt werden könnte.

Bern, Dezember 1906.
(Eingegangen 12. Dezember 1906)

– – – –

On the Limit of Validity of the Law of Thermodynamic Equilibrium and on the Possibility of a New Determination of the Elementary Quanta; by A. Einstein

(Annalen der Physik 22 (1907): 569-572)
Dated: Bern, December 1906
Received: 12 December 1906
Published: 5 March 1907

Let the state of a physical system be determined in the thermodynamic sense by parameters λ, µ etc. (e.g., readings of a thermometer, length or volume of a body, amount of a substance of a certain kind in one phase). If, as we assume, the system is not interacting with other systems, then, according to the laws of thermodynamics, equilibrium will occur at particular values λ0, µ0 etc. of the parameters, for which the system’s entropy S is a maximum. However, according to the molecular theory of heat, this is not exactly but only approximately correct; according to this theory, the value of the parameter λ is not constant even at temperature equilibrium, but shows irregular fluctuations, though it is very rarely much different from λ0.

At first glance the theoretical examination of the statistical law that governs these fluctuations would seem to require that certain stipulations regarding the molecular model must be applied. However, this is not the case. Rather, essentially it is sufficient to apply the well-known Boltzmann relation connecting the entropy S with the statistical probability of a state. As we know, this relation is

where R is the constant of the gas equation and N is the number of molecules in one gram-equivalent. We consider a state of the system in which the parameter λ has a value λ0 + ɛ differing very little from λ0. To bring the parameter λ from the value λ0 to the value λ along a reversible path at constant energy E, one will have to supply some work A to the system and to withdraw the corresponding amount of heat. According to thermodynamic relations, we have

or, since the change in question is infinitesimally small and ʃ dE = 0

On the other hand, however, according to the connection between entropy and probability of state, we have

From the last two equations it follows that


The result involves a certain degree of inaccuracy, because in fact one cannot talk about the probability of a state, but only about the probability of a state range. If instead of the equation found we write

then the latter law is exact. The arbitrariness due to our having inserted the differential of λ rather than the differential of some function of λ into the equation will not affect our result.

We now put λ = λ0 + ɛ and restrict ourselves to the case that A can be developed in positive powers of ɛ, and that only the first non-vanishing term of this series contributes noticeably to the value of the exponent at such values of ɛ for which the exponential function is still noticeably different from zero. Thus, we put A = αɛ2 and obtain

Thus, in this case there applies the law of chance errors to the deviations ɛ. For the mean value of the work A one obtains

Hence, the mean value of the square of the fluctuation ɛ of a parameter λ is such that, in order to change the parameter λ from λ0 to at constant energy of the system, the external work A that one would have to apply, if thermodynamics were strictly valid, equals ½ RT/N (i.e., one-third of the mean kinetic energy of one atom).

If one inserts the numerical values for R and N, one obtains approximately

– – – –

We shall now apply the result obtained to a short-circuited condenser of (electrostatically measured) capacitance c. If is the mean (electrostatic) potential difference that the condenser assumes as a result of molecular disorder, then

We assume that the condenser is an air condenser consisting of two interlocking plate systems containing 30 plates each. The average distance between each plate and the adjacent plate of the other system shall be 1 mm. The size of the plates shall be 100 cm2. The capacitance c is then about 5,000. At normal temperature one then obtains

Measured in volts, one obtains

If one imagines that the two plate systems can move relative to one other, so that they can be completely separated, one can get the capacitance to be of order of magnitude 10 after the plates have been moved apart. If π denotes the potential difference resulting from p due to the separation, one obtains

Thus, if the condenser is short-circuited when the plate systems are pushed together, and the plates are pulled apart after the connection has been broken, potential differences of the order of magnitude of one-half millivolt will result between the plate systems.

It does not seem to me out of the question that these potential differences may be accessible to measurement. For if metal parts can be electrically connected and separated without the occurrence of other irregular potential differences of the same order of magnitude as those calculated above, then it must be possible to achieve the goal by combining the above plate condenser with a multiplier. We would then have a phenomenon akin to Brownian motion in the domain of electricity that could be used for the determination of the quantity N.

Bern, December 1906. (Received on 12 December 1906)

– – – –

References to Avogadro’s paper of 1811

D’une manière de déterminer les masses relatives des molécules élémentaires des corps, et les proportions selon lesquelles elles entrent dans ces combinaisons
Journal de Physique, de Chemie et d’Histoire naturelle, 73, 58-76 (1811)

English translation

Bust of Avogadro in Vercelli, Italy.

– – – –

P Mander April 2021

Dmitiri Konovalov (1856-1929) was a Russian chemist who made important contributions to the theory of solutions. He studied the vapor pressure of solutions of liquids in liquids and in 1884 published a book on the subject which gave a scientific foundation to the distillation of solutions and led to the development of industrial distillation processes.

On the subject of partially miscible liquids forming conjugate solutions, Konovalov in 1881 established the following fact: “If two liquid solutions are in equilibrium with each other, their vapor pressures, and the partial pressures of the components in the vapor, are equal.”

J. Willard Gibbs in America had already developed the concept of chemical potential to explain the behavior of coexistent phases in his monumental treatise On the Equilibrium of Heterogeneous Substances (1875-1878). Konovalov was unaware of this work, and independently found a proof on the basis of this astutely reasoned thought experiment:

«Consider Figure 77 shown above. Two liquid layers α and β in coexistent equilibrium are contained in a ring-shaped tube, and above them is vapor. If the pressure of either component in the vapor were greater over α than over β, diffusion of vapor would cause that part lying over β to have a higher partial pressure of the given component than is compatible with equilibrium. Condensation occurs and β is enriched in the specified component. By reason of the changed composition of β however, the equilibrium across the interface of the liquid layers is disturbed and the component deposited by the vapor will pass into the liquid α. The whole process now commences anew and the result is a never-ending circulation of matter round the tube i.e. a perpetual motion, which is impossible. Hence the partial pressures of both components are equal over α and β and therefore also their sum i.e. the total vapor pressure.»

– – – –

Equivalence of vapor pressure and chemical potential

Konovalov showed that the condition of equilibrium in coexistent phases was equality of vapor pressure p for each component. This is consistent with the concept of ‘generalized forces’, a set of intensive variables which drive a thermodynamic system spontaneously from one state to another in the direction of equilibrium. Vapor pressure is one such variable, and chemical potential is another. Hence Gibbs showed that chemical potential μ is a driver of compositional change between coexistent phases and that equilibrium is reached when the chemical potential of each component in each phase is equal. In shorthand the equilibrium position for partially miscible liquids containing components 1 and 2 in coexistent phases α, β and vapor can be stated as:

– – – –

P Mander, July 2020

Peter Mander Author/Editor CarnotCycle blog

To everyone, a big thank you for the quarter of a million visits this blog has received since its inception in 2012. CarnotCycle’s country statistics show that thermodynamics interests many, many people. They come to this blog from all over the world, and they keep coming.

There are now upwards of 80 posts and pages on this blog, covering a sizeable range of topics in thermodynamics and allied disciplines. They are written for enquiring minds, and it is truly gratifying to see so much of the CarnotCycle resource being accessed by so many.

Knowledge is power. And in the case of thermodynamics, that knowledge is especially powerful.

Pascal’s triangle as he drew it in his 1654 book Traité du triangle arithmétique.

I always thought Pascal’s triangle was invented with its origin at the top like this Δ and all the rows ranged below. But when Pascal drew it, he tipped the base of the triangle over so that the other two sides ranged horizontal (Rangs paralleles) and vertical (Rangs perpendiculaires), and numbered the rows and columns as shown. Each number in the array is thus specified by a row-and-column coordinate pair. This turns out to have thermodynamic significance, as we shall see.

– – – –

Think small

Thermodynamics is a big subject but it can equally well be applied to very small systems consisting of just a few atoms. Such systems play by different rules – namely quantum rules – but that’s ok, the rules are known. So let’s imagine that our thermodynamic system is an idealized solid consisting of three atoms, each distinguishable from the others by its unique position in space, and each able to perform simple harmonic oscillations independently of the others.

Harmonic motion is quantized, such that if the energy of the ground state is taken as zero and the energy of the first excited state as ε, then 2ε is the energy of the second excited state, 3ε is the energy of the third excited state, and so on. Suppose that from its thermal surroundings our 3-atom system absorbs one unit of energy ε, sufficient to set one of the atoms oscillating. Clearly, one unit of energy can be distributed among three atoms in three different ways – 100, 010, 001 – or in more compact notation [100|3].

Now let’s consider 2ε of absorbed energy. Our system can do this in two ways, either by promoting one oscillator to its second excited state, or two oscillators to their first excited state. Each of these energy distributions can be achieved in three ways, which we can write [200|3], [110|3]. For 3ε of absorbed energy, there are three distributions: [300|3], [210|6], [111|1].

The distribution of n units of ε (n = 0,1,2,3) among three oscillators (N=3) can be summarized as

n=0:[000|1] = 1 way
n=1:[100|3] = 3 ways
n=2:[200|3],[110|3] = 6 ways
n=3:[300|3],[210|6],[111|1] = 10 ways

Compare this with the distribution among four oscillators (N = 4)

n=0:[0000|1] = 1 way
n=1:[1000|4] = 4 ways
n=2:[2000|4],[1100|6] = 10 ways
n=3:[3000|4],[2100|12],[1110|4] = 20 ways

There is a formula for computing the total number of ways n units of energy can be distributed among N atoms, or to put it another way, the total number of microstates W available to a system of N oscillators with n units of energy

In every case the number is a binomial coefficient, and the numbers generated can be matched to Pascal’s upended triangle by assigning N (1,2,3 …) to the rows and n (0,1,2 …) to the columns as shown below

Here is the connexion between thermodynamics and Pascal’s triangle, which neatly tabulates the total number of microstates available to an idealized solid comprising N atoms with n units of energy, each atom able to perform simple harmonic oscillations independently of the others.

The reason why the first row consists solely of the number 1 is that one atom (N=1) can have only one microstate regardless of the number of energy units it absorbs. It is also to be noted that the rows read the same as the columns due to the property of the binomial coefficient

and that the series of numbers in rows 2, 3, 4, 5 etc are the natural numbers, triangular numbers, tetrahedral numbers, pentatope numbers etc.

– – – –

P Mander October 2020