Ramnath R Iyer

Excellent Intro to the Meaning of Probability

Revisado: 12-31-22

I have been reading E. T. Jaynes’ “Probability Theory: The Logic of Science”, which presents a fantastic explanation and formal derivation of probability as a system of logic (built on plausibility rather than certainty, unlike predicate logic). What I hadn’t known was the historical context around the Bayesian vs frequentist approaches to probability that made Jaynes’ work such an important masterpiece.

Bernoulli’s Fallacy provides this context, starting with Bernoulli’s contributions to the field, working all the way through the development and use (rather, a perversion) of statistics to meet the eugenics agenda, and finally the present day “crisis of replication” that is plaguing research across a variety of fields due to their reliance on statistical significance and p-values as a measure of evaluating hypotheses.

As such, this book, in its initial chapters, presents its core set of ideas. These are not novel ideas, but they are nevertheless poorly understood by the community today, and this book does a great job explaining them in depth. I would summarize these ideas as follows:

- Probability represents a subjective belief in a hypothesis based on information / knowledge that you possess, it is not an objective fact. Any statement that the probability of an event IS some number is incomplete; you must always state your assumptions (knowledge that you possess). All probability is conditional on these assumptions. (Jaynes does a good job of making this explicit via notation.)
- You cannot draw inferences from data alone. What you CAN do is convert prior probabilities (existing degrees of belief) to posterior probabilities through the act of observation (incorporating new data). Data doesn’t ever tell you the whole story; it can only alter the story you already have in terms of its plausibility.
- Unlikely events happen. You cannot infer the truth or falsity of a hypothesis based on the likelihood of an observation. Rather, you can only use an observation to alter your subjective belief in the plausibility of a hypothesis, and that too, relative to OTHER hypotheses that support the same observation. Again, unlikely events do occur (e.g., someone always wins the lottery), and so it’s really the relative likelihood of different hypotheses that you adjust as you learn more (by making more observations). Of particular importance here is the idea that it is up to YOU (not the data) to exhaustively formulate the relevant hypotheses, and assign suitable priors. As Pierre-Simon Laplace supposedly put it (paraphrasing), “extraordinary claims merit extraordinary evidence”, and so new data should alter your belief one way or the other toward a hypothesis based on the RELATIVE priors associated with all potential hypotheses. The more you believe in a hypothesis relative to others, the harder it should be to displace.

One idea this book clarifies is that Bayesian and frequentist are not two “equally valid” schools of thought, but that the Bayesian method underpins the whole idea of probability, whereas the frequentist approach is simply a special case (a sort of unhappy accident of history).

Overall, a well-argued, interesting, and balanced book, despite the seemingly extraordinary conclusion. The evidence is extraordinary and well-presented, though occasionally repetitive and dense.

Has calificado esta reseña.

Reportaste esta reseña

esto le resultó útil a 4 personas