Reading “Noise — a flaw in human judgment” by Daniel Kahneman, Olivier Sibony, and Cass Sunstein
People like me who work in clinical research are no strangers to noise because we deal with this issue almost daily. However, I still very much enjoy reading “Noise — a flaw in human judgment”  by Daniel Kahneman, Olivier Sibony, and Cass Sunstein . This book is a re-examination of an age old concept from a socio-psychological viewpoint, and the result is a nice complement to the statistical treatment of the subject that I know of.
Statistical treatment of noise
From a statistical point of view, any observed or measured value is composed of two components: a true value and an error. The true value is usually unknown. The error component here does not simply mean mistake, but reflects any difference between people and within people, with the latter including measurement error. The error component can be divided into two sub-components: systematic error and random error.
Systematic error may include biases and confounding factors. For instance, the way we select people for a survey, the method we choose for data analysis, and even the selectivity of data for publication can make our conclusion deviated from the truth. We sometimes call systematic error as bias. And, there are hundreds of types of biases that make our research irreproducible or even wrong.
Random error is a much more challenging issue. The dissection of the error has occupied some of the best statistical minds in the world. One of the most eminent minds is Ronald Aylmer Fisher, a genius who is regarded as the father of modern statistical science. In the 1920s, Fisher invented the analysis of variance that partitions the variation of a measurement into two sources: between-groups and within-group variation. In Fisher’s thinking, the between-groups variation represents signal, and within-group variation represents noise. Fisher further derived the probability distribution of the ratio of signal over noise ((i.e., the F-test) to assess whether the difference in the measurement between groups is due to a underlying mechanism or random error.