How should statisticians teach Pearson’s legacy?


Karl Pearson (1857-1936) was a brilliant mathematician whose contribution to modern statistics cannot be overstated. To him we owe chi-square, the Pearson product-moment correlation coefficient, contingency coefficient, coefficient of skewness, kurtosis, regression to the mean, and numerous other important methods. A student of statistics today cannot avoid citing Pearson, his name being ubiquitous within the turn-of-century statistical revolution of Pearson, Spearman and Fisher.

As David Sheskin (2011: 69) comments,

Along with Sir Ronald Fisher, Pearson is probably viewed as having made the greatest contributions to what today is considered the basis of modern statistics.

But this titan of statistics was also a racist, whose racism permeated his chosen scientific discipline: eugenics. This was not an accident, as a recent inquiry at the institution that employed him, University College London (UCL), has revealed. Financially supported by another eugenicist and man of means, Sir Francis Galton, Pearson ran his own laboratory at UCL. Galton (1822-1911) is credited with the introduction of regression and correlation, upon which Pearson built.

A recent investigation of Galton and Pearson’s papers, archived at UCL, reveal something of Pearson’s politics. In April 1934, at a dinner held in his honour at UCL, Pearson said this about his sponsor.

… The climax culminated in Galton’s preaching of Eugenics, and his foundation of the Eugenics Professorship. Did I say “culminate”? No, that lies in the future, perhaps with Reichskanzler Hitler and his proposals to regenerate the German people. In Germany a vast experiment is in hand, and some of you may live to see its results. If it fails it will not be for want of enthusiasm, but rather because the Germans are only just starting the study of mathematical statistics in the modern sense (Pearson, 1934, quoted in the Final Report of the Inquiry into the History of Eugenics at UCL, 2019).

Galton and Pearson were by no means alone. Spearman and Fisher, building on Pearson’s legacy while disputing parts of it, were also integrated into the eugenics project. Thus Spearman’s attempt to factor out ‘innate intelligence’ g was not separable from his work in factor analysis.

Indeed, the concept of intrinsic pre-given social worth was a central obsession of eugenicists. This was the metric they would seek to maximise. In the preface to the second, 1892, edition of Hereditary Genius (1869), Galton clarifies the innate character of his concept:

The fault in the volume that I chiefly regret is the choice of its title of Hereditary Genius, but it cannot be remedied now. There was not the slightest intention on my part to use the word genius in any technical sense, but merely as expressing an ability that was exceptionally high, and at the same time inborn… Hereditary Genius therefore seemed to be a more expressive and just title than Hereditary Ability, for ability does not exclude the effects of education, which genius does (Galton: 1892, viii-ix).

Eugenicists were also obsessed with reproduction. For them, ‘genius’ was an explanandum of social class. But the upper classes tended to have a lower rate of reproduction than the ‘feckless’ lower classes. Left unchecked this would lead to the dilution of this intrinsic ‘genius’ and lead to the decline of civilisation:

…the language of evolutionary writers has endowed the word ‘success’ with a biological meaning, by bestowing it upon those individuals or societies, who, by their superior capacity for survival and reproduction, are progressively replacing their competitors as living inhabitants of the earth. It is this meaning that is conveyed by the expression ‘success in the struggle for existence’ and this, it will probably be conceded, is the true nature of biological success. To social man, however, success in human endeavour is inseparable from the maintenance or attainment of social status; wherever, then, the socially lower occupations are the more fertile, we must face the paradox that the biologically successful members of our society are to be found principally among its social failures, and equally that classes of persons who are prosperous and socially successful are, on the whole, the biological failures, the unfit of the struggle for existence, doomed more or less speedily, according to their social distinction, to be eradicated from the human stock (Fisher, 1930: 221).

The uncomfortable truth is that foundational work in modern statistics — work that we still use today — was developed as part of a project aimed to ‘prove’ the superiority of white over black; of European over Asian; of colonial overlords over their subjects; of slave-owners over slaves. Biology was destiny, and the hierarchies of the British Empire were genetically determined. In domestic policy, eugenics set out to prove the natural order within Britain: the rich were rich because they were more deserving of wealth, more intelligent, etc. than the poor.

Eugenics can be thought of as having two components: an analytic component of biological determinism, the premise that social outcomes are determined by genes; and a policy component of biological intervention, prioritising resources and even lives:

Summary. A redistribution of births, apart from the reason for which it is propounded, or the means by which it may be brought about, would be attended by economic advantages. A moderate social promotion of fertility, such as should maintain a favourable birth-rate, is not incompatible with the economic organization of our own civilization, and would provide a means of combating the current tendency to a decrease of population (Fisher, 1930: 265).

The sexism is implied. And if biology is destiny, woe betide anyone born with a disability.

Eugenics shaped government policy, domestic as well as foreign. In 1909, Sir Cyril Burt attempted to prove the heritability of innate intelligence in a study of a group of boys, 43 from two schools in Oxford, 13 were tradesmen’s sons from an elementary school and 13 from an upper-class preparatory (‘prep’) school. He set them tests of ‘mental functions of varying degrees of complexity’ (later termed ‘IQ tests’). Burt reported that the upper-class boys did better than those further down the social scale.

But did he attribute the difference to environment or inheritance? He dismissed social effects. Unless the child was starving and unable to pay the elementary school fees of 9d. a week, there was no reason to suspect social differences. In other words, despite three distinct schooling systems being applied, with different curricula, to three groups of subjects, Burt rejected the possibility that the schooling system had any effect on the scoring of students on his tests! Such an alternate hypothesis would be clearly devastating to his case. But it would also imply that government policy should prioritise resources to schools with the most deprived children. It should imply a policy opposed to selection at 11, whereas Burt’s later, fraudulent, twin studies would be used to promote the eleven-plus.

In the aftermath of the war eugenics seemed discredited politically by the Nazi Holocaust, but eugenical ideas have surfaced periodically in psychology and medicine. Thus by 1971, biological determinism was on its way back, and with it, scientific racism:

Nature has colour-coded groups of individuals so that statistically reliable predictions of their adaptability to intellectually rewarding and effective lives can easily be made and profitably be used by the pragmatic man in the street (William Shockley, 1971, quoted in Rose (1976)).

One of the allegations against the British Government’s handling of the Covid-19 pandemic concerns whether the government is in reality adopting a ‘herd immunity’ policy, allowing a ‘sufficient’ number of deaths for biology to ‘solve’ the problem of immunisation. Eugenics has never completely disappeared, in part because it provides an alternative narrative for governments wishing to ‘explain’ poverty, etc. as natural and their own inaction in the face of such ‘natural’ outcomes as rational.

Errors of causal presumption

Many examples of eugenics arguments are instances of the causation-correlation fallacy:

the distribution of variable X correlates with variable Y (e.g. a proxy for ‘race’),
therefore X is caused by Y.

To take William Shockley’s example, above, Y = skin colour, X = IQ – note the leap to “adaptability to intellectually rewarding and effective lives” (sic).

Students are often taught that this reasoning is false on Day 1 of a statistics course, but the temptation to slip into this type of reasoning is extremely pervasive.

In simple terms, an observation “X correlates with Y” can equally plausibly be due to

  • X is caused by Y,
  • Y is caused by X,
  • X and Y are caused by Z.
ZX and ZY

The last of these, the “problem of the missing third variable” becomes obvious when unpicking eugenicist claims.

However, this is not the only form in which this error can exist. One way to disguise the argument is to change the language. Students are warned off ‘cause’: use ‘explain’ or ‘predict’ instead, as Shockley does. But the structure of his argument is exactly the same. One can only reliably ‘predict’ like this by presuming that this is the natural order of things; that in different social conditions his ‘prediction’ would hold up. Like the eugenicists of old, Shockley shifts rapidly from ‘analysis’ to policy recommendations.

A second, and often more difficult problem to refute, is to add complexity. Make it look as if you are considering more variables, without considering whether these are the right variables, properly measured. Model-fitting methods are vulnerable to this supposition, lulling the user into a false sense of reliability. Out goes distrustful scientific refutation. In comes an inductive ‘best fit’ to predefined variables with a given set of possible relations between them.

However, if you fit to a particular model you necessarily make mathematical assumptions about the underlying behaviour of the population. And all models are simplifications, and statistical models are no exception.

A good example of alternative ways of thinking about system complexity considers it through the lens of development. By working up from cell biology, Steven Rose (2003) proposed the concept of the ‘lifeline’ of an organism, a pathway of development characterised by a complex interplay between very large numbers of genes producing permutations of proteins at various points in cellular development, and environmental influences (at multiple levels, e.g. of the cell, organ, body, and social group); and — within these levels — the organism’s own impact on that environment. This type of model does not produce a continuous growth curve, but one which contains sudden qualitative leaps forward (‘state changes’) at different levels. 

When set against a sophisticated multi-layered developmental perspective like this, the idea that intelligence might be ‘explained by’ genes and society with different coefficients, as if ‘genes’ and ‘society’ were somehow separable variables, appears superficial and ridiculous.

This leads us to third type of error: a conflation between variation and causation. The concept of heritability (an estimate of the degree to which a phenomenon is inherited) is typically operationalised as the component of variance for phenotypes (that genes ‘explain’ a proportion of the variance of a manifested trait). But this approach to analysis has been comprehensively challenged.

It is well worth reading Richard Lewontin’s essay first published in 1974. He points out that the way genetic and environmental effects vary is rarely additive, such that it makes sense to employ additive models of variance. This finding is simply devastating for the eugenics project.

This fallacy is that a knowledge of the heritability of some trait in a population provides an index of the efficacy of environmental or clinical intervention in altering the trait either in individuals or the population as a whole… A trait can have a heritability of 1.0 in a population at some time, yet this could be completely altered in the future by a simple environmental change. If this were not the case ‘inborn errors of metabolism’ would be forever incurable, which is patently untrue (Lewontin 2006).

As Rose points out, the organism in the environment is a complex system in which the combination of genes and environment play a role at certain key points in its development. Attempting to plot the variance of mean outcomes is unlikely to penetrate the process that brought them into being, and rapidly becomes a scientific dead-end.

Linguistics may not be so explicitly vulnerable to the same methodological errors, but we should be aware of them if only because they draw attention to a key analytical distinction: the analysis of variance is not the same as the analysis of causes. We should be extremely cautious in ascribing mechanisms to explain variation that we can plot.

Pull down the statues!

Photos courtesy of UCL Estates, 2020.

I am writing this post on hearing that UCL has finally ‘denamed’ buildings and lecture theatres that were named after Pearson and Galton. Interestingly the naming of some of these was not ancient, but recent (the Galton Lecture Theatre, in 2000). There has been a lengthy debate at UCL about whether or not these names should be removed.

For what it’s worth, my view is simple: no-one can, or should, try to take away Pearson’s intellectual achievements. We should not rewrite textbooks to deny him credit for his contributions. But that is a very different proposition as to whether UCL, or any other institution, should honour him.

Honouring an academic as a ‘Great Man’ — with the exception of one woman, all UCL buildings are named after men — is a distinct proposition from crediting relevant works (citation). Arguments can be challenged, but statues and monuments are conventionally intended to be impermeable.

Indeed the sceptic rationalist position is that no university buildings, rooms or lecture theatres should be named after forebears, at least not permanently.

How should we treat Pearson’s legacy?

Science proceeds by intellectual critique as well as experiment. Engaging in critique is not ‘rewriting history’; it is a necessary component of scientific progress.

Physics education teaches the Aristotelian view of the universe, not merely to repudiate it, but to explain why this view appeared to be intuitively correct until it was overturned by the Copernican revolution. The more that students are aware of the historical nature of the scientific project, the more open they are likely to be to critically engage with their own assumptions.

It seems to me therefore that the best way to engage with this history, is not to shy away from it, but confront it. One of the themes of this blog is that statistical methods such as tests or regression approaches are often taught in ways that obscure, or fail to properly explain, the underlying mathematical model. Questioning assumptions about these models is essential, but to do this we need to go back to basics.

We should not reject chi-square or multivariate regression simply because methods are capable of misuse! Rather we should teach this difficult past, and the nature of the logical errors that were involved in historic claims, in part as a way of inoculating students against the same mistakes in other disciplines. All students who engage in statistical analysis should be equipped to spot errors in academic papers, and to be self-critical of their own analysis. If students are merely copying a pre-given approved method and substituting their data, they are not equipped to do so.

This does mean being honest about eugenics, and the impact that it has had, in psychology and biology in particular. Eugenics is a warning from our scientific history: it shows that ‘statistical’ arguments purporting to scientific rigour, resting on dubious assumptions, can be used to present structures of social domination — chiefly racism but also sexism and prejudice against disabled people — as natural, and population control as necessary.

In the era of Covid-19, many of these arguments are reappearing in state intervention strategies or gene-vs.-environment explanations of excessive BAME deaths.

The debates are as relevant today as they ever were.


Ball, N. (n.d.). Galton, Sir Francis. Eugenics Archive. » ePublished

Fisher, R.A. (1930). The Genetical Theory of Natural Selection, Oxford: Clarendon. » Archived

Galton (1892). Hereditary Genius. (2nd Ed.) London: Macmillan. » ePublished

Gould, S.J. (1981). The Mismeasure of Man, New York: Norton.

Lewontin, R.C. (1991). All in the Genes? In: R.C. Lewontin, The Doctrine of DNA. London: Penguin.

Lewontin, R.C. (2006). The Analysis of Variance and the Analysis of Causes.  International Journal of Epidemiology, 35: 3, 520-525, DOI: 10.1093/ije/dyl062

Roige, A. (2014). Intelligence and IQ testing. Eugenics Archive. » ePublished

Rose, S. (1976). Scientific Racism and Ideology: The IQ Racket from Galton to Jensen, in H. Rose and S. Rose (eds.) The Political Economy of Science – Ideology of/in the natural sciences, London: Macmillan.

Rose, S. (2003). Lifelines: Life Beyond the Gene. London: Penguin.

Sheskin, D.J. 2011. Handbook of Parametric and Nonparametric Statistical Procedures. (5th Ed.) Boca Raton, Fl: CRC Press.

See also

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.