The musings of Dominic Cummings – it’s science, but not as we know it!
In an article entitled “Don’t let the schools revolution go unfinished” in The Times of 1 September, Dominic Cummings returns to his attack on just about everyone in education excepting himself and Michael Gove. One passage is striking: “Although we tried to replace GCSEs with exams that would enable a genuinely scientific approach to learning, Nick Clegg and David Cameron stopped us, supported by almost everyone powerful in the system.” Alas, this list of opponents is incomplete, for the Gove/Cummings approach to science is also rejected by two of the giants of 20th century thought: Niels Bohr and Ludwig Wittgenstein.
One presumes that Mr Cummings’ reference to a “genuinely scientific view of learning” can be traced to the Gove/Cummings enthusiasm for randomised control trials (RCTs). Mr Gove has already assigned £135 million of taxpayers’ money to RCTs. Also, one of the most controversial features of these new examinations was that they were to be calibrated against the uppermost PISA rankings, which are generated using “item response theory” (IRT). Here’s the difficulty for Gove and Cummings – the output of both RCTs and IRT modelling is, in scientific terms, little more than nonsense. These two men were taken in by what might be termed “blob science.”
One of the most celebrated debates in the history of science is the so-called Bohr-Einstein debate over “quantum entanglement.” Bohr, who was the victor in the debate, interpreted it as addressing a fundamental question: what is meant when one assigns a property to something? To quote Bohr’s careful language directly from his essay Discussion with Einstein on Epistemological Problems in Atomic Physics: there is a “mutual relationship which will always exist between the practical use of any word and attempts at its strict definition”. Bohr makes clear the implications for psychological (and educational) attributes. All intentional attributes only adopt definite values when measured. To treat unmeasured attributes as definite is incorrect. Both IRT and RCTs offend against this central tenet of measurement. As a consequence, the output from such models is meaningless. Precisely the same arguments appear in the later philosophy of Wittgenstein.
Anyone who examines the mathematics of RCTs will very quickly see the intractable problems which arise when this methodology is applied to issues in education. The interested reader should see R.A. Fisher’s (1935) seminal work, “The Design of Experiments” (see also H.B. Mann’s (1949) “Analysis and Design of Experiments”). It is important to be clear that the critique offered here applies only to the study of educational attributes. This reasoning has no implications for RCTs conducted in medicine.
In view of the cost of RCTs, taxpayers should be alarmed by who suggest that what works in medicine should also work in education. This supposed equivalence seems to inform the writing of individuals like Matthew Syed and Ben Goldacre, and to underpin the Sutton Trust’s thinking on “evaluation.” The contrast between these two disciplines couldn’t be greater. The mathematics of RCTs only works for so-called intrinsic attributes. While blood pressure is an intrinsic attribute, all intentional attributes in education (understanding, learning, remembering, thinking, meaning and so on) are non-intrinsic attributes. Medical attributes such as blood pressure are governed by first-person/third-person symmetry, while educational attributes are first-person/third-person asymmetric. While medical RCTs are concerned with causal relationships, the relationship between mind and behaviour in education is acausal. Probability enters medical RCTs as subjective probability. Probability is objective in respect of educational attributes. RCT modelling deals only with reducible uncertainty while uncertainty in educational attributes is irreducible.
In his “Remarks on the Foundations of Mathematics”, for example, Wittgenstein writes in Part VI, §13 that “understanding is a vague concept.” This vagueness is not a deficiency in the attribute “understanding” but reflects an irreducible uncertainty without which the attribute would cease to function. Simple, static models of IRT and RCT can never accommodate such complex attributes for which uncertainty is not the result of poor instrumentation or human ignorance; rather it’s constitutional. When the focus is on educational attributes, the information which would reduce uncertainty simply doesn’t exist!
Unwittingly, Gove and Cummings have been duped into accepting the approach to science of the much-derided “blob.” Their poor grasp of science and mathematics (see Cummings’ extended essay available online) has left them prey to the naïve thinking which underpins the PISA league table and the application of RCTs to education. According to Mr Cummings, a Whitehall official described him as a “mutant virus” who should be “expelled from the organism.” This fits with a long tradition in science fiction cinema where the hapless lieutenant, who commits himself to defeating the brainwashing blob, in the end, succumbs to its influence, and finds that his former comrades are now out to get him.
Dr Hugh Morrison