The flaw at the heart of Dweck and Boaler’s research, and the real source of psychology’s “reproducibility” problem
Dr Hugh Morrison (The Queen’s University of Belfast [retired])
Email: drhmorrison@gmail.com
The Times Higher Education (28.04.2016-04.05.2016) reported that Professor Carol Dweck’s “ideas on education have swept through schools. … In a TED talk that has so far garnered more than four million views online, she shares inspiring tales of pupils in tough, inner-city areas who have zoomed ahead after being trained to believe that their talents are not fixed.” The key word in this quotation is the word “believe”; Professor Dweck is particularly concerned with an individual’s beliefs or “mindsets.” In the fixed mindset, “Success is about proving you’re smart or talented. Validating yourself.” In the growth mindset, on the other hand, success is about “stretching yourself to learn something new. Developing yourself.”
It is vital to Dweck’s theory – a theory currently attracting millions of pounds of education funding in the UK – that beliefs are entities in the mind/brain of the individual. Dweck urges teachers to spur their students to success by concentrating on their “mindsets.” On page 16 of her Mindset book she writes: “You have a choice. Mindsets are just beliefs. They’re powerful beliefs, but they’re just something in your mind, and you can change your mind.” Beliefs/mindsets are properties of the individual. Her writing is replete with references to children “putting themselves” into one mindset or the other, or being “placed into” the growth mindset by a teacher or by psychologists working on Dweck’s “Brainology” programme.
At the outset, there are two issues which engender disquiet about Dweck’s theory. First, her writings ignore the extensive literature demonstrating that beliefs are neither mental states, brain states, nor dispositions (see, for example, Peter Hacker’s 2013 book, The intellectual powers: a study of human nature). Second, and much more importantly, Dweck treats “intelligence” and “ability” as things-in-themselves, to borrow a Kantian conceit. All of Dweck’s writing assumes that the individual’s intelligence exists independent of the efforts of others to learn about it. To borrow a phrase from Crispin Wright (2001), intelligence is an “investigation-independent” entity for Dweck. At the very core of Dweck’s research is the child’s belief that his or her intelligence is either something fixed, or something capable of improvement. Jo Boaler’s research – which applies Dweck’s thinking to the mathematics classroom – similarly views mathematical ability as a thing-in-itself.
To prepare the reader for what follows, consider the following simple thought experiment which suggests that it is meaningless to refer to mathematical ability as a thing-in-itself, existing independent of the teacher’s efforts to measure it. In the UK children who find mathematics a struggle take the “Foundation GCSE” mathematics examination at age 16. The test items on the Foundation mathematics paper aren’t considered mathematically demanding. Now suppose that Richard, a pupil who has completed the Foundation mathematics curriculum, produces a perfect score in the examination. It seems sensible to conjecture that if Einstein were alive, he too would produce a perfect score on this mathematics paper. Since we think of the examination paper as measuring mathematical ability, are we therefore justified in saying that Einstein and Richard have the same mathematical ability?
Paradox results when one erroneously treats mathematical ability as something investigation-independent, entirely divorced from the circumstances of its ascription. It is clear that Richard’s mathematical achievements are dwarfed by Einstein’s. To avoid the paradox one would have to mention the measurement circumstances and say: “Richard and Einstein have the same mathematical ability relative to the Foundation mathematics paper.” When one factors in the measuring instrument, the paradox dissolves away. This thought experiment illustrates how the reasoning which underpins the research of Dweck and Boaler can be undermined since neither researcher makes mention of the measurement context.
Mathematical ability is not a property of the person. Rather it is a joint property of the person and the relevant measuring tool. In short, mathematical ability should be thought of as a property of an interaction. It is a relational attribute rather than an intrinsic property of the person being measured. Definitive statements about mathematical ability – such as “my mathematical ability is fixed” – are difficult to justify. It is only by specifying the measuring instrument in the Foundation mathematics thought experiment that one “communicates unambiguously,” to borrow the quantum physicist Niels Bohr’s words.
It would be unfair to judge Dweck’s ideas by focusing exclusively on her book Mindset, which is clearly intended for the “popular psychology” bookshelves. Dweck’s 2000 book on “self-theories” is published by Psychology Press. It focuses on beliefs about intelligence and is directed at the psychological community. Dweck (2000, p. xi) sets out her programme as follows: “In this book I spell out how people’s beliefs about themselves (their self-theories) can create different psychological worlds, leading them to think, feel and act differently in identical situations.” “Entity theorists” are individuals who believe that their intelligence is fixed, while “incremental theorists” believe that their intelligence is capable of improvement. The former are focused on grades while learning preoccupies the latter. (The parallels with the fixed and growth mindsets are obvious.)
The items which make up the questionnaires Dweck uses to measure children’s self-theories give the clear impression that she takes two things for granted: (i) that intelligence is an intrinsic property of the child, and (ii) that intelligence exists in “amounts.” For example, the first two items in the theory of intelligence scale require the child to consider the statements: “You have a certain amount of intelligence, and you really can’t do much to change it” and “Your intelligence is something about you that you can’t change very much” (p. 177).
The remainder of this article will centre on statements like “I believe my intelligence/ability is fixed” and “I believe my intelligence/ability can be changed.” Such statements sit at the core of Dweck’s theory, but I will argue that they are, in fact, meaningless because neither intelligence nor mathematical ability are things-in-themselves.
The serious conceptual errors in Dweck and Boaler’s research have their origins in the psychologist’s insistence on thinking of the concepts they study as “strongly objective” in the same sense as the concepts of Newtonian physics are strongly objective (Gould, 1996). The attraction of psychology’s Newtonian worldview is obvious in that classical physics holds out the promise of certainty and objectivity. Psychology’s Newtonian “physics envy” is puzzling given that physicists themselves turned their backs on a Newtonian picture of reality, adopting a quantum theoretical one approximately a century ago.
A clear indication that psychology – in common with quantum theory – is “weakly objective” can be seen in psychology’s century-long quest to define what intelligence is and what memory is. The explanation for the futility of this quest? The question “what is intelligence?” and the question “what is memory?” are not meaningful in a weakly objective framework. Outside the strongly objective framework, intelligence and memory cannot be regarded as things-in-themselves about which one can speak meaningfully while disregarding the measurement context.
A recent initiative to reproduce the findings of 100 important papers in the psychological literature succeeded in only 39% of cases. Unless psychology responds definitively, the so-called “reproducibility” project will continue to threaten cherished psychological principle after cherished psychological principle, undermining the very discipline itself. The source of psychology’s current reproducibility ills is its claim to strong objectivity, at a time when science’s most powerful theory – tested to destruction in the laboratory – embraces weak objectivity.
Werner Heisenberg (1958), summarising “the revolution in modern science,” writes: “Since the measuring device has been constructed by the observer … we have to remember that what we observe is not nature in itself but nature exposed to our method of questioning.” His great mentor, Niels Bohr, drew parallels between quantum theory and psychology for all of his professional life, prompting Harvard physicist Abner Shimony to conjecture that: “quantum concepts can be applied to psychology, but not with as much geometrical structure as in quantum physics.” Richardson’s (1999, p. 40) writings (in respect of psychological measurement) echo Heisenberg’s: “[W]e find that the IQ testing movement is not merely describing properties of people: rather, the IQ test has largely created them.”
No scientist of note has ever supported psychology in its claim to strong objectivity. As far back as 1955, the great American physicist Robert Oppenheimer pleaded with psychologists to forsake their adherence to strong objectivity: “It seems to me that the worst of all possible misunderstandings would be that psychology be influenced to model itself after a physics which is not there anymore, which has been quite outdated.” Gigerenzer (1987, p. 11) explains the context of Oppenheimer’s plea: “quantum theory was indeed discussed in the 1940s and ‘50s within psychology. However, it was unequivocally rejected as a new ideal of science.” Psychology shut its ears and (incredibly) continued to teach that the measurement of living beings can be modelled on Newtonian measurement principles developed to model inanimate matter! As physicist Henry Stapp (1993, p. 219) put it: “while psychology has been moving towards the mechanical concepts of nineteenth-century physics, physics itself has moved in just the opposite direction.”
If all this seems fanciful, consider two of psychology’s most studied mental predicates: memory and intelligence. I choose memory and intelligence because both have been studied intensively for over a century and I do not wish to be accused of making the case for weak objectivity in psychology by selecting a poorly researched area of the discipline. The psychologists quoted in what follows are all leading researchers in memory and intelligence.
First, let’s consider memory. Jenkins (1979, p. 431) hinted that psychology should eschew its quest to understand memory as a thing-in-itself: “The memory phenomena that we see depend on what kinds of subject we study, what kinds of acquisition conditions we provide, what kinds of material we choose to work with, and what kinds of criteria measures we obtain.”
Roediger (2008, p. 247) expresses disappointment that after decades of study, the search for an answer to the question “what is memory?” must be abandoned. In his 2008 review article Relativity of remembering: why the laws of memory vanished, he writes: “The most fundamental principle of learning and memory, perhaps its only sort of general law, is that making any generalisation about memory one must add that ‘it depends.’” Roediger (2008, p. 247) – anticipating the Reproducibity Project – suggests that whatever claim a psychologist makes about memory, a sceptic can always say; “Very nice work, but your finding depends on many other conditions. Change those and your effect will go away.” Measurement of memory seems unavoidably dependent upon the measurement context. One can detect the same concerns in Tulving’s (2007) paper entitled: Are there 256 different kinds of memory?
Psychology’s reproducibility problem is cast in a very different light when memory is treated as a joint property of the individual and the measurement context; change the measurement context and the meaning of what is measured must change. Battig (1978) suggests that this dependence on measurement context generalises to all psychological attributes. Battig’s reasoning goes beyond the conservative claims of the Reproducibility Project and would suggest that psychology faces years of damaging criticism if it continues to treat the attributes it studies as strongly objective. What physics has long since come to accept as a fundamental truth, Roediger III (2008, p. 228) bemoans as a matter of regret: “the great truth of the first 120 years of empirical study of human memory is captured in the phrase ‘it depends.’”
Quantum theory has taught physicists to accept that the question “what is an electron?” makes no sense in a weakly objective framework. Because the physicist participates in what he or she “sees,” it must be acknowledged that physics is not concerned with standing back from nature and objectively reporting what one “sees.” Rather, physics is limited to studying humankind’s interaction with nature. Roediger’s “it depends” response is embraced in quantum theory: in experimental arrangement A, the electron manifests as a particle; in experimental arrangement B it manifests as something very different – a wave. How are the statements “the electron is a particle” and “the electron is a wave” to be reconciled? The obvious paradox is avoided by demanding (as Niels Bohr did) that physicists “communicate unambiguously” by always making reference to the measuring instrument.
The statements “the electron is a particle relative to experimental arrangement A” and “the electron is a wave relative to arrangement B” banish the paradox entirely. “Particle” and “wave” are not intrinsic properties of the electron (leading to an obvious contradiction); instead, they are properties of the electron’s interaction with the measuring tool. According to Bohr one is communicating ambiguously – for Bohr, unambiguous communication is the hallmark of science – if one speaks of the electron as a thing-in-itself, independent of how it is observed. All of this has relevance for Dweck’s self-theory notion because all reference to beliefs that “my intelligence is fixed/malleable” are ruled out as meaningless in a weakly objective framework, because intelligence is being wrongly interpreted as a thing-in-itself.
One can get the distinct impression from the psychological literature that the typical psychologist sees herself as a passive, objective observer of what she studies. There is little evidence that psychologists see themselves as participants in what they “see.” This mistaken adherence to strong objectivity also afflicts the thinking of the Reproducibility Project’s researchers who see themselves as mere observers of a re-run of psychology’s signature experiments. The very notion of “reproducibility” sits awkwardly with the teachings of Bohr: “In the study of atomic phenomena, however, we are presented with a situation where the repetition of an experiment with the same arrangements may lead to different recordings” (Bohr, 1958-1962, p. 18). Jammer (1999, p. 234) writes that “Bohr did not regard the world as an objective reality with a given structure … conceptually separable from us as observers. … Thus, there must be limits to the depth of understanding that we can hope to gain of the world, because of our joint role as spectators and actors in the drama of existence.” Misner, Thorne & Wheeler (1973, p. 12) counsel: “’Participator’ is the incontrovertible new concept given by quantum mechanics; it strikes down the term ‘observer’ of classical theory, the man who stands safely behind the thick glass wall and watches what goes on without taking part. It can’t be done, quantum mechanics says. Even with the lowly electron one must participate before one can give any meaning whatsoever to its positon and velocity.”
When attention turns to the construct “intelligence,” the case against psychology’s claim to strong objectivity deepens. Jensen (1998, p. 46) acknowledges that after a century of attempts, a widely endorsed answer to the question “what is intelligence?” has eluded psychologists: “No other term in psychology has proved harder to define than ‘intelligence.’ Not that psychologists haven’t tried. Though they have been attempting to define ‘intelligence’ for at least a century, even experts in this field still cannot agree on a definition. In fact, there are as many different definitions of ‘intelligence’ as there are experts.” One high profile search for a definition of intelligence was documented in 1986 by Sternberg and Detterman. In the concluding paragraph of the book, Detterman sums up the experts’ judgements: “For those who expected to read this volume – entitled “What is intelligence?” – and obtain the definitive definition, I apologise.”
It was argued above that a commitment to communicate unambiguously demands that the measurement context be clearly specified if statements such as “Richard and Einstein have the same mathematical ability” are to be justified. A statement of the mathematics tested in the Foundation examination clarifies that the words “mathematical ability” refer to a very restricted subset of the entire domain of mathematics. In respect of intelligence Gladwell (2007, p. 95) draws conclusions from the Flynn-effect that strike at the heart of Dweck’s research: “For instance, Flynn shows what happens when we recognize that I.Q. is not a freestanding number but a value attached to a specific time and a specific test. … The notion that anyone “has” an I.Q. of a certain number, then, is meaningless unless you know which WISC he took, and when he took it, since there’s a substantial difference between getting a 130 on the WISC(IV) and getting a 130 on the much easier WISC.”
I.Q. is a relational attribute; it is the property of an interaction (between test-taker and test) and not an intrinsic property of the test-taker. This makes it impossible to define what intelligence is as a thing-in-itself. For example, I suspect that Dweck is using the word “intelligence” in the restricted sense of intelligence tests. But Howard Gardner (1983), a highly respected Harvard psychologist, extended intelligence beyond the language and logical-mathematical realms to spatial intelligence, musical intelligence, the use of the body to solve problems or make things, and interpersonal/intrapersonal intelligences. Indeed, why stop at Gardner’s seven so-called “multiple intelligences”? Statements such as “I believe my intelligence is fixed” are devoid of any clear meaning without a precise specification of the measurement context.
The distinguished American physicist David Mermin developed Niels Bohr’s counsel that psychology can learn from quantum theory that there are questions which cannot be answered in a weakly objective framework. It is instructive to quote Mermin’s words in full: “What does it mean for a property to be real? When you study an object how can you be sure you are learning something about the object itself, and not merely discovering some irrelevant feature of the instruments you used in your study? This is a question that has plagued generations of psychologists. When you measure IQ are you learning something about an inherent quality of a person called “intelligence,” or are you merely acquiring information about how the person responds to something you have fancifully called an IQ Test? Until the advent of the quantum theory in 1925 physicists were above such concerns. But since then, with the discovery that experiments at the atomic level necessarily disturb the object of investigation, precisely such reservations have been built into the foundations of physics.”
From the physicist’s perspective: “[I]f we set out to measure the momentum, say, of an electron, what we are actually measuring is the ability of an electron to answer questions about momentum. The electron may, indeed, not have any such property as momentum, in the way we think of it in the everyday world… . We get experimental results – ‘answers’ – which we interpret as measures of momentum. But they are only telling us about the ability of electrons to respond to momentum tests, not their real momentum, just as the results of IQ measurements only tell us about the ability of people to respond to IQ tests, not their real intelligence” (Gribbin, 1995, p. 148).
Weak objectivity views measurement as context-dependent. When the position of an electron is measured, the physicist is not merely checking up on an investigation-independent property inherent in the election. What one is really measuring is the interaction between the electron and the measuring instrument. The measurement outcome is a joint property of the electron and the measurement instrument. Niels Bohr considered questions which treated the electron as a thing-in-itself (such as “what is an electron?”) as meaningless in a weakly objective framework. As anyone who has watched popular science programmes will recognise, in quantum theory an electron manifests as a wave in one measurement context, and as a particle in another. It is therefore meaningless to ask what an electron is as a thing-in-itself, without reference to the measurement context.
An electron is a wave relative to one measurement context, and a particle relative to another. According to his close colleague, Aage Petersen, Bohr summarised the switch from strong to weak objectivity as follows: “It is wrong to think that the task of physics is to find out how nature is. Physics concerns what we can say about nature.” He uses the word “say” because in a weakly objective framework, in order to communicate ambiguously one must provide a description of the measurement apparatus. Changes in the measurement context therefore have consequences for the very meaning of what is measured.
Bohr labelled this tendency of quantum entities to manifest as wave or particle, depending on the measurement context, as “complementarity.” He regarded complementarity as the central concept in quantum theory. Physicists treat these two characteristic manifestations as opposites given that a particle is confined to a tiny region of space, while a wave spreads throughout space. A strikingly similar concept arising in the study of mind appears in the writings of one of the greats of modern philosophy, Ludwig Wittgenstein. In the secondary literature derived from Wittgenstein’s later philosophy, it has become known as “first-person/third-person asymmetry.” This asymmetry applies to intentional predicates in general and to intelligence and mathematical ability in particular. Incidentally, this analogue of complementarity in respect of mind avoids all of the difficulties associated with both Cartesian dualism and behaviourism.
I will quote in full from Colin McGinn’s book The Character of Mind in order to illustrate that an analogue of complementarity informs fundamental thinking about mind. McGinn (1996, pp. 6-7) writes: “Mental concepts are unique in that they are ascribed in two, seemingly very different, sorts of circumstances: we apply them to ourselves on the strength of ‘inner’ awareness of our mental states, as when a person judges of himself that he has a headache; and we apply them to others on the strength of their ‘outer’ manifestations in behaviour and/or speech. These two [opposite] ways of ascribing mental concepts are referred to as first-person and third-person ascriptions. … It would be fine if we could somehow, as theorists, prescind from both perspectives and just contemplate how mental phenomena are, so to say, in themselves; but this is precisely what seems conceptually unfeasible … we seem to need the idea of a single mental reality somehow neutral between the first- and third-person perspectives; the problem is that there does not appear to be any such idea.”
Physics has taught us that a statement such as “the electron’s velocity is constant” is utterly meaningless in a weakly objective framework such as the quantum framework. Velocity is not an intrinsic property of the electron. Quantum theory rules out all reference to the velocity of an electron without a clear description of the particular measurement context. One gets nonsense when one omits the context of ascription. Similarly, investigation-independent statements such as “I believe that my intelligence is fixed” and “I believe my mathematical ability can grow” make little or no sense. Both of these statements wrongly present mind as a carrier of definite states. Mind is a carrier of potentiality (just like the microentity) and not a carrier of definite states. Intelligence and mathematical ability are not inner states which are somehow the source of behaviour. In respect of the first statement above, Mermin’s reasoning (see above) rejects the notion that intelligence can ever be an intrinsic property of the individual.
But what of mathematical ability as Boaler construes it? Just as in the case of intelligence, mathematical ability (as a thing-in-itself) is a potentiality rather than a state. A pupil who has grasped the concept “even number” has the potential to non-collusively modify his or her behaviour so that it is in accord with accepted mathematical practice in respect of the even numbers. Mathematical ability is therefore a joint property of the individual and the fiduciary (to borrow Polanyi’s term) framework within which he or she has been educated. As with intelligence, mathematical ability is not an intrinsic property of the person.
In conclusion, it is instructive to demonstrate the gulf between Dweck and Boaler’s research and the writings of one of the greatest physicists of all time. On page 96 of the first volume of his essays Bohr (1934) considers the connection between “the conscious analysis of any concept” and “its immediate application.” For example, how is the ability to apply the concept “even number” in accord with established mathematical practice connected to the possession of an introspectable mental “object,” namely, the formula which generates the even numbers: Un = 2n? How does having the formula in mind (a concept capable of “conscious analysis”, in Bohr’s terms) connect with one’s ability to say or write out the even numbers in accord with established mathematical practice (“immediate application”, in Bohr’s terms)?
The following connection immediately suggests itself: the mental image of the formula is the source of the individual’s ability to write out the even numbers. Two entirely separate realms are suggested here: on the one hand, the individual’s understanding of the even numbers (to have the formula in mind is to understand the even numbers), and, on the other hand, the application of that understanding in the writing out of the even numbers in accord with established practice. In this picture the inner world of understanding is divorced entirely from the public realm of application. From this viewpoint it is tempting to think of the formula-in-mind as representing mathematical understanding as a thing-in-itself. For Bohr, this strongly objective picture of the connection between “inner” and outer must be invalid.
Bohr considered this Cartesian picture (with its Newtonian, self-standing mental “objects”) as entirely wrong. To understand Bohr’s reasoning one must turn to the later philosophy of Wittgenstein and to his writings on rule-following in particular. The error in the strongly objective Cartesian picture, in which mathematical ability in respect of the even numbers is entirely divorced from subsequent application, is that the inner formula cannot be the source of correct application because it has no guidance properties whatever. The formula as a thing-in-itself simply cannot determine subsequent application. One can only derive guidance from a mathematical formula by being trained in the practice of using that formula. However, mathematical practice is a feature of the entirely separate public realm where the formula is to be applied. Because the formula-in-mind doesn’t have its applications written into it, it cannot guide.
It is the experience of mathematics teachers throughout the world that the appearance of the quadratic formula in the formula sheet made available to pupils taking public examinations is no guarantee that all pupils taking the examination will be able to successfully apply that formula. Wittgenstein demonstrates that when understanding as a thing-in-itself is separated from application, any attempt to explain how these two realms are connected leads to a destructive “regress of interpretations.” When one defines “understanding of the even numbers” as having a mental object (the formula Un = 2n) in mind, divorced entirely from the mathematical practice which gives the formula its life, one descends into confusion (see Oakeshott, 1975). Even experienced mathematicians often fail to recognise the role played by their long apprenticeship in the discipline. Malcolm (1965, p. 102) questions the experienced mathematician’s mistaken intuition that formulae in themselves determine their applications: “You would like to think that your understanding of the formula determines in advance the steps to be taken, that when you understand or meant the formula in a certain way ‘your mind as it were flew ahead and took all the steps before you physically arrived at this one or that one’ (Wittgenstein, 1953, §188).”
Wittgenstein and Bohr resolve the regress of interpretations problem by treating the two realms as conceptually inseparable. This allows the difficulties of both Cartesian dualism and behaviourism to be avoided. Kenny (2004, p. 49) explains the pivotal role played by Wittgenstein’s notion of criteria: “According to him the connection between mental states and physical [application] is neither one of logical reduction (as in behaviourist theory) nor one of causal connection (as in Cartesian theory). According to him the physical expression of the mental process is a criterion for that process; that is to say, it is part of the concept of a mental process of a particular kind that it should have a characteristic manifestation. The criteria by which we attribute states of mind and mental acts, Wittgenstein showed, are bodily states and activities.”
Mind is expressed in behaviour; the teacher cannot help but “see” the child’s understanding of the concept “even number” in the ease with which she applies the rule for generating the even numbers. This is how the connection is made. In an appropriate school context, the ability to unhesitatingly write the next 100 even numbers, starting with 2088, for example, serves as a criterion which justifies the teacher in saying that the child understands the concept “even number.” The Cartesian causal connection between mind, on the one hand, and behaviour, on the other, is replaced by a picture in which mind and behaviour are treated as an indivisible whole.
Malcolm (1965, pp. 101-102) summarises Wittgenstein’s (and Bohr’s) rejection of the notion that mathematical ability is a thing-in-itself, entirely divorced from application: “But the question of whether one understands the rule cannot be divorced from the question of whether one will go on in that one particular way that we call ‘right.’ The correct use is a criterion of understanding. … You would like to think that your understanding of the formula determines in advance the steps to be taken, that when you understood or meant the formula in a certain way “your mind as it were flew ahead and took all the steps before you physically arrived at this one or that one” (§188). But how you meant it is not independent of how in fact you use it. … How he meant the formula determines his subsequent use of it, only in the sense that the latter is a criterion of how he meant it.”
References
Battig, W.F. (1978). Parsimony or psychology? Presidential address, Rocky Mountain Psychological Association, Denvir, CO.
Bohr, N. (1934). The philosophical writings of Niels Bohr. Woodbridge: Ox Bow Press.
Bohr, N. (1958-1962). Essays 1958-1962 o atomic physics and human knowledge. Woodbridge: Ox Bow Press.
Dweck, C. S. (2000). Self-theories: their role in motivation, personality, and development. Philadelphia, PA: Psychology Press.
Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Random House.
Gardner, H. (1983). Frames of mind. New York: Basic Books.
Gigerenzer, G. (1987). Probabilistic thinking and the fight against subjectivity. In L. Kruger, G. Gigerenzer, & M.S. Morgan (Eds.), The probabilistic revolution – Volume 2: Ideas in the sciences (pp. 11-33). Cambridge, MA: The Massachusetts Institute of Technology Press.
Gladwell, M. (Dec. 17, 2007). None of the above. New Yorker magazine.
Gould, S.J. (1996). The mismeasure of man. London: Penguin Books.
Gribbin, J. (1995). Schrödinger’s kittens and the search for reality. London: Weidenfeld & Nicolson.
Hacker P.M.S. (1997). Insight and illusion: Themes in the philosophy of Wittgenstein. Bristol: Thoemmes Press.
Hacker, P.M.S. (2013). The intellectual powers: A study of human nature. Oxford: Wiley Blackwell.
Heisenberg, W. (1958). Physics and philosophy: the revolution in modern science. New York: Prometheus Books.
Jammer, M. (1999). Einstein and religion. Princeton, NJ: Princeton University Press.
Jenkins, J.J. (1979). Four points to remember: a tetrahedral model of memory experiments. In L.S. Cremak, & F.I.M. Craik (Eds.), Levels of processing in human memory (pp. 429-446). Hillsdale, NJ: Lawrence Erlbaum.
Jensen, A.R. (1998). The g factor: the science of mental ability. Westport, CT: Praeger.
Kenny, A. (2004). The unknown God. London: Continuum.
Malcolm, N. (1963). Knowledge and certainty. Englewood Cliffs, NJ: Prentice-Hall.
McGinn, C. (1996). The character of mind. Oxford: Oxford University Press.
Mermin, N.D. (1993). Lecture given at the British Association Annual Science Festival. London: British Association for the Advancement of Science.
Misner, C.W., Thorne, K.S., & Wheeler, J.A. (1973). Gravitation. San Francisco: Freeman.
Oakeshott, M. (1975). On human conduct. Oxford: Clarendon Press.
Oppenheimer, R. (1955, September 4). Analogy in science. Paper presented at the 63rd Annual Meeting of the American Psychology Association, San Francisco, CA.
Richardson, K. (1999). The making of intelligence. London: Weidenfeld & Nicolson.
Roediger III, H.L. (2008). Relativity of remembering: Why the laws of memory vanished. Annual Review of Psychology, 59, 225-254.
Stapp, H.P. (1993). Mind, matter, and quantum mechanics. Berlin: Springer-Verlag.
Sternberg, R.J., & Detterman, D.G. (Eds.). (1986). What is intelligence? Contemporary viewpoints on its nature and definitions. Norwood, NJ: Ablex Publishing Corporation.
Tulving, E. (2007). Are there 256 different kinds of memory? In J.S. Nairne (Ed.), The foundations of remembering: Essays in honour of Henry L. Roediger III (pp. 39-52). New York: Psychological Press.
Wittgenstein, L. (1953). Philosophical investigations. Oxford: Blackwell.
Wright, C. (2001). Rails to infinity: Essays on themes from Wittgenstein’s Philosophical Investigations. Cambridge, MA: Harvard University Press.
