Language rights in Mother Ireland

Tags

, , , , , , ,

The Decline of the Celtic Languages

The Northern Ireland Assembly’s failure to deliver “language rights” is a constant refrain of Sinn Fein politicians. One such right must be: “the right of every parent to exclude their child from all aspects of the teaching and assessment of Irish in school.”

The Irish language now has an absolutely unique place in the history of teaching and assessment in that thousands of Irish parents have sought out psychologists to confirm that their child has a disability which precludes them from sitting examinations in Irish. This purported disability appears to be unique to Irish in that many of the children suffering from it nevertheless present for the relevant examinations in French, German, Spanish etc. According to the banner headline on the front page of The Irish Times (14.06.2018): “The Department of Education is reviewing the granting of exemptions from studying Irish amid evidence that thousands of students who secure them are sitting exams in foreign languages.” The fact that “[s]tudents or their parents are required to submit a psychologist’s report in order to secure an exemption on the grounds of disability” may give the impression that negative attitudes to the teaching and assessment of Irish are a new development, or are the preserve of the rich. Alas, history reveals that strong negative attitudes to learning Irish in school transcend class in Ireland.

Irish Times exemptions

Writing in The Decline of the Celtic Languages, Victor Durkacz notes: “Every interest, claimed the Society for Educating the Poor in Ireland, ‘every ambition, every means of advancement and hope of profit for the peasantry, depend upon their acquisition of English. Thus the Commissioners of National Education could truthfully claim that at no time had the parents shown an inclination for the schools to cultivate Irish. On the contrary, they had demonstrated their anxiety that their children should learn English.”

Advertisements

The fundamental flaw in the University of Cambridge’s psychographics algorithm: private traits are not predictable from Facebook “likes.

Tags

, , , , , , , , , , , , , , , , , , , , , , , , ,

Dr Hugh Morrison (The Queen’s University of Belfast [retired])

Despite the fact that research carried out at the University of Cambridge’s Psychometrics Centre is at the very heart of the Cambridge Analytica debate, this research seems to have been subject to little or no serious scrutiny.  This is puzzling given that an analysis of the Apply Magic Sauce algorithm, developed at the University, could settle, once and for all, whether tools developed at Cambridge could have influenced the Brexit or Trump votes.  The impression has been created that the Cambridge academics, armed only with an individual’s Facebook “likes,” could somehow use this Apply Magic Sauce algorithm to peer into the mind of that individual.

The Cambridge academics (erroneously) portray psychological constructs such as personality and intelligence as “inner”, “private” traits, somehow hidden in mind.  It is claimed, however, that provided one has access to an individual’s Facebook likes, one can use the Apply Magic Sauce algorithm to represent numerically that individual’s five-trait personality profile, together with his or her intelligence.  It is easy to see how the title of a paper by Michal Kosinski, David Stillwell & Thore Graepel (2013)[1], published in PNAS – “Private traits and attributes are predictable from digital records of human behaviour” – gives the impression that tools developed at Cambridge can somehow “reveal” an individual’s personality/intelligence, given his or her digital footprint.  It will be argued in this essay that no scientific basis exists for this claim.

Nina Burleigh, writing in Newsweek (18.06.2017), paints a picture of a dystopian future in which algorithms can be used to infer psychological profiles by stealth: “Big Data, artificial intelligence and algorithms designed and manipulated by strategists like the folks at Cambridge have turned our world into a Panopticon, the 19th century circular prison designed so that guards, without moving, could observe every inmate every minute of every day.”  The Cambridge Analytica whistleblower, Cristopher Wylie, goes further in the Observer newspaper of 18.03.2018.  According to Wylie, because “personality traits could be a precursor to political behaviour,” (p. 10) then the tools developed at Cambridge could represent a “psychological warfare mind**** tool” (p. 9) capable of disrupting the democratic process itself.

This brief essay asks the important question, “is there a shred of scientific truth in the claim that one can exploit the link between inner mental states (personality traits, intelligence etc.) and voting intention, in order to nudge an individual’s behaviour in the polling booth?”  The unequivocal response is “No,” because the Cambridge academics’ claim to have developed an algorithm capable of inferring personality from Facebook likes has no scientific basis.

The Apply Magic Sauce algorithm claims to predict (and quantify) personality traits.  This cannot be so for a very simple reason: such traits do not exist.  This we know from the extensive rule-following literature (see, for example, Crispin Wright’s 2001 book, “Rails to Infinity”[2]).  Dynamic intrinsic attributes are the preserve of Newtonian mechanics, and are not available to psychologists.  Personality is a property of the interaction between person and measuring tool rather than an intrinsic attribute of a person; it is a joint property of the individual and the instrument used to measure personality.

One of the towering figures of 20th century thought, Ludwig Wittgenstein (1958, p. 143) dismissed unequivocally the notion of inner mental traits posited by the Cambridge team: “There is a kind of disease of thinking which always looks for (and finds) what would be called a mental state from which all our acts spring as from a reservoir.”[3]  One cannot capture an individual’s personality in a number because personality isn’t a property of the individual.  Moreover, it is inconceivable that the Cambridge academics were not aware that their interpretation of personality as a quantifiable trait was entirely at odds with serious scholarship in their own discipline.

Ross and Nisbett’s classic textbook The Person and the Situation[4] is a staple of undergraduate psychology.  This book argues that personality is so entangled with the context in which it is expressed that it is meaningless to conceive of personality as some free-standing quantifiable inner state.  One of psychology’s most respected thinkers, Jerome Kagan, has identified this tendency for some psychologists to picture psychological attributes as traits hidden in mind as undermining their profession.

On page xvii of Psychology’s Ghosts: The Crisis in the Profession and the Way Back[5], Kagan “concludes that ‘agents in a context’ should replace the current, restricted focus on stable properties of individuals that, like their eye colour, are presumably available for expression in all settings.”  Professor Kagan points out that measurement in psychology is not a matter of checking up on traits which already exist in mind.  Kagan (1998, p. 16)[6] writes: “Most investigators who study ‘anxiety’ or ‘fear’ use answers on a standard questionnaire or responses to an interviewer to decide which of their subjects are anxious or fearful.  A smaller number of scientists asks close friends or relatives of each subject to evaluate how anxious that person is.  A still smaller group measures the heart rate, blood pressure, galvanic skin response, or salivary cortisol level of their subjects.  Unfortunately, these three sources of information rarely agree.”

Kagan is making the point that in order to communicate unambiguously (the hallmark of science) one cannot omit the measuring instrument.  Psychological predicates only have definite properties relative to a specified measuring tool.  One cannot attribute a definite value to a psychological attribute construed as a property of a person; this attribute only has definite properties relative to a particular instrument.  Kagan (1998, p. 77) cautions: “Modern physicists appreciate that light can behave as a wave or a particle depending on the method of measurement.  But some contemporary psychologists write as if that maxim did not apply to consciousness, intelligence or fear.”  This poses a fundamental problem for the Cambridge project because personalities predicted from Facebook likes involve no interaction with an appropriate measuring tool.  In these circumstances ascribing a definite personality makes no sense.

In Malcolm Gladwell’s popular text The Tipping Point,[7] he depicts the incoherence which is the consequence of divorcing psychological predicates from the contexts in which they are expressed, and treating them as traits.  Indeed, psychology has a name for the error which afflicts the Cambridge algorithm: “The Fundamental Attribution Error.”  According to the Fundamental Attribution Error, behaviour is not determined by some theoretical mental trait within the individual; rather, behaviour can only be understood by examining the interaction between the individual and the measuring instrument.  Finally, even if one were to accept that psychological predicates can be represented as latent traits, such models break down at the level of individual (see Borsboom, Mellenbergh & van Heerden, 2003, p. 217)[8], presenting intractable problems for any “personalisation machine” capable of “microtargeting.”

Turning to the claim that one can measure intelligence from Facebook likes, one finds the Cambridge algorithm at odds with one of the central tenets of quantum mechanics (see below).  The reader will be aware of the decades-old debate: do intelligence tests measure some inner, hidden mental state which psychologists call “intelligence”, or do they merely measure the ability of the testee to answer a series of questions on something psychologists call an “intelligence test”?

Psychology was aware – as far back as the 1930s – of the profound difficulties associated with interpreting intelligence as a trait: “He [Carl Brigham] recognized that a test score could not be reified as an entity inside a person’s head: Most psychologists working in the test field have been guilty of a naming fallacy which easily enables them to slide mysteriously from the score in the test to the hypothetical faculty suggested by the name of the test. Thus, they speak of sensory discrimination, perception, memory, intelligence, and the like while reference is to a certain objective test situation” (Gould, 1996, p. 262)[9].

The distinguished American physicist, David Mermin (1993, p. 1)[10] notes: “When you measure IQ are you learning something about an inherent quality of a person called “intelligence,” or are you merely acquiring information about how the person responds to something you have fancifully called an IQ test?  Until the advent of quantum theory in 1925 physicists were above such concerns.  But since then, with the discovery that experiments at the atomic level necessarily disturb the objects of investigation, precisely such reservations have been built into the foundations of physics.”  In modern physics, momentum is not an intrinsic property of an electron; rather, it is a property of the electron’s interaction with the measuring instrument.

The introduction to Werner Heisenberg’s 1989 book[11] contains a single sentence with profound implications for the Apply Magic Sauce algorithm: “[T]he reality is in the observations [interactions], not in the electron.”  The lesson for Apply Magic Sauce is clear: one can only speak meaningfully about an individual’s intelligence when that person interacts with an intelligence test.  Intelligence is not an inner trait and the Cambridge algorithm can no more predict intelligence than it can predict personality.

The Cambridge academics’ central claim for their algorithm is that it can predict an individual’s personality without requiring that person to take a personality test.  Alas, removing the measuring instrument actually deprives any references to personality or intelligence of their very meaning.  Those who feared that academics at Cambridge had actually developed an algorithm which could derive an individual’s intimate traits from his or her digital footprint, without requiring that person to take the relevant test, can take comfort from the words of the physicist Asher Peres: “Unperformed experiments have no results.”[12]  In summary, it seems inconceivable that an algorithm with the conceptual difficulties of Apply Magic Sauce could be deployed to nudge voting intention in any predetermined direction.

 

drhmorrison@gmail.com

[1] Kosinski, M., Stillwell, D., & Graepel, T. (2013).  Private traits and attributes are predictable from digital records of human behaviour.  Proceedings of the National Academy of Sciences of the United States of America, 110 (15), 5802-5805.

[2] Wright, C. (2001).  Rails to infinity.  Cambridge, MA: Harvard University Press.

[3] Wittgenstein, L. (1958).  The blue and brown books.  Oxford: Blackwell.

[4] Ross, L, & Nisbett, R.E. (1991).  The person and the situation.  Philadelphia: Temple University Press.

[5] Kagan, J. (2012).  Psychology’s ghosts.  New Haven: Yale University Press.

[6] Kagan, J. (1998).  Three seductive ideas.  Cambridge, MA: Harvard University Press.

[7] Gladwell, M. (2000).  The tipping point.  Boston: Little, Brown & Company.

[8] Borsboom, D., Mellenbergh, G.J., & van Heerden, J. (2003).  The theoretical status of latent variables.  Psychological Review, 110 (2), 203-219.

[9] Gould, S.J. (1996).  The mismeasure of man.  London: Penguin Books.

[10] Mermin, D. (1993).  Lecture given at the British Association Annual Science Festival.  London: British Association for the Advancement of Science.

[11] Heisenberg, W. (1989).  Physics and philosophy.  London: Penguin Books.

[12] Peres, A. (1978).  Unperformed experiments have no results. American Journal of Physics, 46, 745-747.

Big Data, Big Lies, Big Payday

Tags

, , , , , , , , , , , , , , , , , , , , , , , , , ,

Why the “psychographic” techniques which underpin the Big Data ideas of Michal Kosinski and Alexander Nix have no scientific foundation.

Nix2KosinskiFacebook Like

Dr Hugh Morrison (The Queen’s University of Belfast [retired])

drhmorrison@gmail.com

“There is a general disease of thinking which always looks for (and finds) what would be called a mental state from which all our acts spring as from a reservoir.”  Ludwig Wittgenstein

The data mining techniques of the software company Cambridge Analytica have come in for particular scrutiny in the aftermath of the 2016 American presidential election.  In recent days, two names in particular have become associated with using the Facebook “likes” of individuals to gain access to their personality: Michal Kosinski, a past Operations Director at the Cambridge University Psychometrics Centre, and Alexander Nix of the company Cambridge Analytica.  While there is clearly much that divides these two men, they appear to have at least one belief in common, namely, that “Big Data” techniques make it possible to access the personality profile of an individual from his or her Facebook likes.  I will make the case that this claim has no scientific merit.  “Psychographics,” as it has come to be known, has no basis whatever in science.

 

It is important to investigate the claims made by those who use data mining to infer personality.  Aside from the eye-watering sums of money involved, it is important that the general public be aware of the profound limitations of this use of Big Data.  The June 16, 2017 issue of Newsweek captures the degree to which we should all be concerned: “Big Data, artificial intelligence and algorithms designed and manipulated by strategists like the folks at Cambridge [Analytica] have turned our world into a Panopticon, the 19th century circular prison designed so that guards, without moving, could observe every inmate every minute of every day.”

 

The central claim advanced by those who advocate the use of Big Data to uncover information about personality is that information derived from an individual’s Facebook likes can be used to infer something about that individual’s personality.  This claim has no scientific basis for a very simple reason; personality is not a property of the individual which can be represented numerically.  Personality is a joint property of the individual and the context in which it was manifest.  Personality isn’t a trait which the individual somehow carries within themselves from context to context.  Rather, personality varies with context: a child may be extrovert at home, but quiet and reserved in the classroom.  She may be extrovert in the company of the children who live next door, but introvert when interacting with strangers in unfamiliar settings.

 

The research documented in Malcolm Gladwell’s best-seller The Tipping Point leaves one in very little doubt that – pace Kosinski and Nix – personality cannot be a trait, ascribable to the individual, and amenable to quantification.  On page 186 of his book Surprise, Uncertainty, and Mental Structures, the distinguished Harvard psychologist Jerome Kagan writes: “Some men are loyal to their wives and affectionate with their children but disloyal and hostile in their relations with colleagues at work.”  On page 188 he cautions: “[C]onclusions about personality that are based only on questionnaires or interviews have a meaning that is as limited as Ptolemy’s conclusions about the cosmos based on the reports of observers staring at the sky without telescopes.”  Every undergraduate physicist learns (from the teaching of Niels Bohr) that the mind is not a carrier of definite states which determine behaviour, but a carrier of potentiality which cannot be represented by real numbers.

 

In short, in order to communicate unambiguously (the hallmark of science) one must describe the context in which a particular facet of personality is manifest.  Data, no matter how “big”, is powerless to capture the complex interactions etc. that comprise human situations; we must rely on language in any attempt to represent context.  On page 135 of Werner Heisenberg’s book Physics and Beyond, the following advice appears: “For if we want to say anything at all about nature – and what else does science try to do? – we must somehow pass from mathematical to everyday language.”  The lesson for psychologists who use questionnaires to measure personality is that it is meaningful to speak about someone’s personality only if one details the questionnaire; personality is a relational attribute rather than an attribute intrinsic to the person.

 

In an article published in Proceedings of the National Academy of Sciences, Van Bavel, Mende-Siedlecki, Brady and Reinero demonstrate that the centrality of context (or environment) is something well-known to all undergraduate psychologists.  “Indeed, the insight that behaviour is a function of both the person and the environment – elegantly captured by Lewin’s equation: B = f(P, E) –  has shaped the direction of social psychological research for more than half a century.  During that time, psychologists and other social scientists have paid considerable attention to the influence of context on the individual and have found extensive evidence that contextual factors alter human behaviour.”  If personality is a joint property of the person and the context in which it is manifest, then unambiguous communication demands that a description of the context must be integral to any attempt to represent personality.

 

Finally, one only appreciates the incoherence of psychographics when one notes the intellectual standing of those whose writings oppose the thinking of Kosinski and Nix.  Three thinkers who stand out among those who argue that psychological attributes cannot be separated from the context in which they are manifest are the Nobel laureate Herbert A. Simon and two of the 20th century’s greatest intellectuals: the father of quantum theory, Niels Bohr, and the philosopher Ludwig Wittgenstein.

 

All three reject the notion that one can ignore context and treat behaviour as wholly analysable in terms of traits and inner processes (and therefore quantifiable).  Indeed, psychology itself has a name for the error at the heart of psychographics, a name familiar to all undergraduate psychologists.  Gerd Gigerenzer of the Max Planck Institute writes: “The tendency to explain behaviour internally without analysing the environment is known as the ‘fundamental attribution error.’”

 

First, Herbert Simon uses a scissors metaphor to indicate the degree to which a psychological attribute cannot be disentangled from the context in which it is manifest.  Herbert writes: “Human rational behaviour is shaped by a scissors whose blades are the structure of the environment and the computational capabilities of the actor.”

 

Secondly, Niels Bohr – in his Discussion with Einstein on Epistemological Problems in Atomic Physics – uses quantum complementarity to argue that first-person ascriptions [the contribution of the individual] and third-person ascriptions [the contribution of the environment] of psychological attributes form an “indivisible whole.”

 

Finally, on page 143 of his Blue and Brown Books, Wittgenstein highlights the error at the heart of the psychographics project: “There is a general disease of thinking which always looks for (and finds) what would be called a mental state from which all our acts spring as from a reservoir.”

 

 

 

 

 

 

 

 

 

 

 

 

“Research intensity” – fake news in higher education

Tags

, , , , , , , , , , , , , , ,

Ligo Caltech

One of the greatest advances in modern physics – the detection of gravitational waves first postulated, a century ago, by Einstein in his general theory of relativity – was made by physicists at the Laser Interferometer Gravitational-Wave Observatory (LIGO).  To paraphrase Richard Feynman, LIGO’s measurement precision can be expressed as follows: If you were to measure the distance between earth and the nearest star with this precision, it would be exact to the thickness of a human hair.  Such incredible accuracy alone would more than justify the £150 million construction costs of LIGO as a feat of engineering alone.

THE Intensity rankings

 

It is instructive to contrast the measurement properties of the UK’s Research Excellence Framework (REF), which aims to rank-order the research quality of UK universities, with those of LIGO.  While the REF league table has no discernible measurement properties whatsoever, its cost far exceeds LIGO’s construction costs, coming in at a staggering one quarter of a billion pounds.

THE REF winners

A recent issue of the Times Higher Education (1 – 7 February 2018) included a booklet published by the Queen’s University of Belfast which illustrates the extremes to which universities are prepared to go in using highly questionable data derived from REF ranks for the purposes of self-promotion.  Page 5 of the Queen’s booklet consists of a single statement.  At the centre of a black A4 page the words “Ranked in the top 10 in the UK for research intensity” (in white print and large font) sit in isolation.  (In a much smaller font the university attributes this ranking to the Times Higher Education.)  Pages 6 to 9 offer pen portraits of nine “world-leading academics” employed by Queen’s University.  The university has used this “research intensity” claim to market the university ever since the publication of the 2014 REF.  One cannot browse the university’s website without encountering the claim at every turn.  It has appeared on university billboards, in promotional materials and was central to the university’s ubiquitous claim: “we are exceptional.”  Why did no one notice that research intensity is a meaningless concept?

QUB THE insert cover

 

In the Research Excellence Framework, the research quality of journal articles, books etc. is assessed and reported on a four-point scale (five-point if one includes the ‘unclassified’ category).  The scale is ordinal in the sense that a 3* article is deemed superior to one rated 1* or 2*, and inferior to an article rated 4*.  Any appeal to arithmetic is impermissible because an article rated 4* is not 4 times the quality of a 1* article; a 2* article is not two-thirds the quality of a 3* article, and so on.  The rules of arithmetic do not apply.  (Needless to say, the challenge of assessing research quality would remain  unchanged if numbers were abandoned for the grades A, B, C and D.)  These whole-number ratings are then used by the Times Higher Education to compute a university’s all-important “research intensity” (reported to two decimal place accuracy) using simple arithmetic.  But, as every sixth-form statistician knows, arithmetical operations are not meaningful when applied to an ordinal scale.

QUB THE p7

 

How can it be that no world-class scientist at Queen’s seems to have pointed out to those charged with marketing the university that “research intensity” is a highly questionable measure?  Are we to believe that no UK scientist has written to the Times Higher Education pointing out the magazine’s error?  One gets the clear impression from those charged with marketing Queen’s University that the Belfast campus is crammed to the rafters with ”world-leading” scientists.  How could any scientist worthy of the label endorse the notion of “research intensity” when his or her sixth-form mathematics training would identify the notion as nonsensical.  In particular, why have none of the university’s world-leading academics, listed on pages 6 through 9 of the Queen’s promotional booklet, questioned the nonsensical claim printed on page 5?”

 

The Queen’s University of Belfast and the Times Higher Education must clarify their positions on this matter.

QUB THE insert p5

 

 

 

 

 

 

Plagiarism: One law for pupils and another for teachers

Tags

, , , , , , , , , ,

 

Stephen Elliott – Chair: Parental Alliance for Choice in Education

 

In the closing days of January 2018 it was revealed that Dermot Mullan, headteacher at Our Lady and St Patrick’s College in Belfast, was accused on plagiarising the work of another teacher.  Mr Mullan immediately confessed to the offence and that, it would seem, is to be the end of the matter.  His Board of Governors made no comment, the Catholic Church made no comment, and – most concerning of all – Northern Ireland’s General Teaching Council remained silent.  This silence is puzzling given that Mr Mullan heads a school which makes much of its lofty Catholic principles.  How does a plagiarist urge honesty and integrity on pupils in general (and pupils taking GCSE and GCE examinations, in particular)?  How does Mr Mullan discipline a pupil suspected of copying the coursework of another pupil?  Surely the parents of the culprit will detect a double standard here: there seems to be one rule for the children and another for their principal?

 

 

Dr Neill Morton pictured with Professor Tony Gallagher at QUB graduation.

The existence of a disturbing double standard is nowhere better illustrated than in the intervention of Neill Morton, the self-styled “emeritus” headmaster of Portora Royal School.  Despite being the Education Chair of Northern Ireland’s Examination Council (CCEA), Dr Morton appeared on BBCNI television Newsline on Monday 29th January 2018 to assure the public that the whole issue of Mr Mullan’s plagiarism was overblown.  This clearly demonstrates one law for pupils taking examinations and another for their teachers: if Dr Morton’s view of Mr Mullan’s indiscretion were applied to pupils, then the entire concept of public examinations would collapse.  In short, Dr Morton’s comments on Mr Mullan’s plagiarism should immediately disqualify him from any public office concerned with public examinations.

Dr Morton’s failure to condemn Mr Mullan’s activities outright is even more surprising given that he has recently completed a Doctorate in Education at The Queen’s University of Belfast.  A glance at that university’s website or a random walk through its McClay library will quickly reveal the seriousness with which it views plagiarism.

When pupils are charged with plagiarism the consequences can be drastic: their grades can be deleted; they may be expelled and the pupil whose work was plagiarised may fall under suspicion.  One doesn’t seem to encounter the same clarity of decision-making when it comes to settling the fate of a highly-salaried headteacher like Mr Mullan.  One encounters the same imbalance in respect of university students and their teachers: one can spend many hours searching for a well-defined Queens policy on staff accused of appropriating the work of other academics.

 

The claims advanced here deserve a response.  It is completely unacceptable that Dr Morton’s judgement of Mr Mullan’s plagiarism is entirely at odds with the treatment of examination candidates guilty of the same offence.  How must the parents of children judged to have plagiarised in an assessment have reacted to CCEA’s Education Chair making little of a headteacher facing the same charge?  Why have the Governors of Mr Mullan’s school not made a statement?  Why is the Catholic Church silent on what is a failure in morality in a person charged with leading by example?  Finally, why are teachers, pupils and parents yet to hear a word from Northern Ireland’s General Teaching Council?

The AQE CEA and GL Assessment Test Results: Advice to parents: 2018

Tags

, , , , , , ,

Undoubtedly, thanks mainly to media pressure, the results of the 2017/18 transfer tests will be the subject of conversations in families all over Northern Ireland this weekend and for months beyond. The Parental Alliance for Choice in Education wish to offer our congratulations to all pupils who took the tests and express our hope that pupils are offered a place in the school of their choice. Unfortunately as with any competition based on opportunity not everyone will be able to avoid some disappointment.

Breen Wilson transfer test 2004

Politicians, teacher unions and school principals are determined to end testing for transfer at 11

Perhaps an expression of thanks should be offered by parents & guardians for the provision of these “unofficial” or “unregulated” tests. Without the dedication, commitment, psychometric expertise, and adherence to available international standards all pupils would be attending comprehensive schools. This was the expressed aim and intention of successive Education Ministers and remains the aim of the Department of Education. This is particularly relevant given the collapse of the Executive and Assembly. Indeed it is remarkable that very little support for academic selection to grammar schools can be found in the media. This stands in stark contrast to the commercial greed of newspapers promoting the publication of schools league tables, transfer test practice booklets (while AQE provide all past papers at no cost) and just this week a transfer test guide suggesting, albeit inaccurately,  scores that will get your child a place in a particular school.

As has been widely reported the number of applications for the AQE and GL Assessment tests has continued to grow. In the academic year 2016/17 14,491 test entries were received. This resulted in 11,570 applications to grammar schools for the 8,743 places available. Therefore 3 out of 4 applications succeeded. This year will be similar.

BT PPTC single test

Victoria College, Belfast have operated the questionable and non-transparent practice of dualling (accepting applications from pupils who have taken both tests or either) yet in this Belfast Telegraph article the principal Patricia Slevin proposes a single test. The dualling practice will have undoubtedly created misclassifications, resulting inmany pupils being denied  a place in the school through the error of suggesting that two different tests, measuring different constructs, can simply be merged into one.

Single Test Impossible_20170128_0001_NEW

 

 

SingleTest Robinson 2012

 

The Parental Alliance has sought engagement with both AQE and GL Assessment, the test providers. GL Assessment refuse to engage citing their customer, PPTC, in a commercial contract. Since much is made of the fact that GL Assessment tests are free to pupils, who pays GL Assessment their charge for providing the multiple-choice, computer scanned and marked test? This raises the question of why this high stakes transfer test remains shrouded in secrecy. Recognised international standards suggest that pupils and their parents should be provided with exemplars of the questions likely to appear on tests. Neither the PPTC nor GL Assessment meet the standard. Indeed no past paper from GL Assessment has ever been published. The media have conspicuously not sought answers to this issue. Every year the BBCNI will broadcast a package on results day, invariably it will be from a school using the AQE test. No questions have been raised by the media about the dualling tables and their origin.

PPTC Practice Papers

There is no shortage of commercial Practice Papers available to purchase. Note the term “PPTC-style” All AQE past papers are made available to primary schools at no charge.

Another major distinction between the two tests is that GLA pupils only have one attempt at the examination. The time required for familiarisation, practice and the actual assessments in English & Maths exceeds that for many GCSE examinations.

Pupils taking AQE have three opportunities, allowing for a possible “off day” due to testing anxiety.

Details of GLA tests

 

Concerns have been raised this year about the use of content from the work of Charles Dickens in the PPTC  GL Assessment English paper. Most pupils may have difficulty distinguishing between the author of fourteen and a half novels and the contemporary magician pictured below. Charles Dickens was famously known for being paid by the word published. The version of David Copperfield featured has 745 pages of text. The two exams are contrasted for parents to discuss.

 

Cover to DC bookDC back coverDavid Cooperfield Magician

 

dc-page228.jpg

 

 

Contrast the above passage (randomly selected from the 745 pages in the book imaged above with the prose passage taken from an AQE 2017 paper. AQE tests are always unique; never repeated.

AQE 2017 Prose

 

The quest and motivation for a single transfer test must be critically examined by parents. In whose interests has the project been adopted? When the CCEA Transfer Test was ended, without a replacement examination in place, by Caitriona Ruane the prospect of compulsory comprehensive post primary schools loomed. A single (one provider) test was offered by AQE. This was quickly rejected by those mainly representing Catholic grammars. To be clear, the single test project is a manufactured crisis, clearly in the hands of politicians, civil servants, and school principals. Former DUP Education Minister kept the project alive by inviting Peter Tymms of Durham University to report on the matter. Tymms has a history with Northern Ireland primary school pupils via the now abandoned Incas assessments used in primary schools. (see blog search engine for articles).

The report from Peter Tymms was published by the Northern Ireland Executive Office close to the last day of the collapsed Assembly

Concerns raised with AQE joint CEO, Stephen Connolly about entering any process proposed by the Department of Education to work on a single test were met with a promise to express further reservations. It is understood that Stephen Connolly subsequently continued to meet with DENI officials

 

 

 

Screenshot_2

Another difficulty for parents is the fact that many grammar schools are not using academic selection for all pupils. Read the admission criteria carefully before applying to schools. It may be the case that your child is denied a place in favour of a pupil who did not take the tests. In the graphic below it is clear that Royal School Dungannon, Royal School Armagh and Sullivan Upper do not select 100% of their pupils by academic testing. Strathern School use bands rather than rank order of marks so that it will be impossible to reassure a child getting results today that their score will get them a place.

 

BT AQEGLA2018

 

The problem is even more acute when the dualling schools are examined. The obvious issue of the integrated schools pretending to be grammars can only be matched by those Catholic grammar schools which no longer use academic selection.

Dualling18 BT

Lagan and Slemish are not grammar schools. They were permitted by the anti-selection DENI to use 11-plus tests to select 35% of their pupils.

 

Wallace High School in Lisburn, another grammar school which uses bands to report test scores only selects 87% of year 8 pupils. The minimum score reported is 101. Wallace High admit 170 pupils to year 8 so a total of 22 pupils get places without use of academic selection.

During October 2017 Wallace High School attracted attention for restricting the number of primary 7 pupils allowed to sit AQE tests at the school. It became clear that this was not a matter of physical capacity but the willingness of teachers to make themselves available on Saturday mornings.

https://www.lisburntoday.co.uk/news/schools-are-criticised-over-their-handling-of-aqe-test-situation-1-8249218

Lisburn Star

 

Advice to Parents on AQE & GL Transfer Tests 2017/18

Tags

, ,

Undoubtedly, thanks mainly to media pressure, the results of the 2017/18 transfer tests will be the subject of conversations in families all over Northern Ireland this weekend and for months beyond. The Parental Alliance for Choice in Education wish to offer our congratulations to all pupils who took the tests and express our hope that pupils are offered a place in the school of their choice. Unfortunately as with any competition based on opportunity not everyone will be able to avoid some disappointment.

Breen Wilson transfer test 2004

Politicians, teacher unions and school principals are determined to end testing for transfer at 11

Perhaps an expression of thanks should be offered by parents & guardians for the provision of these “unofficial” or “unregulated” tests. Without the dedication, commitment, psychometric expertise, and adherence to available international standards all pupils would be attending comprehensive schools. This was the expressed aim and intention of successive Education Ministers and remains the aim of the Department of Education. This is particularly relevant given the collapse of the Executive and Assembly. Indeed it is remarkable that very little support for academic selection to grammar schools can be found in the media. This stands in stark contrast to the commercial greed of newspapers promoting the publication of schools league tables, transfer test practice booklets (while AQE provide all past papers at no cost) and just this week a transfer test guide suggesting, albeit inaccurately,  scores that will get your child a place in a particular school.

As has been widely reported the number of applications for the AQE and GL Assessment tests has continued to grow. In the academic year 2016/17 14,491 test entries were received. This resulted in 11,570 applications to grammar schools for the 8,743 places available. Therefore 3 out of 4 applications succeeded. This year will be similar.

BT PPTC single test

Victoria College, Belfast have operated the questionable and non-transparent practice of dualling (accepting applications from pupils who have taken both tests or either) yet in this Belfast Telegraph article the principal Patricia Slevin proposes a single test. The dualling practice will have undoubtedly created misclassifications, resulting inmany pupils being denied  a place in the school through the error of suggesting that two different tests, measuring different constructs, can simply be merged into one.

Single Test Impossible_20170128_0001_NEW

 

 

SingleTest Robinson 2012

 

The Parental Alliance has sought engagement with both AQE and GL Assessment, the test providers. GL Assessment refuse to engage citing their customer, PPTC, in a commercial contract. Since much is made of the fact that GL Assessment tests are free to pupils, who pays GL Assessment their charge for providing the multiple-choice, computer scanned and marked test? This raises the question of why this high stakes transfer test remains shrouded in secrecy. Recognised international standards suggest that pupils and their parents should be provided with exemplars of the questions likely to appear on tests. Neither the PPTC nor GL Assessment meet the standard. Indeed no past paper from GL Assessment has ever been published. The media have conspicuously not sought answers to this issue. Every year the BBCNI will broadcast a package on results day, invariably it will be from a school using the AQE test. No questions have been raised by the media about the dualling tables and their origin.

PPTC Practice Papers

There is no shortage of commercial Practice Papers available to purchase. Note the term “PPTC-style” All AQE past papers are made available to primary schools at no charge.

Another major distinction between the two tests is that GLA pupils only have one attempt at the examination. The time required for familiarisation, practice and the actual assessments in English & Maths exceeds that for many GCSE examinations.

Pupils taking AQE have three opportunities, allowing for a possible “off day” due to testing anxiety.

Details of GLA tests

 

Concerns have been raised this year about the use of content from the work of Charles Dickens in the PPTC  GL Assessment English paper. Most pupils may have difficulty distinguishing between the author of fourteen and a half novels and the contemporary magician pictured below. Charles Dickens was famously known for being paid by the word published. The version of David Copperfield featured has 745 pages of text. The two exams are contrasted for parents to discuss.

 

Cover to DC bookDC back coverDavid Cooperfield Magician

 

dc-page228.jpg

 

 

Contrast the above passage (randomly selected from the 745 pages in the book imaged above with the prose passage taken from an AQE 2017 paper. AQE tests are always unique; never repeated.

AQE 2017 Prose

 

The quest and motivation for a single transfer test must be critically examined by parents. In whose interests has the project been adopted? When the CCEA Transfer Test was ended, without a replacement examination in place, by Caitriona Ruane the prospect of compulsory comprehensive post primary schools loomed. A single (one provider) test was offered by AQE. This was quickly rejected by those mainly representing Catholic grammars. To be clear, the single test project is a manufactured crisis, clearly in the hands of politicians, civil servants, and school principals. Former DUP Education Minister kept the project alive by inviting Peter Tymms of Durham University to report on the matter. Tymms has a history with Northern Ireland primary school pupils via the now abandoned Incas assessments used in primary schools. (see blog search engine for articles).

The report from Peter Tymms was published by the Northern Ireland Executive Office close to the last day of the collapsed Assembly

Concerns raised with AQE joint CEO, Stephen Connolly about entering any process proposed by the Department of Education to work on a single test were met with a promise to express further reservations. It is understood that Stephen Connolly subsequently continued to meet with DENI officials

 

 

 

Screenshot_2

Another difficulty for parents is the fact that many grammar schools are not using academic selection for all pupils. Read the admission criteria carefully before applying to schools. It may be the case that your child is denied a place in favour of a pupil who did not take the tests. In the graphic below it is clear that Royal School Dungannon, Royal School Armagh and Sullivan Upper do not select 100% of their pupils by academic testing. Strathern School use bands rather than rank order of marks so that it will be impossible to reassure a child getting results today that their score will get them a place.

 

BT AQEGLA2018

 

The problem is even more acute when the dualling schools are examined. The obvious issue of the integrated schools pretending to be grammars can only be matched by those Catholic grammar schools which no longer use academic selection.

Dualling18 BT

Lagan and Slemish are not grammar schools. They were permitted by the anti-selection DENI to use 11-plus tests to select 35% of their pupils.

 

Wallace High School in Lisburn, another grammar school which uses bands to report test scores only selects 87% of year 8 pupils. The minimum score reported is 101. Wallace High admit 170 pupils to year 8 so a total of 22 pupils get places without use of academic selection.

During October 2017 Wallace High School attracted attention for restricting the number of primary 7 pupils allowed to sit AQE tests at the school. It became clear that this was not a matter of physical capacity but the willingness of teachers to make themselves available on Saturday mornings.

https://www.lisburntoday.co.uk/news/schools-are-criticised-over-their-handling-of-aqe-test-situation-1-8249218

Lisburn Star

 

Proof that there is a profound error at the heart of Carol Dweck’s Mindsets Project

Tags

, , , , , , , , , , , , , ,

Dr Hugh Morrison (The Queen’s University of Belfast [retired])

drhmorrison@gmail.com

The Brookings Institute recently published a study – entitled New Evidence that Students’ Beliefs about their Brains Drive Learning, by Susana Claro & Susanna Loeb – which drew heavily on the ideas of Stanford University Professor Carol Dweck.  The article appears to take Dweck’s research on so-called “Mindsets” at face value.

This  essay offers a proof that Dweck’s Mindset concept is simply wrong.

https://www.brookings.edu/research/new-evidence-that-students-beliefs-about-their-brains-drive-learning/

The central psychological concept in Dweck’s work is “belief”; according to Dweck, Mindsets are nothing more than beliefs.  Dweck never addresses, in any detail, the nature of belief.  She simply accepts the common-sense view that beliefs are entities carried in the minds/brains of individuals.  Despite a complete absence of any evidence in support of her claim that beliefs/mindsets are carried in the heads of individuals, Dweck has become something of a celebrity in psychological and educational circles; she has given “Ted-talks” and millions of dollars have been spent implementing her ideas in classrooms across the world.

It isn’t difficult to see why Dweck makes no attempt to analyse the psychological predicate around which all her research revolves.  If Dweck were to consult a standard text such as Peter Hacker’s The Intellectual Powers: A Study of Human Nature (published in 2013 by Wiley) she would find sixty pages of carefully-considered analysis directly addressing the notion of belief.  Alas, nothing contained in these pages supports her claim that beliefs are mental states or processes in the mind/brain of the individual.

Needless to say, such a claim is bound to have enormous popular support; when all is said and done, where else would one expect to find beliefs but in the head of the believer?  Alas, however, this appealing notion turns out to be wrong; beliefs are not objects in the mind/brain.  For example, on page 227 of Hacker’s book she would find the conclusion: “believing is not a state of mind,” and the title of section 8 on page 230 is “Why believing something cannot be a state of the brain.”  If beliefs cannot be construed as the property of the mind or brain of the individual who “has” them, then Dweck’s reasoning is vitiated.

In respect of the study published in Brookings the findings are invalidated if one cannot ascribe a definite mindset to everyone who took part in the study.  The purpose of the remainder of this essay is to offer a proof that Dweck is wrong about beliefs; mindsets, as Dweck construes them, do not exist.  The author hopes that the Brookings Institute will invite Professor Dweck to identify any shortcomings she can identify in the proof set out in the paragraphs below.

The proof centres on what Ludwig Wittgenstein referred to as “Moore’s Paradox.”  The philosopher G.E. Moore identified statements such as “It is raining, and I don’t believe it is raining” or “I believe it is raining and it isn’t raining,” as patently absurd.  The first part of these two conjunctions clearly contradicts the second part.  Indeed, all statements of the form “I believe p, and p is not true” are absurd according to Moore.

However, if one accepts Dweck’s claim that beliefs are carried in the mind/brain of the individual, the conjunction is perfectly intelligible because no contradiction exists between the two parts of the conjunction.  Why?  Because, on Dweck’s reasoning they refer to entirely different things: “it is raining” refers to the weather (something in the “outer” world), while “I don’t believe it is raining” refers to a mental state (something “inside” the head of the believer).

Carol Dweck’s only escape from this paradox is to accept that a belief is not an inner state or process in the mind/brain.  The following brief extract from Wittgenstein’s 1944 letter to Moore captures the depth of the conceptual flaw at the core of the Mindset thesis:

I should like to tell you how glad I am that you read us a paper yesterday.  It seems to me that the most important point was the “absurdity” of the assertion “There is a fire in this room and I don’t believe there is.” … If I ask someone “Is there a fire in the next room?” and he answers, “I believe there is” I can’t say: “Don’t be irrelevant.  I asked you about the fire, not about your state of mind!”

 

 

 

 

 

 

 

 

 

 

 

 

 

Why Randomised Controlled Trials in education are a waste of money.

Tags

, , , , , , , , , , , , , , , , ,

 

Ludwig Wittgenstein

Ludwig Wittgenstein

 

The National Foundation for Education Research (NFER) via their Education Trials Unit have issued a challenge to Professor Stephen Gorard  (Durham University) regarding his book on Randomised Controlled Trials in education. Unfortunately, both parties have it completely wrong on the matter as made clear by the writings of Ludwig Wittgenstein

Wittgenstein Brown Book

   The mathematics behind randomised trials only works if the attribute being studied is first-person/third person symmetric, like the properties of plants in Fisher’s Cambridge garden.  However, attributes of interest in education are, with few exceptions, first/third asymmetric.  So-called evidenced-based education research is nonsense, pure and simple. Those who fund it are wasting public money.  In his “Brown Book” Wittgenstein described the error of confusing symmetry with asymmetry in the ascription of attributes (via criteria) as “a disease of thought.”

 

To understand the full extent of NFER’s misunderstanding of the limitations of RCTs in education watching Ben Styles’ performance during the U Tube video referenced below may be illustrative.

 

Academic selection is not a problem for poor children, but the curriculum is and policy makers are responsible.

Tags

, , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

It’s the Curriculum, Stupid: why the Executive Office IliAD Report was a Waste of Public Money

Darwin 3

If the misery of our poor be caused not by the laws of nature, but by our institutions, great is our sin.” Charles Darwin

Introduction

This essay concerns the Investigating Links in Achievement and Deprivation (ILiAD) Report, volume three of which was placed in the public domain on September 5th 2017 by the Northern Ireland Executive Office (formerly OFMDFM). The research was commissioned from Queen’s University Belfast with Professor Ruth Leitch as principal investigator  in 2012.   In the next section, I will argue that the focus on the role academic selection plays in explaining the attainment gap between rich and poor is unjustified.  I will identify the real culprit, namely, the curriculum and demonstrate that my analysis is confirmed by the international literature.  Furthermore, given the Curriculum Vitae of the Queen’s University academics, why was £0.3 million awarded to individuals without detailed knowledge of the literature on curriculum, and little or no direct experience of the practicalities of the classroom.

 

Iliad vol 3

Where are the missing Volume 1 & Volume 2 including the literature review?

 

In the final section, I demonstrate that my claim that the culprit is the curriculum (a Revised curriculum foisted on every child in Northern Ireland many years ago), rather than academic selection, is confirmed by the international literature.  Given the sheer scale and quality of this international research – a highly visible century-long literature review and the most sophisticated and extensive experiment in the history of education – how could the Queen’s academics, even with their limited expertise, not have known about the consequences of skills-based curricula like the Northern Ireland Revised curriculum for the life chances of the poor?  Why were those who funded the ILiAD project not aware that the solution, for which they paid £0.3 million from the public purse, already existed?

When is a School of Education not a School of Education?

The BBC’s presentation (on both radio and television) of volume three of the ILiAD Report gave the clear impression that academic selection was a culprit, if not the culprit, in the attainment gap which separates rich and poor schoolchildren in Northern Ireland.  The ILiAD report (a product of The Queen’s University of Belfast’s “School of Education”) has always been surrounded by controversy both in respect of its eye-watering cost to the public purse, and the fact that it has been embargoed by the Executive Office for so long.

Given this background, it is intriguing that no one at the BBC thought to question the motives of the civil servant(s) who triggered the publication of ILiAD now.  Moreover, William Crawley, host of the BBC Radio Ulster Talkback programme afforded Tony Gallagher, professor at the School of Education, the opportunity to interpret the ILiAD findings pretty-much unopposed.  William Crawley failed to raise with Professor Gallagher how the ILiAD report could be construed as a School of Education report when none of the Queens authors has a background in education, let alone experience in the classroom.

The authors comprised two psychologists, one anthropologist, one “built environment” academic, one sociologist, and a researcher working in conflict resolution/transformation.  It is interesting to note, in passing, that of the many professors of education employed by the School of Education, none have experience of the post-primary classroom.  William Crawley’s claim on a previous Talkback programme that Tony Gallagher had expertise in three of the fundamental facets of education is bound to puzzle those who are aware that the professor has never even been employed by a school.

Economist Scotland

But here is where the BBC, professor Gallagher and the authors of the ILiAD report are most at fault.  The conclusions of the ILiAD report, and the central message of the BBC’s coverage, is undermined by this simplest of counterexamples: Scotland has a similar attainment gap (see Economist 27.08.2016)  to Northern Ireland, but that gap cannot be accounted for by academic selection because Scotland doesn’t have grammar schools.  Moreover, this simple counterexample points the way to the real source of the attainment gap – something Scotland and Northern Ireland have in common – a progressivist curriculum.  The Economist study attributes Scotland’s difficulties not to selection but to its progressivist curriculum.

We have now arrived at an explanation of the attainment gap which accords with all the extant high quality international research but is entirely at odds with the reasoning set out in ILiAD: Northern Ireland and Scotland share a common problem in that policy-makers (supported by “educationalists”) in both countries have adopted models of curriculum known to be damaging to the life chances of the poor.  The true culprits are not the grammar schools but former CCEA leaders Gavin Boyd and Carmel Gallagher who developed a progressivist “Revised” curriculum which required schools to adopt incoherent notions such as “learning to learn,” Assessment for Learning and so-called thinking skills.

 

Gallagher Trojan Horse

 

It’s the curriculum, stupid

OFMDFM (now The Executive Office) made a profound mistake when it supported the ILiAD approach to the problem of the achievement gap between rich and poor.  OFMDFM funded ILiAD to investigate a question that had already been answered in what has been described as the largest educational experiment ever conducted: Project Follow Through.

It is important for the reader to appreciate how Project Follow Through dwarfs the ILiAD project both in scope and ambition.  The generalizability of Project Follow Through’s findings is far beyond anything ILiAD could offer. Project Follow Through focused on the classroom and sought to identify the teaching method that would raise the academic standards of the poor to middle class levels.  The most successful method?  Traditional teaching (so-called “direct instruction” (DI)) stood head and shoulders above all other teaching techniques.

With a price tag of a fraction of a billion dollars (a lot of money in the 1960/70s), Ian Ayres summarises Project Follow Through as follows: “Concerned that ‘poor children tend to do poorly in school,’ the USA’s Office of Education and the Office of Economic Opportunity sought to determine what types of education models could best break this cycle of failure.  The result was Project Follow Through, an ambitious effort that studied 79,000 children in 180 low-income communities for 20 years.”

Traditional teaching methods (DI) outperformed all its rivals in getting disadvantaged children to perform at middle class standards.  Richard Nadler writes: “When the testing was over, students in DI classrooms had been placed first in reading, first in math, first in spelling, and first in language.  No other model came close.”  Independent evaluations were subsequently carried out by the American Federation of Teachers and by the American Institutes for Research, with the same conclusions. The message for Northern Ireland is simple: If one wants to address the ill-effects of poverty, scrap the Revised Curriculum.

Curriculum models categorised as “learning-to-learn” performed very poorly in Project Follow Through.  Given that Northern Ireland’s Revised Curriculum has ‘learning-to-learn’ principles at its core, the explanation for the underachievement of disadvantaged children (Catholic, Protestant, whatever) isn’t hard to find.  The missing link between poverty and underachievement could have been identified without giving a penny of public money to “educationalists” at Queens.  Individuals like Gavin Boyd – who now commands one of the highest salaries in the public sector as head of the new Education Authority – have bequeathed to children who live in poverty a dysfunctional and damaging curriculum.

Jeanne Chall Academic Achievement

In her 1990 book The Academic Achievement Challenge, the distinguished Harvard academic Jean Chall conducted a detailed study of a century of research on the effective teaching of disadvantaged children, finding no evidence for the efficacy of methods which depart from traditional teacher-centred methods.  On page 171 Chall writes: “Whenever the students were identified as coming from families of low socioeconomic status, they achieved at higher levels when they received a more formal, traditional education.  Overall, while the traditional, teacher-centred approach produced higher achievement than the progressive, student-centred approach among all students, its effects were even stronger for those students who were less well prepared.  The teacher-centred approach was also more effective for students with learning disabilities at all social levels.  overall, the research showed that at-risk students at all social levels achieved better academically when given a more traditional education.”

On page 182, Chall draws this conclusion from a century of published evidence: “The major conclusion of my study in this book is that a traditional, teacher-centred approach to education generally results in higher academic achievement than a progressive student-centred approach.  This is particularly so among students who are less well prepared for academic learning – poor children and those with learning difficulties at all social and economic levels.”