Why the Belfast Telegraph and the Irish News must correct their claim that St Dominic’s High School is Northern Ireland’s top grammar school.

Tags

Belfast Telegraph, DENI, Department of Education, Friends School Lisburn, Irish News, Rebecca Black, school league tables, Simon Doyle, St Dominic's Grammar School

 

 

In May 2016 the Belfast Telegraph published a league table of Northern Ireland grammar schools, based on the Advanced Level grades achieved by grammar school pupils in the school year 2014/2015.  The Belfast Telegraph’s editor failed to reply to correspondence asserting that fundamental errors in the design of her newspaper’s league table could result in unfair reputational damage to schools (see,  “Why the Belfast Telegraph and Irish News must set the record straight on grammar school league tables”).  In the recent past, the Irish News has published a similarly designed league table based on the 2015/2016 examinations.  This table has precisely the same design fault as the Belfast Telegraph table.  Once again, there is potential for reputational damage to a significant number of grammar schools.

 

It isn’t difficult to spot the error in the tables.  In assigning ranks to the various grammar schools, grades C, B, A and A* are treated as equal in value.  This is clearly wrong; a grade B represents a higher standard than a grade C, a grade A represents a higher standard than a grade B, and a grade A* represents a higher standard than an A.  Any instrument which treats a grade C the same as a grade A* simply cannot claim to measure academic excellence. The following scenario illustrates just how unfit-for-purpose these league tables are as tools for identifying high performing grammar schools: if every middle-sixth pupil in every school discipline across the entire grammar school estate were to simultaneously improve from C standard to A* standard, the tables published in these two newspapers would be completely powerless to detect any change whatsoever in standards.

 

Furthermore, both newspapers are, in effect, using highly questionable analysis to call into question the quality of teaching and learning in all “non-Catholic” schools.  Here are a few quotations from Rebecca Black’s Belfast Telegraph piece: “It is impossible not to be impressed at the consistently high performances of our top Catholic schools;” “By contrast, some of the best known non-Catholic grammars have slipped below the Northern Ireland average;” “If someone could bottle the ethos for success in these (Catholic) schools, they could run the world;” “Sean Rafferty, principal of St Louis Grammar, makes a very salient point in today’s Belfast Telegraph in calling for the Department of Education to examine what makes the top Catholic schools so successful, to learn the lessons and spread that magic across the school estate.”

 

Is the Belfast Telegraph not guilty of promulgating what  has been labelled “fake news”?  Based on highly dubious reasoning, Rebecca Black is distorting the debate on the relative efficacy of Catholic and non-Catholic education in Northern Ireland.  For Ms Black, the superiority of St Dominic’s High School (the school ranked first in both the Belfast Telegraph and the Irish News league tables) over, for example, Friends School Lisburn (ranked 12) is explained by the Catholic ethos of the former.  However, when a Grade Point Average approach is used to compute ranks, the order is reversed and Friends (a “non-Catholic” school) is the superior school!  The simplistic hypothesis that Catholic education is superior to that offered in non-Catholic schools is goes up in smoke when proper account is taken of the ordinal nature of examination grades: C, B, A and A*.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Advertisements

The AQE CEA and GL Assessment Test Results: Advice to parents: 2017

Tags

AQE, Belfast Telegraph, GL Assessment, Grade Point Average, Grammar School Exam Performance, Irish News, Methodist College, Simon Doyle, St Dominic's Belfast

All parents who have received a letter notifying them of the results of their applications to a post-primary school(s) would benefit from reading this post.

Why the Belfast Telegraph and Irish News must set the record straight on grammar school league table libel

Decisions made to apply to a particular grammar school based, even in part, on newspaper claims about exam performance are unsafe.

It is important to understand that the Grammar School Exam Performance lists ( league table libel) presented in the form used by the Belfast Telegraph and the Irish News represent, at best, a marketing tool used by the newspapers to increase sales in a declining print media environment.

In the Irish News of May 22, 2017 Simon Doyle boldly claims:

The Irish News performance lists are anticipated annually and some schools advertise their positions on their websites

Here is an example from St Dominic’s High School/Grammar School in Belfast.

https://www.stdominics.org.uk/news-archive/2017/5/22/top-performing-school-in-northern-ireland

Note the circular self-referencing between the Irish News and St. Dominic’s High / Grammar School. The school website is ambivalent Without a hint of irony Mr Doyle avoids acknowledging that  the Irish News  advertises  a version of league table libel. Discerning parents will note that there is no presentation of information to explain the methodology used to compile the tables but it is not difficult to suggest that if a C grade is treated exactly the same as an A* grade in value and all examination subjects treated as if they will equally passport a pupil on to a university course the wheels come off these meaningless tables whether they are ‘anticipated’ or not.

Surprisingly, Mr Doyle fails to mention the complete absence from his list of his own grammar school, the Methodist College, Belfast.

Where is the equivalent breakdown for St. Dominic’s High School, Belfast?

 

 

Why the Belfast Telegraph and Irish News must set the record straight on grammar school league table libel

Tags

All-girls grammar with 25% of Pupils on Free School Meals tops results list, Belfast Telegraph, Carol McCann, Dr Hugh Morrison, Irish News, league table libel, OECD Pisa, Rebecca Black, Sean Rafferty, Simon Doyle

Why the Belfast Telegraph and Irish News must set the record straight on grammar school league tables.

Dr Hugh Morrison

Immediately below is a short letter  sent to the editor of the Belfast Telegraph approximately one year ago.  It concerned a conceptual error in the paper’s A-Level league tables.  Despite repeated requests to make the public aware of their  error in the rank order, the letter never appeared in print.  On May 22, 2017 the Irish News published a league table generated by precisely the same flawed algorithm.

 

To be assigned a low rank in the Belfast Telegraph’s recently published league tables is likely to do little for the reputation of a school.  Given the potential reputational damage, it is vital to ensure that the numerical rank assigned to each school is meaningful.  Examination grades are reported on what is known as an ordinal scale.  There is an ordering of standards associated with the various grades: An A* grade represents a higher standard than an A grade, which in turn represents a higher standard than a B grade, and so on.  In the Belfast Telegraph’s league tables, ranks are computed by adding grades.  Alas, this produces meaningless numbers because arithmetic in general, and addition in particular, is impermissible for ordinal scales.  The Belfast Telegraph must make this error clear to its readers.

 

The Belfast Telegraph clearly believes that its league tables measure academic excellence.   It refers to the likely impact of cuts on “excellence,” schools which are “top of the class,” and “top performing” schools.  Rebecca Black writes: “Whether you believe the highest priority for a school should be academic excellence or not, it is impossible not to be impressed at the consistently high performance of our top Catholic schools.”  But are such inferences justified?  Are these league tables capable of even identifying excellent schools?

 

The Belfast Telegraph eschews the orthodox method used throughout the world by newspapers when publishing school league tables, the so-called “Grade Point Average” procedure.  This attempts to respect the fact that the scale of standards implicit in the various grades is ordinal by assigning a weight to each grade.  For example, with regard to the A-level league table, 12 points might be assigned to an A* grade, 10 to an A grade, 8 to a B grade, and 6 to a C grade.  However, in the Belfast Telegraph’s methodology, all four grades are assigned exactly the same value.  Unfortunately, a league table which treats a CCC profile as indistinguishable from a profile of A*A*A* cannot lay claim to distinguish schools on the basis of their academic excellence.

 

 

 

The Irish News story on CCEA and why the accounts do not add up.

Tags

Barney Magill, CCEA, FOI, Freedom of Information, KPMG, Simon Doyle, The Irish News

Last Friday May 27th 2011 the Irish News published a front page headline announcing that a “Schools body charters private flights” The detail suggested that over a period of just seven years CCEA held a series of conferences council meetings and workshops at hotels. At one conference £2,176 was spent on drinks, an amount that was four times that of the dinner expense. Many other instances could not be broken down because of lack of detail provided in response to requests for information made under the Freedom of Information Act 2000.

What was instantly remarkable was the absence of the name of the education correspondent, Simon Doyle from this and subsequent Irish News articles on the CCEA spending. Simon Doyle has graced the front page of the Irish News for many years with his education related exclusives and accounts including attacks on academic selection and grammar schools. Simon Doyle’s absence from this story is a story in itself.

The Irish News claimed the following day that their information had been provided by a concerned teacher, Barney Magill using FOI. Careful reading of the account on page 13 suggested that the DENI told him he would have to pay for the information requested. How the DENI became involved in a request for information from CCEA, the public body holding the information, has yet to be addressed by journalists.

Below is a typical response from CCEA to requests for detailed information. Compare this to the claim from the Irish News on Mr Magill’s request and ask why the difference in responses and , to date, why no copies of requests or responses have been made available in PDF form by the Irish News.

 

Not in the public interest

 Now it would hardly be the case that these revelations on CCEA come at a convenient time to allow the passage of the ESA Bills – or would it?

As ever there will be more to follow.

Additional concerns raised over PPTC GL Assessment tests.

Tags

GL Assessment, Irish News, NFER, Post Primary Consortium, Post Primary Transfer Consortium, PPTC, Simon Doyle

The Irish News   http://www.Irishnews.com    (Thursday November 18, 2010) has further highlighted concerns already raised by PACE NI on the GL Assessment tests to be taken by thousands of pupils seeking places in Catholic grammar and a few  schools this weekend. https://paceni.wordpress.com/2010/09/25/aqe-and-gl-assessment-testsattempts-to-confuse-parents-over-a-common-test/

GL Assessment have admitted that the questions used from their inherited bank of NFER questions could be recycled, i.e. resold elsewhere. Simon Doyle, education correspondent,  strangely fails to explain to Irish News readers the nonsense statement from GL Assessment that their unwillingness to release past papers was simply;

“In order to maintain the security and integrity of our tests”

 

GL Assessment spokeperson

never mind

 the important principle addressed in testing circles – the vaidity and reliability of high stakes tests.

 No mention at all  by the Irish News or GL Assessment or the Post Primary Transfer Consortium of any measures of the validity and reliability of any inferences to be drawn from the tests.

The PPTC are directly misleading parents in their claim that the PPTC GL tests are based on the Northern Ireland Key Stage 2 revised curriculum. Since the tests were not designed exclusively for Northern Ireland and predate the introduction of the revised curriculum such a claim is unwarranted and misleadng. This issue underscores the difference between the PPTC GL tests and those of AQE whose CEA test questions match the former 11-plus tests prepared by NFER.  It also highlights the nonsense claim published in the Belfast Telegraph regarding Victoria College’s use of the two tests to determine admission. There can be no equivalence between the two tests. Victoria College are simply pandering to parents from two different communities in order to keep their application numbers up.

Of course those adopting the inferior GL Assessment tests have persisted with the false claim  to parents that talks to agree one test are likely to achieve a positive outcome. There is absolutely no likelihood of that happening since it was the anti-academic selection PPTC who broke away from the AQE in the first instance with a GL Assessment test  to counter the fear that Catholic parents would abandon Catholic grammars if testing  was withdrawn. Parents should note the continued silence from Finbar McCallion of the GBA who appeared regularly over the summertime on the airwaves of the BBC promising resolution of the single test conundrum.

Parents may need reminding also of the PPTC’s failure to adhere to standards in relation to testing by their refusal to publish the specification of the GL Assessment test.  Prior to Saturday’s tests parents are advised to contact their school to demand a copy of the specification. Any refusal to provide same may provide a basis for appeal if entry to the school was ultimately rejected. Ronnie Hazzard of Ballymena Academy and the PPTC has persisted in his refusal to supply a copy of the specification to PACE in response to an FOI request. Can parents have any confidence that their childrens’ interests will be best represented by a school which fails to comply with the FOIA law?

One year on the Irish News catches up on massive ESA costs scandal

Tags

Education Committee, ESA, ESAIT, Frank Bunting, Irish News, Parental Alliance for Choice in Education, Simon Doyle

Last December The Parental Alliance for Choice in Education published an article highlighting the costs of the Education And Skills Authority (ESA). https://paceni.wordpress.com/2009/12/10/esa-the-real-costs/

On Saturday November 6th, 2010 the Irish News ran the front page headline;

£2 million waste of schools money on ESA staff.

Two million pounds  is a mere  fraction of the waste incurred by this stealthy body. No use by Simon Doyle, the Irish News Education Correspondent of the term “double-jobber” to describe Gavin Boyd the ESAIT chief executive and chief executive of the problem ridden exams board, CCEA.

 Frank Bunting northern secretary of the teachers union INTO rushed in to create a heirarchy of losers.

” Yes  it’s a waste of money but the bigger waste of money is that ESA has not yet been established…Everyone is a loser.”

Frank Bunting INTO

Perhaps Frank has forgotten his script to the Stormont Education Committee from March 25, 2009 when he said;

“in the interests of efficiency, establishing the ESA is basically a dream come true.”

Frank Bunting may need some remedial numeracy training and careful teaching of accounting principles.

Frank Bunting INTO

 

 

 

The AQE CEA and GL Assessment Test Results: Advice to parents

Tags

Age Standardised Scores, AQE test results, Belfast, Belfast Telegraph, CCEA, GL Assessment test results, Irish News, Northern Ireland transfer tests, Simon Doyle, Standardised Age Scores, Victoria College

UPDATE for 2011. Visit the page of February 4 on the cautionary tale of exam results by computer. https://paceni.wordpress.com/2011/02/04/exam-results-a-cautionary-tale/

Today saw the delivery of results for the two very different tests used to determine entry to grammar schools. The AQE test and the GL Assessment tests. The AQE results were delivered efficiently and effectively but the GL Assessment results encountered some difficulty with attendant stress for pupils http://www.belfasttelegraph.co.uk/breaking-news/uk-ireland/school-exam-results-delivered-late-14670533.html

The GL Assessment results delivered late

AQE CEA Tests

Three tests were offered with the best two scores counting. Most applicants took all three tests. The scores range from 55 to 145 with the average score set at 100.

Scores were adjusted to take account of age. The scores will be split into five bands or quintiles. The top band are marks greater than or equal to 113,  followed by 106-112, 98-105, 88-97 and,  less than or equal to 87. 

Within each quintile fall 20% of the scores obtained by the total number of candidates – so 20% of scores fall between 145 and 113, 20% between 112 and 106 and so on.

Only once all transfer applications have been processed and places allocated will a school be in a position to publish how the application of admissions criteria arrived at final admissions decisions.

There is not a direct relationship between each quintile and the traditional Transfer grades.  The advice of PACE  is that parents should use the Transfer form to put down their chosen schools in order of preference rather than trying to anticipate whether or not the score is sufficient to gain admission.

It was never the intention by AQE that transfer grades (A, B1, B2, C1, C2) would be applied as CCEA did with the regulated transfer test results. Anyone suggesting otherwise is misleading you and should be challenged as to the source of their information. Parents have been conditioned into talking about grades as if they represent some form of ranking but this is not the case. A score of  123 is not the same as 113.

Admission to a grammar school based upon strict rank order is the fairest method. However some grammar schools have adopted different criteria and it is here that parents and pupils are likely to run into difficulty and may be misled. Parents should check the admission criteria published by each of the grammar schools to which the pupil has applied for admission.

The second method claims to be similar to the old 11-plus (it is not)—  taking all pupils in first two score ranges (quintiles) and then using other non-academic criteria if oversubscribed within the other bands of scores.

The third method involves schools taking a proportion of available places e.g. 100 of 150 places based on scores alone. Then a pool of pupils is created (the size of which has been decided in advance) and the school will apply other non-academic criteria to all of this group to select for their final places. (social selection) Such schools undermine the principle of equality of opportunity and the purpose of the AQE CEA test. Parents should exercise caution in applying to these schools for admission since they may attempt to rely upon information from the primary school which is unreliable. Some schools have asked parents to access this information on their behalf. PACE recommend that such requests are refused.

Schools using the third method include Inst and bangor Grammar School.

The GL Assessment Tests

The GL Assessment tests are completely different. The questions were multiple choice and marked by computer. To highlight the misinformation campaign mounted by supporters of the GL Assessment campaign (remember the Catholic grammars and others refused the opportunity to use the AQE CEA tests) it has been reported that remarking can take place. If any educationalists can suggest how the computer can remark a test paper designed to be read by a computer and issue a different result PACE would be delighted to investigate this AI device. Claiming that GL Assessment remarks are “free” underscores the attempt at deception. It will recalled that the GL Assessment tests were” free” to applicants but as yet the identity of the philanthropists paying GL Assessment’s bill has not been made public. Interestingly no political party, educationalist or investigative journalist from the local media have even investigated the matter.

Today’s Irish News makes mention of Standardised Age Scores (SAS). Simon Doyle, their education correspondent must not have access to Google or other search engine because if he did he would have immediately discovered that the correct term is Age Standardised Score (ASS). If the Irish News education correspondent does not know the difference between an ASS and the SAS, readers should not rely on his counsel on important education matters It will be of great interest to learn where GL Assessment obtained their standardised scores from. Was it from template? Where did the norms come from?

Attempts to link the results of GL Assessment tests with CCEA grades only further complicates the issue but the most complicated situation of all arises at Victoria College, Belfast where pupils are to be admitted on the basis of results from both tests. The school Board of Governors was warned about the problems by PACE but refused to accept the advice. 

Victoria is the only school in Northern Ireland admitting pupils using the outcome of both AQE and GL tests. Dr Darrin Barr, the school’s deputy head, has said pupils will be admitted by considering the percentile ranking in the particular assessment sat. The AQE were queried by a school principal who was concerned whenever a percentile was given as a decimal such as .68  It will concern parents to know that numeracy problems extend to primary school principals not just the pupils. Ms Slevin the principal has rightly stayed silent.

In the case of an applicant who sat both assessments, the higher percentile rank will be used. The first pupils to be accepted will be those with a percentile rank of 60 or above.

If there are more within a band than places available, other non-academic criteria will be used.

This must be the most egregious use of an apples to oranges comparison. The two tests have no useful comparison properties.

If there are more within a band (B1 and B2 are five marks apart) than places available, other non-academic criteria will be used.

Age Standardised Scores

The purpose of age standardised scores is to convert a pupil’s actual score in the test (the ‘raw score’) to a standardised score. Standardised scores take into account the

pupil’s age in years and months, giving an indication of how each pupil is performing relative to other pupils of the same age. Age standardised scores do not affect the pupil’s test levels and are for optional use.

 

 

 

 

 

http://testsandexams.qcda.gov.uk/libraryAssets/media/Age_standardised_scores_Factsheet_for_schools.pdf

 

Parents with concerns over test result information should contact PACE at paceni@btinternet.com