In late October 2009 it was widely reported that a computer error in primary school assessment results led to teachers being given false information about the performance of their pupils. The teachers had obligingly also given the false information to parents.
This announcement came just weeks after another blunder in which pupils at more than 300 schools were told they had received incorrect test scores.
Education Minister Caitriona Ruane was forced to admit that the latest lapse highlighted to schools the danger of trying to introduce a system of testing that had not been trialled. She neglected to take responsibility for her department’s approval and funding of the InCAS system. The recommendation to use it came from CCEA in the first place.
The InCAS (Interactive Computerised Assessment) came under fire yesterday when it was admitted that a second mistake had been detected in the standardised scores that had been sent to schools. The InCAS system is used to evaluate whether children are at the expected standard in subjects like maths and reading. It is a diagnostic instrument yet has been prescribed by the DENI.
These standardised scores were claimed by some schools to determine how pupils perform relative to other pupils and were not intended for reporting to parents. InCAS reports were to be used to undermine tests of numeracy and literacy such as the 11-plus transfer test. InCAS was promoted as part of a Pupil Profile aimed to dissuade parents from allowing their children to take unregulated tests for admission to grammar schools Nonetheless parents are entitled to access all information held on their children and many obtained InCAS results from schools including some via FOI requests. It soon became clear that the information was unreliable.
It is claimed that the mistake at CEM InCAS was noticed when principals at nine different schools contacted the examinations body CCEA and raised concerns over some of the information they had received. The CCEA said it was contacting all the schools involved to inform them of this further error and issued an apology.
CCEA said it had no plans to stop the contract with the centre for Evaluating and Monitoring (CEM) at the University of Durham, which administers the InCAS system.
“We want to make sure that it does not undermine the confidence in the InCAS system as it is very valuable for teachers and schools in assessing the potential of the pupils,” said Neil Anderson, chief executive at CCEA.
“We have been very pleased with the service they have provided us with until now and we have had their assurances that the error has been detected and fixed.”
Ms Ruane, the Education Minister attempted to deflect from the failed InCAS software. She said:
“These incidents should highlight to all schools the danger of trying to implement a system of testing young children using breakaway tests that have not been validated or trialled.”
An error relating to the computer based InCAS assessment tool for general maths led to incorrect |results being sent out to more than 300 primary schools across Northern Ireland.
The assessment was supposedly used to support schools in identifying pupils’ strengths and areas where they are having difficulties. The results had already been processed and sent out to parents by 79 of the schools affected.
CEM at the University of Durham claimed the error was caused by a blip in the computer programme and CCEA were assured that this had been fixed. The ending of the contract is just another example of the many failed initiatives floated by CCEA at great taxpayers expense.
While confidence in examination results plummet school inspectors promote self evaluation.
September 2, 2010
Evidence of the inability of the Department of Education, Northern Ireland to achieve real improvement in numeracy and literacy outcomes comes via the press release by the DENI’s Education and Training Inspectorate (ETI).
The final section of the release states:
“A key element in any self-evaluative process is the quality of evidence on which the evaluations are based. A range of sources can be used in this process, including first-hand observation of learning and teaching and a thorough examination of the assessment data that the school or organisation collates. The more rigorous and honest such evaluations are, the more useful will be the outcomes, and these should be used as the basis for securing more effective teaching and learning and in raising the standards achieved by learners.”
Perhaps Stanley Goudie has missed news of the ongoing investigation by Ofqual of CCEA’s failure on A-level Chemistry results. The evidence quality is very high of CCEA’s failure in a quantative measure. 151 pupils were given the wrong results.
What has not be addressed by the local media is the obvious question; “How can wrong marks be given for a multiple choice question paper?
Answer: A software/programming failure.
Does CCEA have other recent evidence of such failure? Yes.
- the Incas Pupil Profile from the CEM at Durham University spat out inaccurate results last year resulting in ministerial, CCEA and University of Durham apologies and investigations. The result: a further commitment to Incas.
Instead of rejecting this flawed instrument the DENI have further cemented its use. One of the major proponents of Incas is Prof John Gardner of QUB. His Assessment for Learning group are advocates of the ETI self assessment programme. Professor Gardner was also the man behind the attacks on the 11-plus transfer test. His campaign and advice to the DENI have simply resulted in a deregulation of the tests.
When will the politicians actually hold to account the educationalists whose conduct of education reform in Northern Ireland is replete with flaws and failure?
Incas and Sparkebox: FOI revelations “without them knowing there were any problems”
February 9, 2010
For those parents suggesting that Incas assessment scores should be used to determine admission to grammar schools instead of objective testing of attainment- a note of caution from PACE.
When the problems surrounding the expensive CEM software failures were first raised in late 2009 PACE obtained confidential communications from the Incas team to CCEA using the Freedom of Information Act 2000 ( FOI).
“What we would like to do is to extract the relevant files from the schools’ file servers automatically without having to contact the schools directly. We understand that C2k can access files on the schools’ file servers directly. If we can have these files forwarded to us, perhaps through some secure ftp method, we can update schools’ feedback without them knowing there were any problems.”
“It is also worth noting that the subscale scores are for information for teachers, not for parents.”
The Department of Education also cautioned against Incas.
The deputy secretary of the Department of Education, Robson Davison, wrote to the chairs of the board of governors and principals of schools setting the new entrance exams.
“It is important that I remind schools that such practice would be an inappropriate use of outcomes from an assessment tool that was neither designed nor intended for such a purpose.”
He also told them there were problems with the scoring awarded to pupils in InCAS because of a computer error.
“The department’s advice is that on no account should grammar schools consider using InCAS assessment outcomes reported to parents to inform decisions on selection.”
So while parents are crying out for accurate information on the results of transfer tests they are being kept in the dark by unaccountable educationalists profiting from testing activities.
and a further concern ….
Sparkebox link to DENI and C2k
http://www.deni.gov.uk/ministers-statement-sparklebox-080210-english.pdf
Sparklebox was originally blocked in late 2009 by councils. Concerns over the sites security were cited in an email to teachers explaining the block. Originally many teachers were confused and some angered over the block, as it was a site that many relied on for essential classroom resources. Kent County Council issued a statement confirming the block, “we feel it right to block the site centrally until more information is available and review whether this site should be blocked permanently after consulting schools and other sources.”
It seems that the Minister, Caitriona Ruane, her Department and computer infrastructure team, C2k must not have accessed the internet in early January otherwise they could not have missed the following post http://www.neowin.net/news/main/10/01/14/uk-schools-blocking-sparklebox-as-owner-is-a-paedo
Are you confident in your child’s security with computers at school or reassured by the Minister’s statement?
Incas Tests: software altered at the request of CCEA and DENI before the system failure.
November 18, 2009
When PACE were contacted by parents concerned about the results of Incas assessments given to them during parent/teacher meetings they undertook to write to the CEM at Durham University, CCEA and the DENI.
The first response from CEM Incas staff was unexpected and raised further concerns. Queries on Incas were being directed to CCEA, the Northern Ireland Council for Curriculum Examinations and Assessment who had recommended to the DENI that the software use become mandatory for use in primary schools.
The initial correspondence from CEM at Durham University stated:
________________________________________________________________________________________________________________
“If you have any specific queries about InCAS, Paul Wright at the Council for Curriculum Examinations and Assessment would be the best person to help you, pwright@ccea.org.uk. “
Amanda Mayman, Project Coordinator, PIPS & InCAS Projects Tel: 01913344220 Fax: 01913344180
__________________________________________________________________________________________________________________
Since CCEA were not understood to be the software owners this re-direction of concerns to CCEA seemed inappropriate and an attempt to downplay rising concerns.
Further contact with CEM at Durham University resulted in the following received on November 3rd, 2009
_______________________________________________________________________________________________________-
Dear Mr Elliott,
Amanda Mayman forwarded your query to CCEA last week and someone from there will be contacting you to discuss your questions.
Dr Christine Merrell, Acting Primary Director,
CEM, Durham University,Mountjoy Research Centre, Durham, DH1 3UZ. Tel. 0191 3344226
________________________________________________________________________________________________________
The following correspondence was received by PACE from CEM at Durham University on November 17th, 2009
_______________________________________________________________________________________________________________
Dear Mr Elliott,
Many thanks for your interest in InCAS. Your email has been passed on to me to respond to.
CCEA are not the owners of the InCAS software. InCAS was an existing assessment system that, upon the recommendation of CCEA, was adopted for use by the Western Education & Library Board and C2k, with the approval of the Department of Education.
The terms of the grant of license for InCAS are as follows:
’Installation and Use. Durham University grants you the right to install and use multiple copies of InCAS on your computers running validly licensed copies of the operating system for which InCAS was designed [e.g., Windows(r) 95; Windows(r) 98; Windows NT(r) etc.] within your institution.
Neither information distributed throughout the year for use with InCAS nor information generated by InCAS (be it numerical or graphical) may be published in any manner or form outside your institution without the express permission of Durham University.’
While InCAS was an existing assessment, under the terms of the contract we undertook to adapt it to better fit the requirements of the curriculum in Northern Ireland. With the introduction of this bespoke version of InCAS in Northern Ireland – including the provision of training to teachers – arranged by CCEA, it was agreed that all queries relating to the use of InCAS in Northern Ireland schools would be best dealt with by CCEA themselves.
I hope this provides a satisfactory answer to your questions from 27 October but please let me know if you have additional questions. However, as indicated above, CCEA may be able to provide a more complete response.
Mark Wightman, Operations Manager, CEM
_________________________________________________________________________________________________________________
It now becomes clear that CCEA had contracted to take complete control of the Incas assessment system that they recommended for procurement to the DENI, CCEA sought alterations to the software and when the software failed CCEA took control of the information provided to schools, principals, teachers, parents and the media via their press releases downplaying the extent of the problem. In summary CCEA took absolute control of the information management around a product owned by a third party. It is remarkable that the local media swallowed the explanation whole. Perhaps they were preoccupied with unregulated tests instead of the flawed taxpayer-funded Incas Assessment system.
GCSE questions “easier than old 11-plus”
November 15, 2009
The Royal Society
In a shocking revelation in the Sunday Times http://www.timesonline.co.uk/tol/life_and_style/education/article6917210.ece a report was passed to the government in July, only a few weeks before GCSE results were released, when Balls accused critics of exam standards of “rubbishing the achievements of young people”.
The government-backed study has undermined claims by Ed Balls, the schools secretary, that GCSE standards have been maintained, by showing that some science papers include questions so simple that they require no knowledge of the subject.
Sir Martin Taylor, vice-president of the Royal Society, Britain’s foremost scientific body said:
“If we have science exams that do not test the quality of mathematics needed to do good science, or if we have questions that do not require scientific knowledge to answer them, then we do not have an examination system that is fit for purpose.”
The findings also demonstrated that examination boards were allowing scientifically wrong answers to be marked as correct and that maths was only being tested “in a very limited way”.
"No knowledge required"
Sir Cyril would be aghast at the policy of the Department of Education in Northern Ireland which has removed the statutory requirement for pupils to take GCSE English/Irish or Maths but has made it compulsory for primary school pupils to be assessed using a flawed assessment system form the CEM Centre at the University of Durham.
11-plus practice papers
Richard Pike, chief executive of the Royal Society of Chemistry, said the maths paper was easier than 11+ practice papers from 1960 with which he had compared it.
“That is an extraordinary indictment of the current UK education system,” said Pike. “We cannot continue to live the lie of ever-increasing standards while businesses struggle to recruit staff with numeracy skills, or who understand the quantitative basis of science.”
Richard Pike, chief executive of the Royal Society of Chemistry
How CCEA let down parents on Incas
October 29, 2009
Following the unprecedented admissions of failure with Incas by the DENI, CCEA and the CEM at Durham University it may help parents to see how the false reassurances from the Project Management Board from CCEA were not matched by reality.
No software errors have been reported
Incas assessment tests fail Northern Ireland pupils
October 22, 2009
According to the DENI and CCEA parents of primary school pupils were assured on Incas assessments that;
These assessments provide parents and teachers with valuable information about a child’s progress in these important areas of the curriculum.
In an attempt to minimise the scale of the failure of Incas CCEA and the DENI have initiated a hastily organised cover-up of the expensive University of Durham’s CEM flop. Two press releases were issued by CCEA painting a picture of minor effect and maximum concern. Nothing could be further from the truth given that parents are expected to rely upon these assessment results as a main measure of attainment in primary school. The assessments are mandatory but the checks on the validity and reliability of the software aren’t. Note the language of the/ reassurance offered by the DENI and CCEA.
No commitment to parents and pupils. Is there a pattern here from Minister Ruane’s Department of Education and CCEA?
Source: CCEA letter to school principals.
In summary, CCEA buy Incas software for Pupil Profiling and Parent Reporting. Incas falls over because of inherent problems with the software. CCEA were unaware of the extent of the problem but ask the vendor of the flawed product to give them an excuse to sell to parents and schools. CCEA issue press releases seeking to reassure parents and principals and even write instruction letters for principals and teachers to placate parents. CCEA now claim to have initiated checks on Incas and seek to reassure that all is well.
Parents simply ask: How can they have any trust in information provided by schools on their children’s attainment in learning at primary schools?
Burying bad news: a CCEA speciality
October 5, 2009
PACE gave warning about the use of the CCEA Pupil Profile and Incas tests on December 18, 2008
http://paceni.wordpress.com/2008/12/18/the-ccea-pupil-profile-and-incas-tests/
It has proven to be one of the most read posts on the site and has recently undergone a renewed level of interest.
Those visitors to the NICurriculum site attempting to open links to documents cited are in for a major disappointment. They will be met with 404 page not found messages. This is CCEA’s answer to nosy parents.
