Printable versions: Word version PDF Version

Response to ACME Questionnaire on Assessment of 14-19 Mathematics

Initial comments

The time scale for producing a report on this crucial, but extremely difficult and controversial, topic is very short. Furthermore the timing is unfortunate because of the need to take account of the recent report of the Post 14 Mathematics Inquiry and the imminence of Tomlinson's final report.

The Mathematical Association produced a position paper on assessment dated February 2002, a copy of which is appended to this response. This outlined our view that summative assessment is given a prominence which is detrimental to effective learning of mathematics, when there is a considerable body of evidence that the use of formative assessment in a variety of forms can greatly enhance learning and thereby raise standards.

We are pleased that ACME is giving attention to both forms of assessment. However, we greatly fear that the current interest in reviewing the mathematics curriculum and its assessment linked to the Smith and Tomlinson inquiries will result in too great a focus on summative assessment at the expense of formative assessment. Formative assessment is not widely understood and is not a widespread feature of current school mathematics teaching in many schools. Both summative and formative aspects of assessment merit extensive consideration and wide consultation if appropriate policies are to be put forward.

Our response to the questionnaire at this stage must inevitably be limited, but the Association will be very happy to be involved in any serious attempts to debate the two very different sets of issues in depth.

1. External/Summative Assessment/Assessment of Learning (AoL)

a) How would you suggest that the full range of mathematical skills could be more efficiently and effectively assessed than at present?

Timed written papers provide an objectivity that will not be available in any other way on a large scale in the foreseeable future. Online tests will have some role eventually, but that would need very substantial development work if they are not to have a narrowing influence. Current tests focus on routine procedures and problems. It is essential that fluency, deeper understanding and the ability to solve unstructured problems are properly assessed at every level. The very serious difficulty is that examinations would then become harder with a consequent pressure to reduce pass marks when it is actually more appropriate to set higher thresholds so that grades at any level indicated mastery. For example, grade A should not be awarded for less than 75% of the available marks on a paper and a mark significantly higher than 50% should be expected on a paper where the highest grade awarded is C.

b) Could on-going teacher assessments be used more in external assessment, while still maintaining objectivity in the examination process and without overburdening teachers?

It is difficult to see how this could be done in the foreseeable future, particularly given the pressures that would be created for teachers by the critical importance for both students and schools of attaining certain bench mark grades. Teachers are good at estimating grades, but the pressures to boost such estimates if they formed a formal part of the ultimate grade would cast massive doubt on the validity of results.

c) What are the influences of external assessment on curriculum, classroom practice and other aspects of teaching and learning?

There is widespread 'teaching to the test' at all levels with narrow concentration on practising past questions, strategic concentration on topics that are considered to be easier, mindless learning of rules at the expense of deeper understanding and little attempt to challenge students to think by providing suitably challenging unstructured problems. The potential value of coursework tasks is often lost because of the distortion of aims caused by the narrow focus of many marking schemes.

d) To what extent does modularisation affect the assessment of the full range of mathematical skills?

Modularisation results in examinations being too frequent and in students restricting their attention narrowly to the requirements of the next test rather than seeing the whole picture. On the other hand some students appear to benefit from being given short term targets which help them to work consistently throughout the course and provide them with frequent objective feedback.

e) How might the range of mathematical ability be assessed?

By assessing whether students can use mathematics to solve unstructured problems.

f) What are the advantages and disadvantages of having a number of Examination Boards setting papers at GCSE and at AS and A2?

A single awarding body with no choice of specification would ensure fairness and reduce wasteful paperwork caused by competitive pressures, but it would eliminate choice and discourage innovation. A single awarding body offering a choice of specifications has some attractions if it is subject to suitable monitoring.

g) What are the possibilities and the risks/limitations of using online, computer-based testing for mathematics assessment?

See a) above. There are dangers of reinforcing the current narrow focus on testing skills and procedures alone rather than the ability to solve unstructured problems where the student is required to produce extended reasoned arguments.

h) To what extent is the appropriate use of computer and calculator technology assessed under present arrangements? How might this be improved?

Current arrangements are not ideal: the testing of fluency needs to be separated from the ability to use calculators and computers sensibly in appropriate mathematical contexts. Understanding, explaining, proving and solving unstructured problems must be a central part of what is assessed.

2. Internal/Formative Assessment/Assessment for Learning (AfL)

a) What strategies are being used for assessment in schools/

There are an increasing number of schools which make good use of formative assessment, but the general picture is that many schools still rely on internal tests, often past examination papers, as their main means of assessment. The constant and excessive emphasis on summative assessment acts as a real barrier to giving greater emphasis to potentially much more productive formative assessment.

b) How, if at all, are these used to inform learning?

Marks are used to indicate attainment and potential. Teachers use information about weaknesses to determine revision priorities, but shortage of time makes it difficult to target individual weaknesses and the response is often to explain the idea again rather than to probe the real source of difficulties.

It is now widely recognised that giving marks is much less effective than just making comments in improving students' attitudes and performance. Even when marks are accompanied by comments students focus on the mark rather than the comment (see p.15 in the QCA materials referred to in c) below). Students need detailed advice about what they can do to improve: vague admonitions to 'work harder' or 'pay more attention in class' are ineffective.

One valuable recent development has been the use of various ways of getting immediate feedback during lessons using digit cards, traffic lights and individual whiteboards. This tend to be confined to lesson starters, but they can be used throughout a lesson to check that all students are making sense of what is going on and modifying the teaching in response.

Other techniques focus on the sort of questions that are asked - rich questions, which reveal misconceptions, and open ended questions give the teacher greater feedback on students' understanding.

c) Are you aware of the QCA support and guidance materials on Assessment for Learning? Have you any comments on them, or any evidence of how, if at all, they are used by teachers?

The materials offer useful advice, but do not seem to be widely known. Similar useful advice is to be found in Ofsted’s recent publication Good Assessment Practice in Mathematics (HMI 1477). There is a real need for practical and specific materials which integrate formative assessment with the teaching and learning of particular topics and with maintaining a continuous overview of a range of key skills and ideas. The 11 to 16 sub-committee of the Association is currently working on material of this nature for future publication.

d) Should formative assessment issues be an essential constituent of on-going teacher CPD programmes?

Yes! This is of vital importance, but who does it and what form it takes are serious questions in a situation where there is limited expertise.

3. Recommendations

If you were to make three recommendations relating to assessment at 14 - 19 mathematics, what would they be?

a) The pressures induced by targets and league tables at all levels should be removed because they cause widespread narrow 'teaching to the test'.

b) Summative assessment leading to certification should only take place compulsorily at the end of Key Stage 4 and at the end of the post 16 stage. If the high public profile given to the results of statutory assessments at the end of each of Key Stages 1, 2 and 3 cannot be eliminated they should be abandoned. High stakes tests do not encourage good pedagogy.

c) Substantial work should be done to clarify the way in which classroom teachers can use formative assessment to enhance learning in mathematics. Practical techniques should be widely propagated and encouraged through suitable large scale professional development and the provision of materials, including textbooks, which integrate these ideas into the learning of every topic.

15.03.04.

Appendix

The Mathematical Association

A Position Paper on Assessment

The purposes of a national assessment system should be to:

· certify the attainment of individual students;
· monitor national and school performance;
· improve the quality of teaching and learning.

We have serious doubts about the effectiveness of present assessment arrangements at all levels in achieving these purposes. Assessment takes two distinct forms – summative and formative – summarised in the table below. Both forms of assessment are necessary, but it is crucially important that the right balance is achieved between them.

Whilst there is some overlap between the two categories, they are essentially distinct. Current systems and many traditional practices, particularly in mathematics, give a disproportionate emphasis to summative assessment with little attention being given to the extensive possibilities of formative assessment for raising standards. Extensive work has been done by Paul Black and Dylan Wiliam of King’s College, London¹, who have gathered evidence concerning the effectiveness of formative assessment. One of the most significant pieces of evidence they offer challenges the effectiveness of much current assessment practice both at national and classroom level. Evidence suggests that students take little notice of teachers’ comments when they are accompanied by marks or grades. When marks are abandoned attitudes change and the focus is on how to improve learning rather than on how to get better marks.

Our concern about current assessment practices is centred on the unfortunate effects of the ‘teaching to the test’ which inevitably occurs when too much emphasis is given to marks or grades. Mathematics education suffers particularly because, with English and science, success in mathematics is rightly regarded as a critical component of education for the individual and for society at large. Current national testing policies and the accompanying ‘high stakes’ targets at all levels are skewing all classroom activity in the core subjects towards the short term goal of maximising test results. This is resulting in a serious narrowing of curricular aims because formal tests cannot assess all aspects of learning. Confining attention to what is readily testable has serious effects on the attitudes and morale of both teachers and students. The long term goals of mathematical education at all levels require the development of a knowledge, fluency and understanding which enables students to use and apply mathematical ideas with confidence and enjoyment.

Society does have to measure the performance of the education system and provide certification to indicate the attainment of individual students, but this should be done in ways which ensure that the long term goals of education are not compromised. To this end we would suggest the following:

• monitoring national and school performance requires a very much lighter touch and should be done much less publicly;
  
• national performance would be monitored much more effectively, and acceptably, if done by an independent body using sampling techniques, as used to be the case with the Assessment of Performance Unit (APU), together with using more qualitative evidence obtained through a sensitive system of inspection whose priority was to offer immediate and specific advice for improvement;
  
• targets for schools, LEAs and individual teachers based on proportions of students achieving particular levels or grades, and the related league tables, should be abandoned: schools and teachers need constant encouraging advice on how to improve the quality of students’ learning rather than exhortation and pressure to maximise test results;
  
• the National Numeracy Strategy and the Mathematics Strand of the Key Stage 3 National Strategy are having many beneficial effects on achievement and attitudes which are in danger of being jeopardised by the pressure to give undue emphasis to preparation for tests;
  
• the system requires a substantial change of emphasis from summative to formative assessment and from a focus on easily testable skills to embracing much wider and more long term goals;
  
• a change in national culture is needed so that education is valued much more for its intrinsic benefits to the personal development of the individual as well as for the benefits an educated workforce confers on society.

National testing at the end of key stages 2 and 3, GCSE and A level together with other post-16 qualifications, have a useful and acceptable place, subject to the provisos above, but additional national testing, mandatory or optional, beyond these 4 occasions in 13 years is excessive and counter-productive. It is far too early to give formal tests to students in Key Stage 1 – as a consequence some students are inevitably labeled as ‘failures’ at the age of 7. At Key Stage 2, the principle of national tests is more acceptable, but steps must be taken to ensure that the overall balance of the curriculum is not distorted by excessive time spent on narrow preparation for tests in the core subjects and that pressures do not result in students being accelerated to achieve higher levels to the detriment of deeper understanding and breadth of knowledge. At Key Stage 3 the national tests in mathematics were initially widely accepted, and were possibly contributing to raising standards and monitoring effectiveness across the system, but the advent of targets has led to a much narrower concentration on maximising results.

GCSE is widely accepted as an acceptable form of summative assessment at the end of year 11. In mathematics, attainment of a grade C is an important basic qualification which should be retained and not compromised by provision of alternatives whose credibility is less likely to be widely accepted, although it may take a different ‘mature’ form at the post-16 stage. We strongly support moves to a two tier GCSE so that grade C will be achieved through a common route by all students. We do not support pressures to encourage early entry for GCSE in mathematics for the same reasons given above for Key Stage 2. We would prefer to see some sort of double or extended award for very able students to be taken at the same time as the basic GCSE examination.

A level does have some shortcomings but, like GCSE, it is widely recognized as a measure of achievement and as such should be retained for the foreseeable future. AS level is much more problematic in the light of the serious and ongoing shortcomings of Curriculum 2000, which have been disastrous for mathematics. Whilst we fully support the principle of broadening the post-16 curriculum, it makes no sense to have high profile national examinations in three consecutive years. It is clear that this is resulting in excessive pressures on students and excessive time devoted to narrow preparation for examinations with a consequent lack of enthusiasm for the potential opportunities to develop broader interests and deeper understanding.

To summarise, the system as a whole, and mathematics in particular, needs far less emphasis on summative assessment – tests and associated targets – and far more emphasis on formative assessment where informal everyday classroom strategies are designed to improve the learning of individual students. Excessive formal testing distorts curricular aims and is demotivating and demoralising for both students and teachers.

7.09.02.

¹ Black, P. et al (2002) Working Inside the Black Box King’s College, London