The NFER blog

Evidence for excellence in education

Is ‘what works’ enough?

1 Comment

Our first post is by guest blogger Sue Rossiter, CEO of NFER

I’ve just read Ben Levin’s latest paper To know is not enough: research knowledge and its use (in the new online journal – Review of Education, 1: p2 to31). I am delighted to see the current and growing interest in better evidence and better use of evidence. Hardly a day goes by recently without something interesting arriving in my inbox relating to either or both of these, and this change in climate opens up new opportunities and potential partnerships with like-minded others. We can all achieve so much more if we work together.

I like Levin’s simple model of the three contexts – research production, research use and research mediation and, particularly, the overlap and inter-dependence of these three aspects in relation to what he calls ‘knowledge mobilisation’.

blog1a

Some thoughts on research production

As others have said already (see Ben Goldacre), knowing what works is only part of the answer. Knowing why, how and when something works is perhaps more important; and knowing what you need to do to try this out in your own school or service and being motivated and supported to do so are critical.

In the current climate there is a small danger of some seeing randomised controlled trials (RCTs) as the only worthwhile approach to gathering evidence. To do so would be to miss out on the other forms of evidence which, if used appropriately, can also enrich understanding of whether and how to do something differently. These include: evaluations which look more closely at how a change has been implemented and why it might have the effects seen; systematic research reviews and syntheses which bring together all of the evidence on a topic and indicate what weight can be attributed to different findings; even local practice examples which have some underpinning evidence of success can point the way to changes worth investigating further. (See also Daniel Willingham)

We need clear standards of evidence for all of these different types of research, and to understand what can be said – and with what certainty – from the different sources of evidence.

I doubt that we need new bodies charged with producing evidence or syntheses – there are plenty of organisations, including my own, that can (and do) do this well. But we do need to agree methods for organising and sharing evidence so that a teacher or school leader knows how and where to look for the best evidence on a particular topic.

Creating a rich evidence base, brought together in ways that make relevant information readily available is perhaps the easy part. We need to think much harder about how to increase the interest, motivation and capacity of so called research users to do something with it.

Some thoughts on research use

We would benefit from a shared view of what the important questions are – and for whom. And to be truly meaningful and relevant, the importance of such questions can only really be determined by answering more fundamental questions – what is our shared view of the purpose of education and hence what would an effective education look like in this country. Of course, these fundamental questions take us into political and philosophical domains but some shared – or at least broadly held – view must be important.

We need a shared and quite detailed view of how to interest and engage teachers and other practitioners in the value, use and creation of evidence – a theory of change. This is an under-researched area but a working version of a national model or theory would be of benefit – based upon what IS known already, and constantly updated as new evidence emerges of how good evidence can impact on practice.

NFER’s interest in this whole area is because we are a charity, charged with providing independent evidence that improves education and learning and hence the lives of learners. So as well as producing good evidence, we are keen to see that evidence plays its proper part in decision making about education, in Whitehall and in each classroom, school and service. Others who have invested large sums in research, such as the Education Endowment Foundation, are also keen to find ways to ensure that relevant findings are taken up at scale.

Unless there is a widely shared, holistic and self-sustaining approach to increasing educational research usefulness to leaders and practitioners, the current surge of interest in what works and in evidence-informed practice will not make a lasting difference. History tells us this would not be the first time.

This suggests at the very least some leadership or coordination is needed, given the many players involved. The Alliance for Useful Evidence (NESTA) promotes the notion of more ‘What Works’ centres to improve the links between the supply of evidence and the demand for and use of evidence. How such centres coordinate and draw from the work of the many players in educational research is unclear, as is the relationship with representative user bodies such as any future Royal College of Teachers. The latter, for example, might take on a leadership role in developing research user capacity within the teaching profession.

Wherever this system leadership debate leads, we should grab the opportunity of this ‘Evidence Spring’ to create significantly better alignment between what researchers do and the needs of service managers, teachers and others who want to provide the best possible education and support for children and young people.

Author: thenferblog

National Foundation for Educational Research

One thought on “Is ‘what works’ enough?

  1. Better evidence ( which of course we all strive for) in the caring services sounds fine if governments act on it. For example we hear that management practices in the health service which focus on finance rather than on caring have been problematic, so the government institutes a new regime for trainee nurses not management. Its so much easier to criticise the workers than pay for the changes which often conflict with government ideology. For years we have had excellent evidence on the effectiveness of schemes to improve reading in schools and many other highly effective schemes in the public services but they run for a short while and then then funding is ended. Sadly professionals often do know what works (contrary to Ben Goldacres ill-informed comments) but sadly governments do not want to pay for them – they often involve employing caring, highly trained human beings in better ratios than currently and governments only want to allow this for parents or individuals who have the money to pay for one to one tuition or care. For example increasing the number of children a nursery nurse can look after goes against all the evidence about quality in education but it becomes policy for ideological reasons. Whereas randomised controlled trials might be great for testing the effect of drugs on individuals – they are of much less use in education. A classroom of thirty children and one teacher is a complex, dynamic, highly interactive environment where the quality of human relationships are central to effective teaching. The main challenge for the teacher is not really any specific ‘treatment’ in itself but the fact that they have to manage, assess, motivate and teach a large group of highly diverse individuals all at once. Thankfully doctors don’t have this problem and neither to children whose parents can afford them the better ratios of private education. In reception alone the reading age can vary by 8yrs with all the consequent problems of appropriate differentiation and management and motivation all going on simultaneously. All the preparation for this happens in the teacher’s own time after what is an emotionally and intellectually draining day job. If you want evidence look to the retention figures – even keen and able young recruits leave the profession very quickly before they can acquire the complex understanding that years of experience of teaching and learning can bring. For those of us who have always worked in education and given our lives to it is nothing less than infuriating for ignorant politicians and advisors with no experience of the area whatsoever to come up with apparently simplistic solutions to highly complex problems. In addition to giving teachers a largely impossible daily task they spend millions of pounds on a punitive inspection regime for which there is absolutely no evidence to show that it improves education in anyway whatsoever. Finland do not have an inspection regime and are higher in the eductaional league tables than us. That money could be spent in schools where children need more quality human support. The truth is that unless the ‘evidence’ that research comes up ith makes our services cheaper governments are not really interested. They really want to privatise our public services and are determined to denigrate both the insitutions and the researchers so that their friends can further invade the territory and make yet more money out of ordinary people. It is ordinary people who have born the financial brunt of the privatisation of cleaning services, transport, energy, pensions, old age care etc, and will doubtless suffer when education and health go the same way. I think for the sake of the future well-being of this country we should resist more forcefully the continuous propaganda that government churns out at every possible opportunity and reduce their air-space.

Leave a Reply