p. 1
evidence-based practice in education where does hunch end and evidence begin too much is written and said about school improvement about improvements in teaching and learning with far too little attention to this question this book provides vivid discussion from distinguished protagonists and antagonists about what gets called evidence-based practice reading it all involved in education policymakers and practitioners alike can proceed more confidently professor tim brighouse london schools commissioner the movement towards evidence-based practice in education is as important as it is controversial this edited volume explores the arguments of leading advocates and critics of the movement the first part of the book begins with an explanation of evidence-based practice and discusses some of the ideas of its proponents in the second part of the book contributors examine the use of evidence-based practice addressing in particular the validity of the transposition from its original use in medicine to its application in educational policy the third part of the book looks at criticism of evidence-based practice and questions which have been raised about it for example regarding the transferability of research findings to practice the consequences of the move towards evidence-based practice for governance in education and how particular kinds of evidence may be privileged over others evidence-based practice in education is key reading for all educators educational researchers and policy-makers it is also recommended for professionals in medicine social work and psychology who are interested in evidence-based practice gary thomas is a professor of education at the university of leeds having previously worked as a teacher and an educational psychologist his interests are in inclusive education and research methodology and his books include the making of the inclusive school 1998 and deconstructing special education and constructing inclusion open university press 2001 richard pring is an emeritus fellow of green college oxford previous to this he was professor of educational studies at the university of oxford and lecturer at the university of exeter and the university of london institute of education he also taught in london comprehensive schools and was assistant principal at the department of education and science his book philosophy of educational research was published in 2000 conducting educational research series editor harry torrance evidence-based practice in education cover design kate prentice evidence-based practice in education thomas and pring isbn 0-335-21335-9 9 780335 213351 edited by gary thomas and richard pring
[close]
p. 2
evidence-based practice in education
[close]
p. 3
conducting educational research series editor harry torrance university of sussex this series is aimed at research students in education and those undertaking related professional vocational and social research it takes current methodological debates seriously and offers well-informed advice to students on how to respond to such debates books in the series review and engage with current methodological issues while relating such issues to the sorts of decisions which research students have to make when designing conducting and writing up research thus the series both contributes to methodological debate and has practical orientation by providing students with advice on how to engage with such debate and use particular methods in their work series authors are experienced researchers and supervisors each book provides students with insights into a different form of educational research while also providing them with the critical tools and knowledge necessary to make informed judgements about the strengths and weaknesses of different approaches current titles tony brown and liz jones action research and postmodernism gary thomas and richard pring evidence-based practice in education john schostak understanding designing and conducting qualitative research in education lyn yates what does good education research look like?
[close]
p. 4
evidence-based practice in education edited by gary thomas and richard pring open university press
[close]
p. 5
open university press mcgraw-hill education mcgraw-hill house shoppenhangers road maidenhead berkshire sl6 2ql email enquiries@openup.co.uk world wide web www.openup.co.uk and two penn plaza new york ny 10121-2289 usa first published 2004 copyright © gary thomas and richard pring 2004 all rights reserved except for the quotation of short passages for the purpose of criticism and review no part of this publication may be reproduced stored in a retrieval system or transmitted in any form or by any means electronic mechanical photocopying recording or otherwise without the prior written permission of the publisher or a licence from the copyright licensing agency limited details of such licences for reprographic reproduction may be obtained from the copyright licensing agency ltd of 90 tottenham court road london wip 9he a catalogue record of this book is available from the british library isbn 0 335 21334 0 pb 0 335 21335 9 hb library of congress cataloging-in-publication data cip data has been applied for typeset by refinecatch ltd bungay suffolk printed in great britain by mpg books ltd bodmin cornwall
[close]
p. 6
contents list of contributors 1 introduction evidence and practice gary thomas part 1 what is evidence-based practice 2 systematic reviews and the campbell collaboration philip davies 3 developing evidence-informed policy and practice in education judy sebba 4 systematic research synthesis david gough part 2 evidence-based practice in practice section a in education vii 1 21 34 44 5 between scylla and charybdis the experience of undertaking a systematic review in education richard andrews 6 teachers using evidence using what we know about teaching and learning to reconceptualize evidence-based practice philippa cordingley section b in medicine and allied fields 65 77 7 practice-based evidence michael eraut 8 reflections from medical practice balancing evidence-based practice with practice-based evidence ed peile 91 102
[close]
p. 7
vi evidence-based practice in education section c problems in practice 9 educational research philosophical orthodoxy and unfulfilled promises the quandary of traditional research in us special education deborah j gallagher part 3 questions 10 some questions about evidence-based practice in education martyn hammersley 11 the relationship between research policy and practice phil hodkinson and john k smith 12 making evidence-based practice educational john elliott 13 using action research to generate knowledge about educational practice harry torrance 14 conclusion evidence-based policy and practice richard pring references index 119 133 150 164 187 201 213 234
[close]
p. 8
list of contributors richard andrews is professor of education at the university of york and co-coordinator of the english review group for the eppi-centre he is the author of narrative and argument and the problem with poetry open university press teaching and learning argument cassell and editor of the impact of ict on literacy education routledge falmer he is associate editor of education communication and information and sits on the editorial boards of informal logic and english in australia philippa cordingley is the founder and the chief executive of the centre for the use of research and evidence in education curee as adviser to the dfes the national union of teachers the national college for school leadership the gtc and as chief professional adviser on research to the teacher training agency from 1995 to 2001 she has instigated designed and developed a range of strategies policies and support programmes to increase teacher interest in access to and use of research she is a board member of the education network ten a member of the national steering group for the networked learning communities initiative and a school governor philip davies is director of policy evaluation in the prime minister s strategy unit which is part of the cabinet office previously he was director of social sciences in the department for continuing education at oxford university and a fellow of kellogg college oxford philip was responsible with colleagues in the university of oxford medical school for developing the university of oxford master s programme in evidence-based health care philip is a founder member of the campbell collaboration and is on its international steering committee he is also a visiting honorary fellow of the uk cochrane centre john elliott is professorial fellow within the centre for applied
[close]
p. 9
viii evidence-based practice in education research in education which he directed from 1996 to 1999 at the university of east anglia he is well known internationally for his work in developing action research and has directed a number of collaborative classroom research projects with teachers and schools these include the ford teaching project 197274 and more recently the tta-funded norwich area schools consortium nasc on the `curriculum and pedagogical dimensions of student disaffection 19972001 he is currently an advisory professor to the hong kong institute of education and a consultant to the hong kong government on the strategic development of its curriculum reform proposals michael eraut is a professor of education at the university of sussex his research over the last decade has focused on the nature of professional knowledge and competence the role of tacit knowledge in professional practice how professionals and managers learn in the workplace and factors affecting learning in professional apprenticeships his recent research projects have addressed the training of junior doctors the development of competence and judgement in postgraduate medical education the vocational training of clinical and biomedical scientists and how nurses learn to use scientific knowledge he is editor in chief of a new blackwells journal learning in health and social care deborah j gallagher is professor of education at the university of northern iowa her research interests centre on the philosophy of science as it pertains to research pedagogy and policy in education and special education this work focuses on how choices of methodological and conceptual frameworks affect the possibilities of achieving equitable and inclusive schooling for students labelled as having disabilities among other recent publications she is the lead author of a book entitled challenging orthodoxy in special education dissenting voices love publishing company 2003 with co-authors lous heshusius richard iano and thomas skrtic david gough is reader in social science and deputy director of the social science research unit and its eppi-centre institute of education university of london previously he was senior research fellow at the university of glasgow and professor of social welfare at japan women s university near tokyo his main areas of research interest are the implicit social policy of social interventions for children and families and methods of systematic research synthesis to address all policy practice and community questions he is editor of the journal child abuse review martyn hammersley is professor of educational and social research at the open university much of his work has been concerned with the methodological issues surrounding social and educational research he has written several books with paul atkinson ethnography principles
[close]
p. 10
list of contributors ix in practice 2nd edition routledge 1995 the dilemma of qualitative method routledge 1989 classroom ethnography open university press 1990 reading ethnographic research longman 1991 what s wrong with ethnography routledge 1992 the politics of social research sage 1995 with peter foster and roger gomm constructing educational inequality falmer 1996 taking sides in research routledge 1999 and educational research policy-making and practice paul chapman 2002 phil hodkinson is professor of lifelong learning at the university of leeds he was founder director of the lifelong learning institute which works to interrelate research expertise with that of policy-makers and practitioners he has considerable experience of qualitative case study research and has written about hermeneutical and interpretivist methodologies he has regularly engaged with policy-related research in vocational education and training workplace learning and career progression in one major project on-going at the time of writing he is co-leader of a mixed team of university and practitioner researchers exploring ways in which research can improve teaching and learning in further education ed peile is a professor and head of the division of medical education at warwick medical school university of warwick previously a general practitioner he started the aston clinton surgery in 1983 he has been involved in postgraduate medical education for 16 years as an associate advisor at oxford postgraduate medical deanery he directed the higher professional education programme as well as the new teachers course in one-to-one teaching in primary care in undergraduate education he was associate director of clinical studies at university of oxford where he is hon senior clinical lecturer in primary health care his research interests are in interprofessional education and in process and outcome in gp registrar training richard pring was professor of educational studies at the university of oxford from 1989 to 2003 he was previously professor of education at the university of exeter lecturer in curriculum studies at the university of london institute of education a teacher in london comprehensive schools and assistant principal at the department of education and science his book philosophy of educational research was published by continuum in 2000 his main areas of academic and professional interest are in the philosophy of education especially in relation to vocational education and training religious education and comprehensive schooling he retired from being director of the department of educational studies at oxford in may 2003 and is now an emeritus fellow of green college oxford judy sebba is a professor in education at the university of sussex previously she was senior adviser research standards and effectiveness
[close]
p. 11
x evidence-based practice in education unit department for education and skills dfes where she was responsible for developing the research strategy and quality of research relating to schools she supports the national educational research forum which is developing a research strategy for education and manages the eppi-centre on behalf of the dfes which is developing systematic reviews in education she was previously at the university of cambridge where she was involved in a number of projects on evaluating the use of school effectiveness grants post-inspection action planning in special and primary schools school improvement and inspection and special needs john k smith is professor of education at the university of northern iowa among his numerous publications are two books the nature of social and educational inquiry empiricism versus interpretation and after the demise of empiricism the problem of judging social and educational inquiry gary thomas is a professor of education at the university of leeds having previously worked as a teacher and an educational psychologist his interests are in inclusive education and research methodology and his books include the making of the inclusive school routledge 1998 and deconstructing special education and constructing inclusion open university press 2001 harry torrance is professor of education and head of research in the institute of education manchester metropolitan university he was formerly professor of education at the university of sussex where the research on which his chapter is based was carried out in collaboration with dr john pryor.
[close]
p. 12
1 introduction evidence and practice gary thomas what is evidence none of the contributors to this book denies the importance of evidence in shaping and enhancing practice at issue is not the significance of evidence but its nature and its value contingent on that nature at issue are the potency and value ascribed to certain forms of evidence in supporting propositions that arise in educational practice many kinds of evidence are available to practitioners in support of ideas and propositions that arise as part of their work from observation from documents from the word of others from reason or reflection from research of one kind or another it is the respect and worth credited to these that i shall explore in the first section of this introduction and i shall hinge that exploration around notions of evidence in various spheres of inquiry evidence may take different forms and be valued differently in different places in the legal system in the natural sciences in medicine in the humanities those who promote evidence-based practice in education are not seeking evidence in the way that an historian might seek evidence of the existence of factionalism in england at the fall of cromwell in 1640 neither will they be looking for evidence in the way that a physicist might seek evidence for the existence of higgs boson.1 likewise they are unlikely to be satisfied with the kind of implicit and eclectic evidencegathering involved in the accumulation of the tacit knowledge of which polanyi 1969 speaks they will be following a path of reasoning that encourages the seeking marshalling and dissemination of evidence of a particular kind different from these others and it is the character and distinctiveness of this particular kind of evidence that i shall principally examine in this comparative overview.
[close]
p. 13
2 evidence-based practice in education first a story from one of those other spheres of inquiry mary was scanning the slope when suddenly she saw a tooth projecting from it just a speck of gray fossilized enamel she looked once more and then shouted for me to come together we slowly cleared a little of the rock face with dental picks the ideal tools for such delicate work as the rock came away we discovered that behind the tooth lay another tooth and something more behind that perhaps we thought with growing excitement there might even be an entire jaw and skull leakey and van lawick 1963 whether mary leakey s find yet constituted evidence evidence for a new kind of hominid remained to be established and the potential constitution of the find as evidence rested itself on other work that anyone should bother to look here in this area at all for our human forebears rested on lucky accident and intuition rather than careful pre-planning or strong prima facie evidence the 1931 team of palaeoanthropologists led by louis and mary leakey was drawn to the area by the earlier work of entomologists and geologists who had by chance as part of their insect and rock hunting unearthed a human skeleton this was thought at first to be an ancient fossil but only much later was discovered to be a relatively modern intrusive burial in older deposits its constitution as evidence of the presence of fossil hominoidea drawing palaeoanthropologists to the area like a magnet was thus mistaken and the leakeys subsequent moves around the area were determined as much by what is often called `intuition as `evidence as louis leakey himself put it `for some reason both of us [he and his wife had been drawn again and again to this particular site leakey and van lawick 1963 134 it is the `for some reason that is primarily of interest here for it gives a clue about what is meant by `intuition a meaning residing in personal tacit knowledge built out of information data evidence accumulated both deliberately and fortuitously about the world ideas often emerge out of confluences of circumstantial evidence in the minds of those who are steeped in a problem a discourse or a technology often those individuals have a feeling a hunch that this way or that way is the right way to proceed without being able to articulate its evidential provenance there is a playing around with bits and pieces of everyday evidence that in some way enables practitioners of one kind or another to discover in the words often heard nowadays in relation to evidence-based practice `what works the discovery by the wright brothers that the aeroplane worked see dyson 1997 17 was made outside the theoretical frameworks within which evidence is supposedly used and from which it supposedly emerges the evidence behind the design of their aeroplane came from observation trial and error and from noting the trials and
[close]
p. 14
introduction evidence and practice 3 tribulations of others the evidence of its success came from the fact that it flew no more and no less no randomized controlled trials were necessary the discovery of penicillin the invention of nylon the discovery of superconductivity see de bruyn ouboter 1997 are all well-documented cases of the `intelligent noticing of evidence that emerged outside the intellectual infrastructure from which evidence is expected to materialize clearly all were discovered by able people working with the tools of their trade and immersed in the ideas of their intellectual communities but their significant breakthroughs and this applies as much to the everyday insight in practical activity as to the important breakthrough occurred out of evidence collected almost incidentally and worked on with personal knowledge and the knowledge of the intellectual communities of which they were a part all scientists whether physicists chemists biologists palaeoanthropologists use particular kinds of evidence and meld it in particular ways relevant to their fields of work and the methodological traditions that have developed there as michael eraut puts it in chapter 7 of this volume `the process of evidence generation is situated within the context practices and thinking patterns of its creators the interesting thing about scientists discourse in reflecting on these methodological traditions is that there is generally accepted to be no particular no correct or proper way of generating or marshalling evidence as einstein put it the creative scientist must be an `unscrupulous opportunist the essence of science he said is the seeking `in whatever manner is suitable a simplified and lucid image of the world there is no logical path but only intuition cited in holton 1995 168 in a similar vein feyerabend 1993 14 asserted that thought actually moves forward by `a maze of interactions by accidents and conjunctures and curious juxtapositions of events this use of the word `intuition by einstein in the context of scientific endeavour is an interesting one clearly he is not denying the significance of evidence rather he seems to be promoting a kind of spatchcock use of evidence a playing with almost any piece of seemingly relevant information by scientists in their everyday work and here scientists are very much like teachers or doctors operating on their own and as part of a community and drawing eclectically from many and varied streams of information a bricolage of potential evidence leakey for example describes how what could have been mere informational noise became evidence first of all came the pieces of supporting evidence to complete a jigsaw after several days we had all the pieces out and began putting our fossil jigsaw puzzle together at last we could see what we had mary
[close]
p. 15
4 evidence-based practice in education had discovered a nearly complete skull of proconsul africanus an early miocene creature leakey and van lawick 1963 135 then came corroborative evidence of the fossil s age from potassium argon dating and confirmation of its significance from other experts in the field the presence of a canine fossa for example is accepted in the scientific community as evidence of presence on the branch to hominoidea the canine serving as it does as an anchor for a muscle which controls the movement of the upper lip and therefore probably being important in some form of proto-speech the example of palaeoanthropology is given here to address some common themes in the search for evidence in palaeoanthropology and elsewhere in order for information to constitute evidence that information has to pass a number of tests first its relevance has to be determined for the notion of evidence assumes that something less than established fact an assertion a proposition an hypothesis has been put forward and data of some kind is wanted in support of that position the data inscribed on my beatles revolver cd is not in itself evidence for anything unless assertions are made about it for example that the beatles were an outstanding pop group in which case the data would constitute evidence about which judgements would have to be made evidence is thus information supporting or refuting an assertion and must pass the test of relevance if it is to move from informational noise to potential evidence through to prima facie evidence in helping to determine relevance will be other pieces of information and this raises the second test of sufficiency the potential evidence has to be considered with other pieces of information to determine its place in support of the assertion is there in other words corroborating evidence as russell points out the person who asserts that unsupported bodies in air fall has merely generalized from insufficient evidence and is liable to be refuted by balloons butterflies and aeroplanes 1956 91 quality and sufficiency of evidence here begin to be related to the epistemological ambitions of the proposition third and again linked to sufficiency decisions have to be made about the veracity of the evidence were these in the leakeys example more modern `intrusions and if so what was their status were they perhaps fakes as had been the case with piltdown man or are commitments so intense that the strong possibility of selectivity and misinterpretation make the `evidence base untrustworthy?2 there are cases in the education archive of commitment being so powerful that deliberate manipulation and distortion of evidence can be postulated.3 such questions arise principally because of the interests that exist in any research enterprise
[close]