A collection of key reading. Updated as new learning takes place.
Last updated: November 2022
Critical engagement with research
The Open Door: How to be a Research-Sensitive School
Abercrombie. Haslam. 2021.
Paper
‘Research evidence has an impact on the culture of the school, shaping ideas of the right way of going about daily tasks. This is closely linked with the school’s moral purpose, of achieving the best outcomes for children. By being evidence-informed, the choices made within the school are seen to be based on firm foundations, rather than the latest whim or fad.’
‘Rather than choosing an answer because it feels right, a person who uses critical thinking subjects all available options to scrutiny and skepticism. Using the tools at their disposal, they’ll eliminate everything but the most useful and reliable information.’
Evidence Rubric for educators and teachers
Australian Education Research Organisation. No date.
Tool
‘The Evidence rubric assists education practitioners and policymakers in:
– self-assessing how confident they can be that a certain approach is effective in their context
– deciding how to implement the approach given their level of confidence
– collecting evidence that may increase their confidence in the effectiveness of the approach.’
‘First identify a piece of research evidence on a particular approach that you are considering implementing. Then, answer the series of guiding questions below that will prompt you to consider: what the research says; how relevant the research is to your context; whether you should implement the approach; and what you can do to ensure successful implementation.’
‘Rigorous evidence is defined as evidence produced using research methods (whether qualitative, quantitative or mixed methods) that isolate the specific impact of a particular educational approach. Relevant evidence is defined as evidence produced in contexts that are similar to one’s own. Evidence is also relevant when it is derived from a large number of studies conducted over a wide range of contexts, as this suggests that the educational approach is not dependent on any particular contextual factor.’
‘Cognitive biases are just tools, useful in the right contexts, harmful in others. They’re the only tools we’ve got, and they’re even pretty good at what they’re meant to do. We might as well get familiar with them and even appreciate that we at least have some ability to process the universe with our mysterious brains.’
‘Step 4: Normalize (Advanced level). Take steps to reduce the time and energy others have to spend challenging your blind spots and recruiting you to address the damage that you’ve contributed to. For example: actively seek out information and perspectives that challenge your own. Invite the best representatives of positions you don’t agree with to productive disagreements. Actively attempt to falsify your own beliefs.’
Landmark research integrity survey finds questionable practices are surprisingly common
de Vrieze. 2021.
Blog
‘More than half of Dutch scientists regularly engage in questionable research practices, such as hiding flaws in their research design or selectively citing literature, according to a new study. And one in 12 admitted to committing a more serious form of research misconduct within the past 3 years: the fabrication or falsification of research results.’
‘Education and teaching is not just one thing at a time. It can be many different things and even contradictory things.’
‘Use the term ‘evidence-informed’ teaching. This term emphasises that teaching, as a complex, situated professional practice, draws on a range of evidence and professional judgment, rather than being based on a particular form of evidence.’
‘To use evidence well, we must acknowledge the merits – and the limits – of good ideas.’
‘Evidence based practice… is about empowering teachers, and setting a profession free from governments, ministers and civil servants who are often overly keen on sending out edicts, insisting that their new idea is the best in town.’
‘For research evidence to inform decision making then it needs to be accessible to policy, practice, and individual decision makers. Research evidence can inform and researchers can advise but cannot alone make the decision.’
Bringing evidence into the classroom
Haslam, 2010
Blog
‘This lack of clarity about where to find reliable evidence means that both policy makers and practitioners can fall for the smooth presentations of new programmes and approaches. Neither group necessarily has the skill or time to search behind the slick marketing claims to find the impartial research that supports these programmes (or not).’
The importance of evidence
Haslam, 2011
Blog
‘Researchers and educators who develop practices and programmes with good evidence of effectiveness often struggle to develop their findings into something that can be easily applied in schools.’
Introduction to research: Judging the quality and trustworthiness of research evidence
Jones. Netolicky. 2019.
Article and poster
‘We need to consider the conditions under which the evidence was obtained, and whether that is reasonable to apply to the conditions in which we are working.’
‘The evidence needs to be contextualised for practice and presented in clear, accessible formats, using media which is accessible and includes practical guidelines for implementation, rather than simply being produced in its raw form.’
‘It is important to protect people against being misinformed, either by making them resilient against misinformation before it is encountered or by debunking it after people have been exposed.’
‘Consider for the moment the last three important decisions you made. What or who had a bias on those decisions?’
Facts Are More Important Than Novelty: Replication in the Education Sciences
Makel. Plucker. 2014.
Paper
‘If education research is to be relied upon to develop sound policy and practice, then conducting replications on important findings is essential to moving toward a more reliable and trustworthy understanding of educational environments.’
‘Cognitive bias is a form of thinking that appears to be erroneously based on logical and rational reasoning, leading to incorrect analysis and judgments. These shortcuts of the mind are mostly unconscious and systematic. Thus, although we can act on them after the fact, we cannot prevent the brain from producing them. The best way to counteract them is first to be aware that they exist, then to know how our three systems of thought work, and finally, as developmental psychology researcher and neuroscientist Olivier Houdé argues, to develop our “cognitive resistance” or “learning to think against ourselves”.
‘The evidence needs to be contextualised for practice and presented in clear, accessible formats, using media which is accessible and includes practical guidelines for implementation, rather than simply being produced in its raw form.’
‘Confirmation Bias is the tendency to look for information that supports, rather than rejects, one’s preconceptions, typically by interpreting evidence to confirm existing beliefs while rejecting or ignoring any conflicting data’
‘Studies that show negative findings are systematically rejected because they are typically judged to have been poorly designed. The result is that the literature is skewed towards positive representations of the efficacy of given treatments and programmes, with no countervailing evidence to challenge it.’
‘The whole cathedral is built on an edifice of misrepresentations and mythology.’
‘Questionable research practices” are ubiquitous in scientific research… To produce findings that get published in these journals, scientists engage in a wide range of methodological and statistical gymnastics to contort their results into a form that makes them publishable.’
‘Given varied research findings, confirmation bias becomes crucial. We tend to uncritically accept research that confirms our existing beliefs while seeking fault in research that contradicts them.’
‘Research is cherry-picked, bent and twisted to support an argument and back an approach, selling an ideology is seemingly more important than whether it actually may or may not work.’
‘A good choice is one that uses the best information available at the time. For education choices, this includes using the best available evidence of intervention effects. Good choices don’t guarantee good outcomes, but they make good outcomes more likely.’
‘It is a common misconception that in order to use research evidence in their work, an educator just needs to find the right piece of research to tell them what to do in their classroom.
Usually, the perfect piece of research evidence does not exist. Instead, educators need to identify, understand, interpret and apply evidence from research to their classroom practice.’
Pride = Thinking you know better than the data.
Lust = Relations with unclean, but enticing data.
‘People are failing to implement new ideas due to ideology, trust, time, resources, habit, or one of many other reasons. Simply announcing new ideas is insufficient.’
‘It turns out that the dissemination of evidence is a process that is laced with emotions, relationships, trust and values.’
‘In theory, the goals of edu research are to build knowledge & improve decision-making & outcomes for teachers & students. In practice, edu research is shaped by the practices & priorities of researchers, not teachers or school/system leaders.’
‘When I’m facing a big reading list, I want to get stuck in quickly to the topic positively and get some ideas bouncing around. Lively and short texts are a good way to do that. Once I’m in the zone I’ll tend to tackle meatier tomes.’
Decolonising research
‘I’m just very interested in how indigenous peoples talk about knowledge and and how their knowledge is not seen as as sort of anti-science but rather a form of science, a form of inclusive science that is millennia old and tested and just as scientific as everything else.’
‘The time has come for researchers to start those uncomfortable conversations on how we can make research collaboration inclusive, respectful and equal.’
‘Researchers and policy-makers should recognise that populations vary considerably in the extent to which they display certain biases, patterns & preferences.’
Decolonising research methodology must include undoing its dirty history
Ndlovu-Gatsheni. 2017.
Blog
‘What is at issue is re-search as a terrain of pitting the interests of the “re-searcher” against those of the “re-searched.” The core concern is about how re-search is still steeped in the Euro-North America-centric worldview. Re-searching continues to give the “re-searcher” the power to define. The “re-searched” appear as “specimens” rather than people.’
‘It is important that research distinguishes between exploratory & confirmatory analyses, & uses appropriate analytical techniques. However, even the most perfect methods will not yield much if we gather data from such a narrow slice of humanity.’
Whose Psychology Is It Anyway? Making Psychological Research More Representative
Reynolds, Cheon, Roberts. 2021.
Audio
‘There’s an automaticity when research is done in WEIRD contexts, that the results will apply everywhere. If research is done in non-WEIRD contexts, the results aren’t seen as universally applicable. The context is expected to be made explicit, whereas in non-weird contexts, it isn’t.’
“It’s the issue that we all like to talk about, but nobody likes to actually change.”
‘You can’t claim that algorithms are neutral… embedded in them is inequality, iniquity, embedded in them is racism and sexism, and a denial of existence of multiple identities.’
‘Decolonisation sensitises us to the existence of dominant discourses and the influence of dominant groups in what / who we research. It makes an appeal to be critical and reflexive throughout the research process. It provides a timely reminder that our assumptions about rationality derive from a literature that developed at a particular time and place.’
‘In considering “non-weird countries” together, it’s lumping together people from cultures as diverse as Mexico, Japan, and South Africa. “Cross-cultural research must be informed with thorough analyses of each and all of the cultural contexts involved.”
Conducting research
‘We recommend that at all stages of a project – from planning through conduct to reporting – educational researchers undertake wide consultation to identify relevant ethical issues, including listening to those in the research context/site(s), stakeholders and sponsors. This means that ethical decision-making becomes an actively deliberative, ongoing and iterative process of assessing and reassessing the situation and issues as they arise.’
‘Action research puts the teacher at the centre, questioning matters within [their] context with an intention of improving them… Action research deals with teachers’ values, which are often magnified through the process of continuous self-reflection.’
‘My role is a coordinating one, concerned with boundary spanning and building bridges to encourage practitioner research and link teacher-researchers across schools and with the wider research community’
‘Making use of key teachers as opinion-formers and research-translators has, in part, bridged the gap between academic research and class teachers.’
‘The laundry list of things that teachers have to do on a teaching day can be inexhaustible… The realities of school life essentially militate how teachers prioritise the deluge of work that needs to be attended to in the current instance over professional development activities, such as engagement in action research, which will most likely bear fruit some time later.’
‘Teachers ‘found it difficult to learn the process quickly enough to be able to implement it with any confidence… This finding is congruent with the experiences of McTaggert et al (1997) who discovered that action research was a difficult process for teachers to learn and sustain because of its complexity and lack of congruence with the hectic nature of life in classrooms.’
‘Get comfortable with asking questions such as “What research are we drawing on here?” or “What evidence do we have that this would work in our setting?”
