Module One:

Gathering Evidence

Learning Objective:

Learn how to find high-quality research and evidence that addresses your purpose.


What You Need to Know

There are different types of educational research that produce evidence.

  • Primary and empirical research studies gather “raw” data that can be used for further analysis, but may not focus as much on practice and policy implications.

  • Secondary research reviews and synthesizes multiple studies by summarizing and further investigating their data and findings.

  • Theoretical or conceptual research, like secondary research, draws on previous studies to construct new theories.

Research organizations, online, searchable databases and clearinghouses make journal articles and peer-reviewed research easily accessible to practitioners and policy makers. Peer-review is an evaluation process in which qualified members in the same field assess the quality and scientific merit of research and its evidence.

Steps

  • 1. Define the Purpose

    Research can be used to serve different purposes, from finding out the efficacy of a program, curriculum, or intervention, to discovering a policy’s effectiveness, to gathering critical information about an intervention.

  • 2. Get Specific

    Make a list of all of the questions associated with the topic or larger research question you are trying to answer (e.g., what is the dosage needed for tutoring to be effective? What should the frequency be? The curriculum?)

  • 3. Identify Multiple Sources

    Evidence will be found in many sources, including academic journals, policy briefs, and reviews presented in clearinghouses and databases (see Resources below). You should also consider connecting with researchers or professional organizations and associations to explore research.

  • 4. Gather the Research and Information

    Review the abstracts, which typically restate the purpose, highlight the key points, and describe the results, findings, and recommendations, to find the research that will best answer your questions.

Guiding Questions

  • What evidence already exists that addresses this question (what do we already know)?

  • Has the research been reviewed by experts other than those who conducted the study? Did the study go through a peer-review process of some kind? 

  • Do the independent reviewers support the claims and methods used by the researchers?

  • Are there multiple sources or studies that support the claims or findings?

 

Case Study

A superintendent of a medium-sized district in Pennsylvania, where the majority of students participated in remote learning during the 2020-21 school year, wants to address unfinished learning by implementing a high-dosage tutoring program for high school students in need of additional support in math.

Tutoring is costly so she wants to understand the impact and use the most up-to-date evidence to determine the key elements of the program.


She begins by making a list of questions:

  • What qualifications and expertise should the tutors have?

  • When is the optimal time for students to engage in tutoring?

  • What is the dosage needed to be effective?

  • What should the tutor student ratio be?

She reaches out to a leader in a neighboring district who shares a research brief on high-dosage tutoring by the Annenberg Foundation. The brief outlines design principles like frequency, group size, and personnel, and links to evidence in research reports and studies. 

Wanting more evidence on impact, she searches “high-dosage tutoring” in ERIC, IES’s searchable database. There, she finds additional briefs, studies, and meta-analyses. A 2020 meta-analysis reveals significant average effects across nearly 100 experimental studies on tutoring, but the majority of the interventions focused on the primary grades. 

She continues to search for evidence specific to secondary math. She finds several studies and after reading their abstracts, she hones in on a recent study called “Not Too Late: Improving Academic Outcomes Among Adolescents” about 9th and 10th graders who received math tutoring during the 2013-14 and 2014-15 school years.