Module Five:

Monitoring Progress

Learning Objective

Learn how to assess and continuously improve an intervention.


What You Need to Know

One way to continuously improve an evidence-based intervention or practice is to use Improvement Science. Improvement science has long been used in other industries, such as health care and manufacturing, to accelerate learning and solve problems of practice. Only recently has it been used more frequently in education. 

Improvement science focuses on small, rapid tests of change to guide development and continued refinement. The objective is a focused learning journey rather than a traditional pilot program that seeks proof of concept.  


Steps

  • 1. Adopt a Learning Stance

    It’s important to realize that implementation marks a new beginning. Maintaining humility while always keeping an eye on reality is essential to understanding what’s working and what needs to be changed.

  • 2. Gather and Analyze Data to Make Improvements

    Set up structures to collect experiential and outcome data on a regular basis and make adjustments to continue to refine and improve the intervention before scaling.

  • 3. Determine Impact

    Identify quantitative (e.g. achievement tests) and qualitative (e.g. teacher reporting and student work) measures for assessing impact over time on students and other stakeholders

  • 4. Start to Scale

    Once impact is determined, start to add more students who are receiving the intervention. Make sure the scaling plan is deliberate and phased.

Guiding Questions

  • What is the experience of those implementing and/or receiving the invention? 

  • What quantitative and qualitative measures can we use to monitor progress and determine impact?

  • How are we documenting adjustments and continuously assessing their impact? 

  • What is our plan for sustaining and scaling the intervention once we’ve assessed impact?

Case Study

With a working group consisting of district and school leaders and teachers, the superintendent creates a detailed implementation plan for high-dosage tutoring for 9th and 10th graders based on evidence from research. The plan covers elements of the intervention, such as frequency and group size, and clearly outlines training and curriculum. 

A critical piece of the implementation plan are structures for monitoring the intervention’s efficacy and the students’ progress. Tutors and teachers will meet every two weeks to discuss how it’s going and review student work and data based on small, rapid tests they conducted (e.g. testing different durations of the tutoring time). Administrators will create channels for tutors, teachers and students to give feedback and input with the goal of continuously improving the intervention throughout the course of implementation.

If after the first year, student growth and experience are positive, the superintendent plans to scale the program to 11th and 12th graders.