A New Way to Monitor Student Progress: The FAST Projection Line™

One of the essential features of a multi-tiered system of support (MTSS) is monitoring the progress of students participating in interventions. Progress data serve an important role in documenting whether the intervention is working.  

Such data can also be part of the information used to determine if a student meets the eligibility criteria for students with a specific learning disability (SLD). Prior research about progress monitoring practices has affirmed that the more data points collected, the more reliable the data (Christ, Zopluoglu, Monaghen, & Van Norman, 2013; Thornblad & Christ, 2014). In addition, there are specific properties that an assessment must have in order to be a reliable progress measure.

Thanks to available technologies, cutting-edge research has demonstrated that it’s possible to interpret student progress data with fewer data points (Christ & Desjardins, 2018). This blog will review important properties for progress measures and explain how patent-pending innovation by the FastBridge research team is providing a way to interpret progress data in less time.

5 Key Progress Data Features

There are five key data features important for progress assessments. Specifically, progress measures need to be brief, simple, sensitive to growth, valid, and reliable. Without these features, the information gathered is not likely to be helpful for instructional planning.

Brief

The most effective progress measures will require limited time to administer on a regular basis. This is important because there are limited minutes in each school day and the best investment of time is in providing instruction. That said, teachers also need to know if the instruction is working, and so brief assessments are essential for progress monitoring.

Simple

To be useful, progress assessments also need to be simple. Specifically, they need to measure the skill that the student is learning and not other skills. If the progress assessment “bundles” too many skills into one test, it will likely capture irrelevant information. Simple assessments that directly assess target skills are best.

Sensitive

Related to simplicity is sensitivity. Measure sensitivity relates to whether the assessment can detect small changes in a student’s skills over time. This is mission critical because seeing a student’s skill changes is the purpose of progress monitoring. And, the progress measures that are the most sensitive happen to be those that focus on one skill at a time.

Valid

Valid assessments are those that actually measure what they are said to measure. For example, a reading assessment that measures reading. Validity is essential for all assessments but in the case of progress monitoring this importance is magnified in relation to the duration of monitoring. If an assessment is found to be invalid after many weeks or months of monitoring, valuable instructional time will have been lost.

Reliable

Finally, effective progress assessments must be reliable. Reliability refers to how well the assessment measures the same specific skill over time. Reliable progress measures have multiple equivalent forms that provide regular information about the student’s skills.

Prior Research about Progress Monitoring

Research about effective progress monitoring practices has been conducted for over 40 years. Key findings from the research include the recognition that the above essential features are important.

In addition, the research has identified that for progress data to be interpretable, there must be a sufficient number of data points. Christ, Zopluoglu, Monaghen, and Van Norman (2013) showed that as many as 12 data points were necessary in order for a student’s actual improvement to be identified.

The requirement for a large number of data points before outcomes can be interpreted is a daunting challenge for educators because students who are already behind need to make faster progress than their peers. And, teachers need to know about the progress as soon as possible in order to adjust instruction.

For these reasons, educators sometimes rely on fewer data points, but, as Thornblad and Christ (2014) showed, doing so comes at the risk of drawing the wrong conclusions about student outcomes.

A New Approach: Bayesian Statistics

The traditional approaches to interpreting student progress data have relied on a method known as ordinary least squares regression (OLSR). This method involves calculating the best-fitting line through the available data points.  

A major limitation of this method is that it relies upon only the scores for one student at a time. Due to the normal variation in students’ scores, as well as error in measurement, the best-fitting line might not be the most accurate predictor of a student’s future progress.

Conditional Probability

An alternative to OLRS uses methods developed by an eighteenth-century mathematician and clergyman named Reverend Thomas Bayes. Bayes’ work was relatively unknown during his lifetime, but after his death a paper he wrote about conditional probability was published. Titled Towards Solving a Problem in the Doctrine of Chances, this paper put forward the idea that prior data from multiple applications of a specific measure could be used to calculate expected future data points.

This calculation method eventually became known as Bayesian statistics or the Bayesian method, however, it was virtually impossible to use for everyday purposes until computers became widely available. As compared to the traditional OLSR method of determining the likely future trend of data, the Bayesian method takes into account existing prior data from the same measure.

For example, curriculum-based measures (CBM) of reading have been widely used for many years as a way to track student reading progress. If all the prior CBM reading progress data could be analyzed and taken into consideration, these data could provide probable trajectories of future student scores.

Another term for the Bayesian method is conditional probability because it calculates predicted future scores using accumulated prior data. These prior data provide actual “conditions” in which students’ scores followed a certain trend. The major benefit from Bayesian method is that it uses data other students who have been monitored with the same assessment to calculate an individual student’s predicted future performance.

As noted above, a significant factor in whether progress data are interpretable is having enough data points. Using data from other students makes it possible to predict individual student performance more quickly.

FAST Projection™ Line

Application of the Bayesian method was not practical until computers became widely available. This is because using this method requires collecting data from prior cases and then calculating the likelihood of a future outcome from those data. Such data were not widely available in schools until progress monitoring student learning became a common practice.

Both the No Child Left Behind (NCLB) Act of 2001 and the Individuals with Disabilities Education Improvement Act (IDEA) of 2004 incorporated provisions that encouraged schools to gather and use student progress data for instructional decisions. Since then, collecting data about student learning progress has become a routine practice in many schools.

In addition, there are many more computer-based assessment tools in use. The data collected through such computer tools provides a mechanism for defining the “conditions” that influence each student’s actual learning outcomes.

Based on the pioneering research by Christ and Desjardins (2018), FastBridge Learning has enhanced its progress monitoring tools to include the FAST Projection™ line for the most widely-used progress measures. The FAST Projection™ line uses Bayes’ conditional probability to calculate a student’s likely future scores. It does this by using a database of progress monitoring scores as additional data related to the likelihood a specific outcome. The result of the new FAST Projection™ line is that the student’s future predicted score progress (i.e., future trend) can be shown on the graph. Here is an example.

The FAST Projection™ line appears as a dotted blue line and extends past the solid trend line through the student’s data points. It can be turned off and on by clicking on the text on the bottom of the graph. The FAST Projection™ line can be compared with the dashed goal line to see if a student is likely to reach the goal. In the above example, the trend line through the student’s data points suggests that the student’s score might not be on track, but the FAST Projection™ line indicates that the student is predicted to reach the learning goal. The FAST Projection™ line is an entirely new innovation and awaits approval of a U.S. patent.

The FAST Projection™ line revolutionizes progress monitoring because it requires fewer data points that traditional calculation methods.  Recall that previous research showed that progress data are not reliable without 12 or more data points. As documented by Christ and Desjardins (2018), by using the FAST Projection™ line, progress data can be interpreted with as few as 6 data points. This is a major difference for students and teachers. If progress monitoring is done weekly, intervention effects can be reviewed after 6 weeks instead of 12. For students who are struggling, making changes to interventions as soon as possible is essential.

Measures Available

The FAST Projection™ line is available for the following FAST progress measures.

• FAST earlyReading English™
  • Decodable Words
  • Letter Names
  • Letter Sounds
  • Nonsense Words
  • Onset Sounds
  • Word Segmenting
  • Sight Words 150
• FAST CBMreading English™
• FAST CBMreading Spanish™
• FAST earlyMath™  
• Decomposing DC-1
• Numeral Identification NI-1
• FAST CBMmath Automaticity™

Faster Results

Although progress monitoring is a well-established practice in schools, data interpretation has been limited by the need for many data points over time. The FAST Projection™ line offers a new tool for teachers that allows them to inspect and interpret student progress with as few as 6 data points.

This innovation builds on the need for progress measures to be brief, simple, sensitive, reliable and valid by providing a tool that helps teachers review and understand data in shorter intervals. FastBridge Learning® is pleased to introduce the FAST Projection™ line as a significant enhancement for promoting student progress.

References

Christ, T. J., & Desjardins, C. D. (2018). Curriculum-based measurement of reading: An evaluation of frequentist and Bayesian methods to model progress monitoring data. Journal of Psychoeducational Assessment, 36, 55-73. doi:10.1177/0734282917712174

Christ, T. J., Zopluoglu, C., Monaghen, B. D., & Van Norman, E. R. (2013). Curriculum-based measurement of oral reading: Multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51, 19-57. doi: 10.1016/j.jsp.2012.11.001

Thornblad, S. C., & Christ, T. J. (2014). Curriculum-based measurement of reading: Is 6 weeks of daily progress monitoring enough? School Psychology Review, 43, 19-29.