Problem Analysis: Using Assessment Data to Find Solutions

By: Yvette Aranas

Problem analysis is often recommended in the school psychology literature to solve various problems in schools (Christ & Arañas, 2014). If this concept is new to you, problem analysis can be defined as the process of using assessment data to understand the nature of a problem and to find a solution to that problem.

Problem analysis pretty much follows some of the same steps of the scientific method. When conducting studies, scientific researchers first identify a problem and formulate a research question, develop a hypothesis, design a procedure to test the hypothesis, collect data, analyze the data, and come up with a tentative conclusion (or revise the hypothesis).

School psychologists and other educators should consider applying the scientific method when finding solutions at school. Problem analysis consists of steps that are similar to the scientific method:

1. Identify the problem.
2. Hypothesize what is causing or maintaining the conditions around the problem.
3. Select methods for assessment.
4. Collect the data.
5. Review and analyze the data.
6. Use the data to form a hypothesized solution for an intervention, or revise your initial hypotheses.

Doing Problem Analysis Effectively

In order to find the best solutions to a problem, there are a few best practices to follow to effectively analyze a problem. First, we strongly suggest to select methods of assessment that don’t rely on too many inferences. In other words, using assessments that collect subjective information (e.g., gut feelings about a student’s reading level, aptitude profiles, personality surveys) is not recommended because they require many assumptions that are often hard to test. Instead, it is more appropriate to use assessment methods that enable you to collect objective and observable data directly. In the case of behavior, school psychologists often conduct direct and systematic observations in class where they keep tallies of a student’s behavior. For academic skills, assessments like the ones provided by FastBridge Learning can be used to provide quantitative estimates of students’ academic performance and progress. Using low-inference assessments should be used when identifying the problem, and when selecting methods for collecting data.

Second, it is important to identify the things that cause a problem to occur, as well as the things that continue to maintain the occurrence of the problem. When identifying causal and maintaining variables, the variables that are examined should be alterable, meaning that they can be changed. When a problem arises, educators too often put the blame on variables they little control over, such as a student’s medical conditions or home life. Instead, we should collect information for variables that can be manipulated, such as instruction, staff-student relationships, and classroom or school rules.

Third, it must be noted that the last step of the problem analysis process requires making hypotheses for an intervention, or an alternative hypothesis that alters the first one. The reason why we make hypotheses (rather than final conclusions) about a solution is because we cannot be 100% certain that a chosen intervention will solve the problem. Instead, interventions should be tested for their effectiveness. One way to test an intervention hypothesis is to collect progress monitoring data, and using those data to decide what to do with the intervention (e.g., discontinue it, keep doing it, or adjust it). For a more in-depth discussion about changing instruction based on data, consider reading our previous blog posts about progress monitoring.

Using Problem Analysis in a Multi-Tiered System of Supports (MTSS)

Obviously, it would be very inefficient to implement problem analysis for each and every individual student in a school. Thus, we recommend analyzing problems at the school-wide, group-wide, and individual level. Using universal screening data at each benchmark period would help to identify problems in behavior, reading, and math. The data can point out which students need additional support, and can also determine whether a particular problem is a school-wide or class-wide problem. The next step after that is to determine problems at the group level. Students who were flagged from the screening assessments would receive small-group interventions.  Progress monitoring data would be collected to determine whether the interventions are working. If a student continues to receive these additional interventions but fails to make adequate progress, more intensive services may be warranted (e.g., one-on-one interventions, special education).

As mentioned, problem analysis is a way of applying scientific practices to solve problems in schools. This is an important point to make because solutions are more likely to be effective when decisions are made based on data and evidence.

Reference
Christ, T.J., & Arañas, Y.A. (2014). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds.) Best Practices in School Psychology VI. Bethesda, MD: National Association of School Psychologists.

Yvette Arañas is a doctoral student at the University of Minnesota. She was a part of FastBridge Learning’s research team for four years and contributed to developing the FAST™ reading assessments. Yvette is currently completing an internship in school psychology at a rural district in Minnesota.