Examining Group Data to Evaluate an MTSS

By:Rachel Brown, Ph.D., NCSP

Recently Dr. Jose Castillo presented a webinar about the importance of reviewing school and district multi-tier system of supports (MTSS) practices using group data. Such data can provide indicators of whether student improvement goals have been met or if additional efforts to support certain groups of students are needed. His presentation highlighted two main factors in successful MTSS outcomes: (a) application of professional learning community (PLC) practices, and (b) utilization of adopted procedures with integrity. This blog will summarize Dr. Castillo’s presentation but readers are encouraged to view the full webinar in the FastBridge Learning® Knowledge Base.

PLC Engagement

The PLC model was developed by Rick Dufour and colleagues (Dufour, Dufour, Eaker, & Karhanek, 2009). This model involves establishing a set of practices and expectations of teachers to become lifelong professional learners. Dr. Castillo noted that the PLC model is ideal for MTSS implementation because it rests on a foundation of mutual trust and continuous improvement. While it is possible to implement an MTSS without using the PLC framework -- or something very similar -- it could make the process much more difficult and results slower. This is because the PLC framework focuses on three “big ideas” for supporting student learning (Dufour, 2004).

1. Ensuring that students learn
2. A culture of collaboration
3. A focus on results

These ideas are very similar to the cornerstones required for an MTSS, including effective instruction, timely assessment, and educator collaboration.

MTSS Data Review Procedures

The second component of effective MTSS practices is to have procedures in place for reviewing aggregated student data. In order to develop such procedures, Dr. Castillo recommended developing answers to the following guiding questions.

1. What are our priorities?
2. What is our buy-in or consensus for this work?
3. What is our capacity to implement?
4. How are we doing with implementation?
5. What are our student outcomes?

Answering the above questions will identify the school’s or district’s specific goals and capacity to reach those goals.

Priorities. One of the biggest challenges facing teachers is that students need to learn more than we can teach in one day, or week, or month, or school year. And each student’s need will be different from other students. This means that educators must identify the most important learning outcomes and focus on those as priorities. Each teacher might have different priorities for students, and this could lead to confusion among students. And, when teachers have different priorities, it will be harder for them to use both the PLC framework and MTSS practices. For this reason, a important step in creating effective student supports is for teachers to come together and identify the most important student learning priorities.

Consensus. Alongside identifying student learning priorities, it is also important to recognize the effort required by teachers to support the learning goals. Effective teaching is very hard work and when educators have shared understanding about the effort and steps needed to support effective practices, students will benefit. To this end, taking time to build consensus among the teachers in a grade and/or building about what steps are necessary to reach the identified priorities is time well spent. Consensus sometimes requires compromise by all involved in order to move forward on the most important goals.

Capacity. Once specific goals are prioritized, and the steps required agreed upon, another important component of reviewing the team’s likelihood of success is to examine its capacity to deliver the desired solutions. Such capacity will be determined by the resources already invested in student success and the deployment of addition supports in response to the team’s priorities. This is a very important aspect of long-term MTSS success. It is one thing to draft a plan that provides support to all students when they need it, but another to map out a detailed schedule for all steps that include:

1. Universal screening;
2. Team review of screening data to identify instructional needs, including Tier 1 enhancements;
3. Setting up interventions and progress monitoring plans;
4. Providing daily intervention sessions and weekly monitoring; and
5. Monthly review of progress data to see if interventions are working.

In order for tiered supports to be truly effective and beneficial, the team must review its capacity to provide the supports with integrity and make changes to daily practices in relation to the established goals.

Implementation. Once teams have worked through setting priorities, building consensus, and reviewing or improving capacity, the real work is to implement student supports on a daily basis. This is the heart and soul of an MTSS and requires daily attention to intervention integrity and data collection. It is important to note that implementing tiered student supports is a marathon and not a sprint. Teachers need to be patient and give time for effective practices to work. That said, having regular times set aside to review student data at least monthly is important. Such data reviews help teachers know which practices are working and what needs to be changed.

Outcomes. The result of all the efforts outlined above will be the students’ data. Importantly, data review for the purpose of evaluating an MTSS system should focus on group outcomes. Such review should consider the effects on all students, including those participating in instruction or intervention at Tiers 1, 2, and 3. Due to the fact that an MTSS can only be as strong as the effects of Tier 1 practices, the best starting point is to review group-level screening data. Here is an example of a district-level FastBridge Learning® report for the measure aReading, a broad measure of reading skills. This example suggests that, for the district as a whole, the students started the school year with better reading skills than they ended. The percentage of students who started the school year at high risk for reading problems was 22% but increased to 35% by the end of the school year.  

At the same time, the percentages of students meeting or exceeding the benchmark goal went down. In order to make sense of these data, the district team needs to look at the specific results for each of the schools as well as each grade level. By reviewing the aggregate student outcomes on a regular basis, school teams will be able to evaluate whether their efforts to address students’ most urgent learning needs have been effective.

Implementing tiered supports that address all students’ learning needs is hard work. In order for such efforts to be effective, schools are encouraged to use the PLC model which focuses on student learning, collaboration, and results. In addition, school teams will benefit from time spent identifying specific student learning priorities, building consensus and capacity to achieve these goals, investing in high quality implementation, and spending time examining student outcomes. These steps will lead to developing new priorities as well as improved implementation procedures.

References

Dufour, R. (2004). What is a professional learning community? Educational leadership, 61(8), pp. 6-11.

DuFour, R., DuFour, R., Eaker, R., & Karhanek, G. (2009). Raising the bar and closing the gap: Whatever it takes. Bloomington, IN: Solution Tree Press.