Managing Ongoing Student Assessments
Today's live chat has ended!
Please scroll down to the bottom of this page to view the transcript of this event.
Progress measures have become a central component of the RTI approach. Contemporary assessment has shifted focus from describing differences between students to measuring their progress toward important educational outcomes.
Curriculum-based measurement procedures are used to monitor basic skills growth, to identify students at risk of learning difficulty, to evaluate efforts to prevent and remediate low achievement, and to aid in making instructional decisions to accelerate learning.
Join our expert, Stanley Deno, Ph.D., best known for his research leading to the development of Curriculum-Based Measurement (CBM) progress monitoring procedures and their use in the Response to Intervention (RTI) model, as he answers your questions on CBM.
Read more about Stanley Deno, Ph.D.
My general view is that your Child Success Teams are the key element in using progress monitoring data to determine whether intervention is required, what level of intervention is required, and whether the intervention has been effective in improving children's rates of growth.
However, I think this is a question better answered by others at the NCLD. Their article on RTINetwork.org titled "A Model for RTI in Pre-K" provides an overview of how the core principles of RTI relate to core early childhood beliefs and practices.
I would start at the National Center on Student Progress Monitoring. They have an excellent web site and can direct you to various resources for getting you CBM progress monitoring system up and running. In particular, they have an introductory article "Getting Started: How do I Implement Progress Monitoring in My School," by Kathleen McLane.
In addition, Dr. Lynn Fuchs has an article, Progress Monitoring Within a Multi-Level Prevention System on RTINetwork.org that provides a quick overview of progress monitoring and describes how progress monitoring is used within a multi-level prevention system. A companion piece to this article details validated forms of progress monitoring in both reading and mathematics.
"Read Naturally" has its own progress monitoring system that is based on the Curriculum-based Measurement (CBM) approach. You should be able to use that system to evaluate the success of "Read Naturally."
If you would prefer that your child's growth be measured independently from Read Naturally, I would recommend you ask your child's school to use one of the progress monitoring approaches approved by the National Center on Student Progress Monitoring.
I wish I could answer your question in such a way as to assure you that we know the answer. I don't think we do. RTI provides a great framework for schools to use in organizing to prevent and remediate academic problems. CBM provides a good basis for evaluating whether modifications in instruction are producing better rates of growth. I believe there is fairly good evidence that students most at risk of reading difficulty do best when provided intense, explicit and systematic instruction.
I also believe that for the small percentage of students who do not learn basic skills in reading and writing even with such instruction that the best approach is individually managed instruction (usually tutoring) by a master teacher who is very resourceful in developing alternative, customized programs that meet the needs of individual students. In such an arrangement, the CBM data become an essential tool for that teacher to determine the success of his or her efforts to teach a child.
I like to paraphrase Abraham Lincoln when it comes to infallible instructional programs:
"A program might work with all of the students for a short time, or a program might with some of the students for a long time, but no instructional program is likely to work with all of the students all of the time."
The assumption on which CBM is based is that development of basic skills is the result of the interaction between the type of learning opportunities a child has and the psychological and physical characteristics of the child. If a child has processing problems and those problems can be addressed through instruction, then the child's growth in basic skills should increase.
CBM is designed to show whether students are developing basic skills, not to monitor changes in a student's psychological and physical characteristics. Speculation regarding a student's psychological processing characteristics can provide hunches about how to teach a student. The CBM data should help to determine whether the hunches derived from those speculations have led to developing a more effective instructional program.
I'm partial to using growth rates for setting goals for students who have been identified as at risk for academic failure or those already receiving special education. Any other approach tends to result in setting goals that exceed the reach of the teacher and the child. Benchmarks based on norms are appropriate, I think, for the general population of students making adequate progress and for universal screening.
When using growth rates to set goals, one should also incorporated a systematic strategy for revising goals upward should the student's progress rate indicate that a higher goal can be attained. Using the original CBM formative evaluation model involves setting year end goals and a goal line connecting the beginning level of performance with the year end goal. The slope of that line enables identification desired level of performance at any time during the year (weekly, monthly, quarterly, etc.)
If you go to the Web site of the National Center for Student Progress Monitoring you will find a list of alternative progress monitoring measures under "Tools." Presumably, you can use any of the measures listed there that provide a sufficient number of alternative passages for doing progress monitoring.
Considerable evidence exists that you can use oral reading of passages to monitor growth in reading proficiency beginning in Grade 1. I think isolated word reading might be acceptable as a progress measure in Grade 1, but I would not use other measures for progress monitoring than oral reading of text or isolated words beginning in Grade 1. For me, the data are clear that oral reading of text can be used beginning in Grade 1 and that isolated word reading is not necessary.
Beginning in Grade 1 and moving beyond, I would recommend using only one of the progress monitoring systems shown on the National Center's Tools Chart. Mixing 2 of the progress monitoring systems will only lead to confusion because it will likely lead to conflicting benchmarks and progress scores inconsistent with the benchmarks. I think it is difficult to use both tools effectively, and I do not recommend it.
I think the biggest challenge for teachers in implementing RTI is how to differentiate instruction when the progress monitoring data indicate that an instructional program is working for some students but not for others. Then, differentiating flexibly so that students can move in and out of different programs compounds the difficulty. The problem is most acute in Tier 1, I think, where general classroom teachers are faced with 25 students and being asked to provide alternative instruction of subsets that might not be homogeneous with regard to their instructional needs. Since Tier 2 & 3 interventions seem to be fairly standard implementations for all students in the Tier, differentiation might be less of a problem.
Information technology can help in tracking and grouping students who are succeeding and who are not and by enabling teachers to monitor who is succeeding and who is not. It might be possible to use technology to aid in grouping and regrouping students in a timely fashion. It should certainly help to prevent the possibility that some students will get lost in the shuffle.
In the end, I think we are still going to be confronted with students for whom are standard interventions have not been successful and who will require customized programs provided by individual teachers.
I'm not sure what you mean when you say "we know it doesn't work for all children." A good General Outcome Measurement system is like a bathroom scale. If it is a good scale it works for anyone who stands on it. If your school is using an approach to measuring growth in basic skills like reading that does not show when children are becoming better readers, then you should seek an alternative progress monitoring system for your school.
I recommend that you go to the web site of the National Center on Student Progress Monitoring for help in identifying approved monitoring systems. Keep in mind that the progress monitoring system will not successfully teach the students. Only the teachers will successfully teach the students. The General Outcome Measurement systems should enable you and the teachers to see whether their instruction is successful.
CBM progress monitoring is most helpful, I think, with students who have not yet become proficient in basic skills. While considerable work has been, and is being, done to develop progress monitoring procedures for secondary students they are not as extensively developed. I think it is possible to use reading, written expression and math progress measures for struggling students well into high school. I do not think it is appropriate to use those measures for high school students who have fairly well developed basic skills.
I would encourage you to contact the Research Institute on Progress Monitoring at the University of Minnesota for more information on CBM reading and written expression measures for secondary students. Probably the best summary of the status of CBM progress measures for math can be found in the article by Foegen, Jiban, and Deno (2007) "Progress Monitoring Measures in Mathematics" Journal of Special Education Volume 41 (2) Pp 121-139.
The Research Institute on Progress Monitoring (RIPM) at the University of Minnesota is conducting research to develop progress monitoring procedures for students with significant cognitive disabilities. I encourage you to contact RIPM for a status report on their efforts to develop the CBM approach with that population of students.
In addition, a set of early childhood measures, called Individual Growth and Development Indicators (IGDIs) for screening and progress monitoring of children 3-5 years old has been developed. You can check out those measures at "Get it, Got it, Go" on the web.
Traditionally the measurement of academic progress has focused on describing differences in the levels of achievement between students. One reason for this approach in developing standardized achievement tests was the prevailing view that academic success was primarily the result of ability differences among students. The assumption was that the information produced by these tests was useful primarily in classifying students and placing them in different programs on the basis of their ability.
Over the past 30 years, an alternative approach to measuring student achievement has been developed that focuses less on describing differences between students and more on measuring their progress toward important educational outcomes. These progress monitoring procedures have been developed to describe the growth rates of students as they are learning reading, writing, and math. A primary assumption underlying the development of these progress monitoring procedures is that differences in rates of academic skill growth are affected by instructional differences and are not simply the result of ability differences. Another important characteristic of progress monitoring measures is that they can be used as regularly and as frequently as necessary throughout the school year both to identify students at risk for academic problems and to evaluate the impact of changing a program on a student’s rate of academic growth. This potential for repeated measurement is in marked contrast to typical standardized achievement tests that are intended to be given only once or twice during the school year.
My article on "Ongoing Student Assessments" on RTINetwork.org provides additional information. In addition, the National Center on Student Progress Monitoring is probably the best resource for answering this question. The Research Institute on Progress Monitoring is also a good resource. You can click on the video where I discuss progress monitoring. Also, Curriculum-Based Measurement: From Skeptic to Advocate, by Jan Hasbrouck, Ph.D., and Candyce Ihnot, M.A., describes on teacher's experiences and provides six case examples to illustrate the benefits of CBM.
Some years ago I worked with the special education program in Idaho Falls, ID, to help implement the Curriculum-based Measurement approach to progress monitoring and formative evaluation of student programs.
The person responsible for the gifted and talented program used the procedures with the gifted and talented students and, much to my surprise, found them to be very useful. Having said that, the progress monitoring procedures used in RTI were designed for low achieving rather than high achieving students. RTI is a generalizable approach to building more effective educational programs that could be used with gifted/talented students.
I agree with you. Many of the design features we used in developing the Curriculum-based Measurement (CBM) approach to progress monitoring had to do with the need to make them efficient and easy to use regularly. I specified those design features in the 1985 article "Curriculum-based Measurement (CBM): The emerging alternative." We actually began to develop comparable measures for behavior, but never achieved the same types of progress measures that we were able to create for basic academic skills. I would say the types of measures you are having to use do not serve well as progress measures.
I published an article many years ago on a classroom observation system for behavior that has been modified and used by the Minneapolis Public Schools in their Special Education Program (I think they call it the Deno-R). You might contact them if you are interested. The article reference is as follows: A direct observation approach to measuring classroom behavior. Deno, S. L.,(1980) Exceptional Children, 46(5), 396-399.
That's a good question, Aaron. I wouldn't say there is a "protocol" in the sense that you're using the term. There is quite a bit of research on the technical characteristics of the repeated measurement data at different levels and with different schedules. One important conclusion from that research is that the reliability and validity of using typical CBM progress monitoring procedures is quite high and consistent even when students read passages that are lower or higher than the level at which they are receiving instruction. Thus, it is possible to monitor student progress using passages from the student's grade level even when the student is not reading at that grade level.
Further, having student's read from easy passages seems to produce steeper growth rates and might be more sensitive to intervention effects. In general, the choice of level and frequency of monitoring is based on how the data are going to be used; that is, what decisions will be made and when those decisions will be made. When progress is to be evaluated frequently (weekly, for example) and changes in program to be considered, then weekly progress monitoring seems appropriate. If decisions are going to be made quarterly, then giving multiple measures at the beginning and end of the quarters might be sufficient.
More than 20 years ago, Lynn and Doug Fuchs did a literature review on the achievement outcomes of using progress monitoring data. One of the most important conclusions from their review was that achievement increased when teachers used explicit decision rules to modify their instruction. A related conclusion was that no particular set of decision rules seemed to produce better achievement. Since then a spate of studies on using decision rules has been conducted and the general conclusion remains the same: It is important to use a set of rules to assure that we are being responsive to the students rate of progress, but no particular set of rules is supreme.
One important qualification to that statement is that the decision rules used should include raising progress monitoring goals when the data indicate the student can grow at a rate that is faster than first anticipated. Thus, revising including a provision for revising goals upward (not downward) is an important component of any decision rule system. Having an explicit set of decision rules in place helps to overcome the tendency to view the child's rate of growth as a function of the child's characteristic rather than the instruction being provided. Too often, progress monitoring becomes an inert process where we watch a child's growth but to attempt to find powerful changes in instruction that can make a difference.
A fair amount of research has been done on the reliability and validity of the maze procedure across all grade levels up through 10th Grade. It doesn't need to be used only as a supplement or oral reading. The maze procedure can be used for screening at all grade levels, but it particularly useful in the intermediate grades and above.
We need to keep in mind that the data are being used for screening and that making high stakes decisions involving changing students programs or allocating additional resources requires following up screening with other sources of information, if possible, to assure that the decisions are accurate. This becomes particularly important at the secondary level where basic skills are only part of what is involved in determining a student's academic success.
This is all new to me. I am working on my master's degree in special education and have really fallen for the progressive RTI style. As I get deeper into the RTI realm I am concerned about tracking data for 400 elementary students, their progress monitoring, and the formation of routines for identifying students within the Tiers... I know that each individual school develops their own system, but there must be templates to get a start and suggestions for best practices! Any ideas?
That concludes our LD Talk for today. Thanks to everyone for the thoughtful questions and thanks to our experts, Dr. Stanley Deno for his time today.
Additional Ongoing Student Assessment Resources on RTINetwork.org
- Ongoing Student Assessment by Stanley Deno, Ph.D.
- Universal Screening for Reading Problems: Why and How Should We Do This? by Joseph R. Jenkins, Ph.D. and Evelyn Johnson, Ed.D.
- Progress Monitoring Within a Multi-Level Prevention System by Lynn S. Fuchs
- The RTI Data Analysis Teaming Process by Joseph F. Kovaleski, Megan Roble, and Michelle Agne