Dorothy M. Millar, Debra L. Shelden, and Paula D. Kohler University of Illinois at Urbana-Champaign
In recent years, several authors have addressed the lack of empirical evidence to support "best practices" in transition services for individuals with disabilities (e.g., Greene & Albright, 1995; Johnson & Rusch, 1993; Kohler, 1993). In response to this concern, Kohler (1995) conducted a four-phase study, resulting in the Taxonomy for Transition Programming, a conceptual framework of transition practices generated and evaluated by transition professionals and researchers. While the taxonomy is a significant beginning toward determining "best practices," Kohler (1995) suggests that additional intervention and outcome research in conjunction with program evaluation is needed to identify and further develop effective practices.
The purpose of this chapter is to present a transition services assessment process, based on the Taxonomy for Transition Programming. Recognizing the importance of program evaluation in improving post-school outcomes for individuals with disabilities and expanding understanding of effective program components, the Midland County Interagency Transition Team (MCITT) in Michigan has developed a comprehensive evaluation tool, the MCITT Transition Self-Assessment instrument, based on the Taxonomy for Transition Programming (Kohler, 1995). (This instrument is included in this monograph in Appendix B.) The MCITT believes that regular evaluation of all components of the transition process is an essential component of their mission. Evaluation is imperative in determining whether or not transition services are resulting in desired effectsÑthat each student's transition from school to adult life is successful, leading to community participation and a quality life.
History of the MCITT
During the past six years, Midland County schools and agencies have focused many resources on the development of a comprehensive transition process that would enable all students receiving special education services to develop and attain their life goals. The Midland County process is driven by five fundamental components: (a) utilizing strategic planning processes to guide interagency collaboration efforts; (b) infusing transition services and plans in individualized education programs; (c) promoting and facilitating awareness and training activities for students, parents and primary caregivers, school and agency staff, and the entire community; (d) utilizing a functional curriculum and community-based instruction; and (e) conducting ongoing evaluations of the transition process. Each of these components is included in the Taxonomy of Transition Programming (Kohler, 1995).
In 1991, key stakeholders in Midland County's transition process established an interagency committee (MCITT) to further the county's efforts in providing quality transition services. Members of the team represent a variety of perspectives, including those of special education, vocational rehabilitation, mental health, social services, independent living, postsecondary education, and adult services. The MCITT began by articulating common beliefs and defining a mission. Using a strategic planning process, the group continued by identifying primary goals; those goals included assessing student outcomes, interagency efforts, and community resources, as well as evaluating the transition process in a comprehensive manner.
Development of the MCITT Transition Self-Assessment
To achieve the evaluation goal, the MCITT utilized a list of questions to guide the development of an evaluation instrument. The questions, modified from those suggested by Ory (1990), included the following:
The following is a discussion of the rationale for and approach to each of these questions.
Evaluation purpose. The guiding purpose of the ongoing evaluation process is to improve transition services for students with disabilities. Specifically, the evaluation should provide feedback on whether or not the transition process is being implemented in a comprehensive manner, whether the components and/or process need to be modified, and whether or not the implementation of the components and/or process is having the desired effects.
Program components to be evaluated. In order to assess the comprehensive nature of transition services, the MCITT used a modified Taxonomy for Transition Programming (Kohler, 1995) to structure the evaluation instrument. Each cluster within the five taxonomy categories and their respective practice statements are identified as program components to be addressed in the evaluation. Additionally, the evaluation tool includes statements from the Individuals with Disabilities Act of 1990 relevant to IEP development and transition planning. Thus, the tool evaluates program components based on current research as well as legislative mandates.
Information collection methods and interpretation. The MCITT Transition Self-Assessment instrument was developed to collect information on the extent to which transition practices are being implemented. This five-page tool is intended to be completed by all individuals involved in providing transition services, including special education teachers, rehabilitation counselors, and administrators. Respondents rate the frequency with which each transition practice statement occurs within their own work on a 3-point Likert scale, 3 (always), 2 (sometimes), or 1 (never). Completed assessments provide a picture of the comprehensive nature of transition services in the county.
Reports and use of evaluation results. Wentling (1989) suggested seven uses and applications of evaluation information: (a) to demonstrate accountability, (b) to convince or gain support, (c) to educate or promote understanding, (d) to involve key individuals, (e) to document processes and results, (f) to direct project modification, and (g) to improve communication. Although all of these applications may be pertinent to the MCITT's mission, the information gleaned from the MCITT Transition Self-Assessment is used primarily to document processes and to direct project modification. A key purpose of the MCITT was to ensure that transition services are provided in a comprehensive, effective manner; the information compiled from the evaluation tool will serve to either document that process based on Kohler's (1995) model or indicate areas of service that are not occurring. Identification of undeveloped areas, in turn, will lead to plans to improve efforts in those areas.
Evaluation information collected with this tool also serves to educate and promote understanding, involve key individuals, and improve communication. These applications are accomplished in part through the actual process of self-assessment, as well as through periodic evaluation reports that inform the community of stakeholders of the county's status and progress in developing transition services.
Pilot Test of the MCITT Transition Self-Assessment Instrument
To date, implementation of the MCITT Transition Self-Assessment has consisted of a pilot test and subsequent revision of the instrument. Instruments were distributed to all 21 members of the MCITT, 19 of whom responded. Of those responding, five were school administrators, seven were school teachers, and seven were community service providers. Community service provider respondents included individuals from vocational rehabilitation, social services, mental health, postsecondary education, independent living, and adult service agencies.
Rather than aggregate data from the completed instruments, the MCITT conducted a series of discussion groups to provide feedback on the pilot evaluation process. Several themes emerged from the discussion groups, including suggestions to reduce redundancy and improve the scale. Other discussions focused on the potential usefulness of the information to be generated.
Other key concerns that emerged from the pilot test focused on the relevance of specific program components to particular participant groups. For example, one agency representative completed the section of the instrument concerning IEPs, most other agency representatives considered the IEP section out of their realm. However, school personnel, indicated that they would like agency personnel to be more knowledgeable of the IEP content and process. The completion of the instrument, then, illuminated the need for increased communication between these two groups of professionals with respect to IEP development. A discrepancy between the two groups concerning the differentiation between shared knowledge and discipline-specific knowledge is apparent.
MCITT members also discussed the need to differentiate between rating the availability versus the utilization of transition services and practices. For example, some participants felt that during the initial assessment, it was unclear what the rating was intended to indicate. The consensus was that both availability and utilization should be rated, as one does not necessarily correlate with the other. Thus, separate scales yield more detailed and meaningful information.
Finally, the MCITT members felt that the instrument and process would be useful tools in their efforts to improve transition services. All agreed that the interagency collaboration that guided the development of the evaluation process was critical to its success. Further, they indicated that information gathered with the assessment instrument would be useful in focusing future discussions and efforts.
The MCITT will use the feedback from this first phase of information collection to revise the instrument. A revised version will then be used to continue the evaluation efforts.
Additional Evaluation Strategies
While the MCITT Transition Self-Assessment promises to be an effective process for evaluating the comprehensive nature of transition services, the MCITT will incorporate other strategies its overall evaluation plan. Onaga (personal communication, April 23, 1994) suggests utilizing eco-maps, case studies, and focus groups in program evaluation. Eco-maps, pictorial representations of relationships among people and organizations, will be particularly useful in providing feedback on interagency collaboration efforts, whereas case studies and focus groups will provide depth to the data collected through the instrument. Additionally, these tools should assist in evaluating the outcomes of the process evaluated through the MCITT Transition Self-Assessment.
Discussion
Ongoing evaluation of the transition process is a key component of the MCITT's mission. Combined with the other strategies noted here, the MCITT Transition Self-Assessment Instrument will assist the committee in understanding the strengths of the county's process, as well as recognizing those areas that need further development or modification. Additionally, the MCITT Transition Self-Assessment Instrument will assist in documenting the transition model posited by Kohler (1995) and, coupled with future outcome data, may lend empirical support to proposed transition "best practices."
Not only has this committee utilized research in best practices to structure its evaluation, the process itself is a reflection of transition best practices identified through research (Kohler, 1995). The importance of ongoing evaluation of transition programs is clearly reflected in the taxonomy, as is the focus on interagency collaboration. The efforts by the MCITT, then, demonstrate significant gains in bridging the gap between research and practice. Through its efforts, the MCITT has lent credence to current research while at the same time demonstrating the applicability of the research findings to its own community.
Author Note
The authors gratefully acknowledge the dedication of the MCITT and core team participants to continually evaluate the effectiveness of the Midland County transition process. Special thanks to Linda Parent, Ray Landis, Rebecca Shuman, and Lynn Harris for their efforts in developing the evaluation instrument. Thanks also to the other members of the MCITT who participated in the evaluation process: Jeff Rowley, Mary Yeoman, Melissa Davert, Carolann Maxwell, Cheryl Morand, Tom Moline, Wayne Peters, Sandy Cobaugh, Marlene Porter, Ormand Hook, Rebecca Krabill, Sherry Kalina, Dave Murley, Laura Freed, Sue Arent, Dan Simonds, Susan Love, Dena Baronn, Ruth Sutton, Nancy Krumel, Dave Corcoran, Leonard lukshiatis, Art LeTourneau, Mike Shea, Mark Handler, Paul Baker, Chris Chamberlain, and all of the past MCITT participants. We would also like to thank the students and their family members for the privilege of working with them.
Please contact the Transition Research Institute for a copy of the MCITT Transition Self-Assessment Instrument.
Greene, G., & Albright, L. (1995). "Best practices" in transition services: Do they exist? Career Development for Exceptional Individuals, 18, 1.
Johnson, J.R., & Rusch, F.R. (1993). Secondary special education and transition services: Identification and recommendations for future research and demonstration. Career Development for Exceptional Individuals, 17, 1-18.
Kohler, P.D. (1993). Best practices in transition: Substantiated or implied? Career Development for Exceptional Individuals, 16, 107-121.
Kohler, P.D. (1995). A taxonomy for transition programming. Champaign: University of Illinois, Transition Research Institute.
Ory, J.C. (1990). Utilizing evaluation results (Evaluation Technical Assistance: Dissemination Series). Champaign: University of Illinois, Department of Special Education, Transition Research Institute.
Wentling, T.L. (1989). Designing and preparing evaluation reports (Evaluation Technical Assistance: Dissemination Series). Champaign: University of Illinois, Department of Special Education, Transition Research Institute.
To cite this article: Millar, D.M., Shelden, D.L., & Kohler, P.D. (1986). Evaluating transition services: Development of a transition services assessment. In P.D. Kohler (Ed.),Taxonomy for transition programming: Linking research and practice (pp. 63-71). Champaign, IL: Transition Research Institute. University of Illinois at Urbana-Champaign.