Conducting the Site Review


[Back to Table of Contents]


Expertise of the Review Team

            Given the dual function of the site review process (i.e., technical assistance to individual UAPs and program criteria compliance monitoring for ADD), it is obvious that the site review team should be very familiar with the nature and mission of UAPs, the settings in which they work (university and community) and the services and supports needed by individuals with developmental disabilities and their families. The team members also need to be able effectively and quickly to assimilate an individual UAP's operation and program components for the time on-site is relatively short and the opportunities for information-gathering are many.

[Top]   [Next]


Supporting Consumer Involvement in the Review Process

            It is important for the review team to be sensitive to, and proactive about, support for consumer involvement in the review process. This extends to consumers who are members of the review team as well as to those consumers who serve in staff, advisory or other functions for the UAP. Care should be taken to ascertain and provide any accommodations necessary to assure full participation by consumers in all aspects of the review process.

[Previous] [Top]   [Next]


Confidentiality

            During its time with a UAP, the team will have access to a great deal of sensitive information about the organization. All information and materials that come into the purview of the review team should be considered privileged. Team members should not discuss negative or sensitive information with anyone not directly involved with the review. The team has a responsibility to treat the review and the team's observations with objectivity and balance. It is a near certainty that any UAP reviewed will be found to be doing some very good things that deserve to be highlighted as well as doing some things which could be strengthened. It is the review team's responsibility both to the individual UAP as well as to the UAP network and to ADD to highlight the positive aspects of the UAP's activities and to suggest constructive ways in which to strengthen those aspects which need attention.

[Previous] [Top]   [Next]


Responsibilities of the Individual Members of the Review Team

            Responsibilities of the Chairperson. The Chairperson has responsiblities to the team, ADD and the UAP under review at three distinct points in the process: pre-visit responsibilities; on-site responsibilities; and post-visit responsibilities.

             Pre-Visit Responsibilities of the Chairperson. Once the review team has been appointed, one of the first things a Chairperson should do is to make a "welcoming" call or other contact to each team member to establish the team's communication lines.

            ADD (or its designee) will see that a standard package of review materials is sent to each team member well in advance of the scheduled visit. At minimum, that package should include:

  1. Handbook for Conducting UAP Site Reviews and Self-Assessments;
  2. Guidelines for the Development and Maintenance of Quality UAPs;
  3. Suggested Interview Questions manual;
  4. Site review team list; and
  5. Various forms and instructions for handling expenses of the visit.

            Each team member will also receive pre-visit information about the UAP to be reviewed. The Chairperson should work with the Director of the UAP to obtain pre-visit copies of information about the UAP that team members need to review ahead of time. This type of information is often packaged into a " Briefing Book" containing information deemed important by the UAP for the team to know including the 's Self-Assessment of its compliance with UAP Program Criteria.

            During the pre-visit time, the Chairperson should serve as the link between the UAP and the team and should remain in close contact with the UAP Director. One important function of this liaison is to develop a site review agenda. Once the agenda is finalized by the Chairperson and Director and agreed upon by an ADD representative, it should be distributed to all team members, ADD, and the UAP.

            One note of caution in the development of the agenda: The number of structured presentations about UAP projects or services should be limited, so as to allow enough time during the visit to accomplish all the information gathering that needs to be done. Most UAPs have many worthy projects and everyone concerned is anxious to share information about them to the review team. The team needs to remember that its objective is to review the program in its entirety, not to review each individual project. Therefore, the objective in developing the agenda should be prudently to samplethe UAP's projects to get an overall sense of its effectiveness.

            In the case of an ADD-initiated review, the Chairperson should arrange a time with the ADD representative for a phone call toward the end of the visit. The purpose of this phone call is to give ADD an early briefing on the preliminary findings of the review team and to discuss any issues that may require guidance from ADD. The call is usually scheduled near the very end of the visit and prior to the oral exit interview with the UAP.

            Chairperson Responsibilities During the Site Visit. The Chairperson's responsibility to serve as the link between the team and the UAP extends to the time on-site as well. The Chairperson should communicate with the Director throughout the site review to address needed changes in the schedule, to request additional information or resolve any issues which may arise. No matter how good the pre-visit preparation has been, changes to the agenda are common and can be expected. Interview schedules for individuals may need to be changed, the need for an interview not anticipated may arise, etc. It might be advantageous to begin each day in an executive session discussing the 's events with the Director.

            The Chairperson is also responsible for briefing the team soon after arrival. Such a briefing may include:

bullet  A review of the site visit's purpose and objectives along with the agenda;

bullet  Site review procedures involving confidentiality, special assignments for team            members and time frames;

bullet  The special needs of the team members;

bullet  Questions which team members may have following their review of the pre-site          review documents;

bullet  The identification of additional materials needed;

bullet  Making arrangements with the team for the development of the oral exit interview and the final report.

            In addition to an early team briefing, it may be useful to begin the site visit with an evening meeting of the review team and the UAP Director and selected staff. In such a meeting, last minute changes to the agenda can be negotiated and explained, the team can be given an overview of the UAP and can identify questions or concerns or special situations. Briefing information beyond that distributed prior to the visit may be provided at this initial meeting as well. Information on specific program components, locations and/or orientation material might prove useful. Additional sources of evidence of program criteria compliance beyond that distributed in the Briefing Book could be presented. For example, a multimedia presentation which would be difficult to distribute prior to the visit could facilitate a clearer concept of the program, its various components and the setting in which it operates.

            Most teams would find it helpful to schedule "check-in" times at the end or beginning of each day to compare notes, assure taht necessary tasks are being accomplished, and to continue the process of developing a team perspective about the UAP and its activities. On the last evening of the visit, the team should schedule a fair amount of time for processing the information gathered and for preparing the oral exit interview which is usually scheduled as the final review activity. At this meeting, the team can decide how to handle the exit interview report, perhaps by assigning each team member the responsibility to report on a different aspect of the review (one person reporting on the team's observations about preservice training, another on the community services component, etc.).

            The final thing the Chairperson needs to do is to be sure that all team members have a clear understanding of their responsibilities for completing the final report to ADD.

            Post-Review Responsibilities of the Chairperson. The primary post-review responsibility of the Chairperson is to coordinate and submit the team's final report of the review to ADD (in the case of an ADD-initiated review) or to the UAP (in the case of a UAP-initiated review). An explanation of the final report is given later in this Handbook and a suggested final report outline is given in APPENDIX C.

            Responsibilities of the ADD Consultant(s).All members of the review team, of course, have the responsibility to provide technical assistance to the UAP and monitor program criteria compliance monitoring for ADD (in the case of ADD-initiated reviews). The primary role of the consultant member(s) is/are to bring to bear her/his/their knowledge of UAP activities, the diversity of UAPs and the communities they serve, and of how the ADD program criteria can be met in a variety of ways. Depending upon the specific situation the review team encounters in its visit, the team may wish to split the duties in such a way as to take advantage of the consultant's specific areas of expertise. For example, if the consultant has particular expertise in preservice training, it may make sense to use that person as the primary team representative in assessing the UAP's interdisciplinary training program.

            Following the site visit, the consultant member(s) of the team will contribute to the writing of the final report.

            Responsibilities of the Consumer Representative(s).As stated above, all members of the review team serve in a general capacity to help the UAP under review strengthen its program as well as being the eyes and ears of the ADD to ensure compliance with UAP program criteria. However, the consumer representative also has an additional special function--to pay close attention to the consumer responsiveness of the UAP and its various programs. Involvement of consumers in all aspects of UAP programs has grown increasingly important to the network and is heavily underscored in the most current listing of UAP program criteria. It is the responsibility of the consumer representative to assure that the consumer involvement in UAP activities is appropriate and consistent with this emphasis.

            Following the site visit, the consumer representative member(s) of the team will contribute to the writing of the final report and, importantly, should make individual statements of his or her overall observations about the UAP from a consumer perspective.

[Previous] [Top]   [Next]


Briefing Information Material to be Available Prior to the Site Visit

            Prior to the site visit, the UAP Director and the review team Chairperson should consult on the development of the pre-visit written materials which need to be distributed to the review team with sufficient lead time to allow team members the opportunity of framing questions to be asked once on-site. This pre-site material development is also an extremely important activity for the UAP and its staff. It allows (and indeed forces) the UAP to look more closely at itself and its activities than it perhaps does on a routine daily basis to assess the degree to which its programs and activities are in line with the ADD program criteria. As is the case with professional program reaccreditation, it is a time to pull back and ask "What are we doing and how well are we doing it " This should be seen as a renewal time, a time to take stock and honestly assess the program's strengths and areas needing further development.

            One way of assembling the results of this pre-visit self-evaluation is for the UAP to prepare aBriefing Book in which is presented sufficient background information that the review team will have a very good idea of the UAP, its programs and activities before they arrive on-site. That way, the majority of the site time can be spent in conversations to clarify questions, corroborate information and provide technical assistance.

            The mandatory items which need to be completed and forwarded to the review team prior to the site visit are:

bullet  The result of the 's self-assessment in the form of the Self-Assessment Check Sheet. A copy of the Check Sheet is provided in APPENDIX B;

bullet  Site visit agenda;

bullet  The latest ADD continuation grant proposal.

            Different UAPs may want to provide somewhat different information in their Briefing Books, but in general the following information may be useful to provide ahead of time to the team:

bullet  UAP Mission Statement

bullet  Brief UAP History and significant stages of its development

bullet  UAP Personnel Rosters (including advisory councils)

bullet  Description of the Consumer Advisory Committee, its activities and outcomes

bullet  Organizational chart

bullet  Governance plan (including the UAP's charter with the university)

bullet  Core function written plans

bullet  Overview of projects and activities

bullet  Latest ADD report and/or continuation grant proposal

bullet  UAP database reports (NIRS)

bullet  Self-Assessment Check Sheet

[Previous] [Top]   [Next]


Documents and Information to be Available On-Site

            Data and documents related to the self-assessment should be made available to the team both prior to and during the site visit. These materials should be carefully reviewed by the team members as part of the effort to document compliance and identify deviation from traditional compliance criteria.

            Materials sent to the review team prior to the visit typically describe and list the activities and productivity of the UAP. Such descriptions often include lists of published materials such as reports, instructional materials, journal articles, books, grant narratives, newsletters, classes taught, workshops conducted, consumers served and students trained. The UAP should make evidence available on-site that verifies such listings. Many UAPs have found it useful to compile the material and evidence in a "documents room" or area to which team members have access and in which material can be displayed in such a way that team members can browse through it or even study it in some detail. Other types of material which may prove useful to the 's investigation and to the UAP's documentation of its activities and program criteria compliance could include:

bullet  Copies of UAP governance plans;

bullet  Interdisciplinary training plans (including student learning plans if available);

bullet  Community Services plans;

bullet  Dissemination plans;

bullet  Detailed organizational charts;

bullet  Curriculum vitae of key administrative and faculty positions;

bullet  Evidence of collaboration with the Developmental Disabilities Council;

bullet  Copies of agreements of affiliation with appropriate local and state agencies and organizations;

bullet  Books, articles, conference proceedings, newsletters and other materials disseminated by the UAP;

bullet  Detailed descriptions of projects and services;

bullet  Any other material deemed useful in documenting the UAP's activities.

The UAP should also provide an explanation of its funding. This could include:

bullet  Sources of support (ADD, university, state/federal grants, contracts, fees);

bullet  The purpose of such support including the activities that are funded;

bullet  The individuals associated with the project (principal investigators, project directors);

bullet  Amount of funding and duration of awards.

The UAP should also supply a history of its funding pattern over the past several years to help site reviewers better understand the program's strengths and weaknesses in leveraging funding for its program components. Fiscal documentation should also include descriptions of indirect costs and recovery rates, fee schedules and techniques or approaches the UAP uses to leverage core support. In addition, the UAP should provide descriptive information on the accounting practices of the host university. Interviews with university budget and grant officers may be helpful.

            The site review team should also review consumer outcome data. This may include consumer satisfaction data as well as evidence of the impact UAP services have had on consumers and their families and on the developmental disabilities service system in general.

[Previous] [Top]   [Next]


Site Review Activities

            Specific activities and combinations of activities will differ from UAP to UAP depending upon the size and comprehensiveness of the program, expertise and assignments of team members, purpose of the site review, and the setting in which it is conducted. Typical site review activities will include interviews with appropriate individuals and groups and other activities necessary to validate the UAP's self-assessment of compliance with the UAP program criteria.

            Interviews. Depending upon the size and complexity of the UAP to be reviewed, the Director and Chairperson may wish to consider developing "tracks" in the site visit agenda in which the review team is split into sub-units to interview different people and groups of people at the same time.

            Where possible, group interviews are an efficient mechanism for the team's information-gathering activities. They provide opportunities for structured presentations by individual UAP staff members, for reviewing major components of larger UAPs, for question and answer interactions with UAP personnel, for meeting with committees (such as the Consumer Council) and for sharing perceptions, data and information.

            Individual interviews are usually limited to key faculty and staff, UAP committee chairs and university administrators. Individual interviews can be scheduled concurrently for one or two team members while other team members meet in groups with other relevant personnel. Individual interviews may also be scheduled for site review teams to meet with consumers, family members, trainees, agency persons, etc. where confidential or sensitive information may be shared.

            Occasionally, key individuals such as administrators of consumer organizations, state programs, former trainees or directors and administrators of major components of the UAP may not be available to participate in on-site meetings with the review team. Telephone interviews (either individually or conference) or other such technology-based arrangements might be made to facilitate the team's connections to those individuals. For example, interactive video connections are now often available between the host university and many State locations.

            Interviews, whether done individually or collectively in groups, are best guided by specific questions formulated around evidence of compliance with ADD UAP program performance criteria and the adequacy of interdisciplinary training, community service activities and dissemination. Specific requests for information and data should also be coupled with more general, open-ended questions which provide the opportunity for the interviewee to expand and extrapolate.

            A separate document supplied to the review team as part of the ADD-supplied materials includes a list of suggested questions. The Suggested Interview Questionsmanual is spiral-bound and tabbed for quick reference to facilitate its use as a tool for physically recording information during the on-site interviews as well as to help the team maintain focus on its information-gathering duties. The questions are grouped into categories as follows:

bullet  Mission and Values

bullet  Governance

bullet  Management

bullet  Decision-Making

bullet  Consumer Responsiveness

bullet  Interdisciplinary Preservice Training

bullet  Community Services: Training, Supports and Direct Services

bullet  Community Services: Technical Assistance

bullet  Dissemination

bullet  Research and Evaluation Activities

bullet  Individual Interviewer Questions

bullet  Notes for Oral Exit Interview

The "Individual Interviewer Questions" section provides an opportunity for each team member to ask specific questions not covered by the questions in the preceding sections. "Notes for Oral Exit Interview" section provides a place for the team to record notes during the site visit to be used in guiding the Exit Interview with the appropriate UAP and university persons.

            The Suggested Interview Questions document is constructed to complement the Suggested Final Report Outline (APPENDIX C) and to inform the development of both the Oral Exit Interview and the Final Report to ADD.

            Validation of the Self-Assessment of Program Criteria. A major purpose of the site visit is to review the program's compliance with the UAP program criteria as established by the ADD. The current program criteria were published in the Federal Register on September 30, 1996. Those criteria are summarized in APPENDIX A.

            The UAP being reviewed must conduct a self-assessment with the objective being to rate its compliance with current program criteria. To help guide that process, a Self-Assessment Check Sheet is provided in APPENDIX B. The Check Sheet summarizes the program criteria and allows the UAP to rate its progress in meeting those criteria on a 5-point scale ranging from "Not Addressed" to "Fully Implemented. " Space is provided for the UAP to write a narrative rationale or to point toward evidence to substantiate its self-rating. The narrative comments may indicate where documents, data or information will be available for review during the site visit. Space is also provided for the review team to write comments and observations about the self-assessment item, for example, as a reminder to ask for more detail or clarification while on-site.

            It is reasonable to expect that a great deal of the compliance documentation will be included in the continuing grant application, the UAP database reports (NIRS), and in annual and semi-annual reports. Other information may only be available during the site visit and can be obtained through interviews, reviewing policies and procedures, documents and confidential records. The site review Chairperson should ensure that assignments to team members are made and that sufficient time is allotted to complete the team's compliance verification process.

            Certain information is best validated through interviews or direct observation. Governance, relationships with state agencies and state Developmental Disabilities Councils, and university and faculty status are often better verified through interviews than by document review. Interviews are often helpful to verify the authenticity or accuracy of the documents reviewed. For example, memoranda of agreement may be written or training plans may be filed, but the determination of whether such plans or agreements are in fact implemented and effective can only be obtained by direct observation or by an interview. Questions about compliance or unique situations should be noted and discussed with the responsible staff.

[Previous] [Top]   [Next]


Site Review of major UAP Functions and Components

            The review team should schedule a significant portion of its time to review the overall UAP mission and program components (management and operations; consumer responsiveness; interdisciplinary training; community services; and dissemination).

            The rules and regulations issued in 1996 to implement the Act as amended (PL 104-183) redefined the mandated core functions of UAPs to be:

1. Interdisciplinary preservice training;

2. Community service activities; and

3. Dissemination activities.

Research and evaluation, although not mandated functions, were recognized as important components of many UAPs.

            Mission. The UAP network is diverse. Individual UAPs are organized differently, have different focuses, emphasize different aspects or functions, etc. But no matter how different individual UAPs might be, the one element which binds UAPs into a network is their common mission and dedication to being an effective force in assisting individuals with developmental disabilities to maximize their opportunities for full citizenship in American society. That dedication is manifested by a commitment to the values of independence, productivity, integration and inclusion of persons with developmental disabilities and their families. No matter how an individual UAP gives life to this mission through its individual activities, it is toward that end that all UAPs aim. The job of the review team is to ascertain that this mission and these values truly give direction to and guidance for an individual UAP's activities and programs. Questions in the Suggested Interview Questions manual will assist the team in determining this.

            Management and Governance.By law, UAPs operate within a university setting and therefore will be as organizationally diverse as are the universities themselves. To operate successfully within a university, linkage with other units such as departments or schools or colleges must be developed and maintained. Evidence of such linkages are generally reflected by the commitment of resources, academic appointments and participation on decision-making committees. The site review team should determine the nature, operational characteristics and evidence of the university's commitment and support for the UAP. Evidence for university commitment may include the degree to which the university provides financial support for UAP personnel and activities, how well the university uses the UAP to promote its mission and the mission of the academic units as well as the manner in which the UAP impacts the university environment. The team should determine how the academic units of the UAP are viewed (i.e., as an integral part of the university's academic community, as a training laboratory, as a community service program, or as a research center) and the perceptions of the university faculty regarding the UAP's importance and impact on the university.

            Many expectations and influences impact a University Affiliated Program. These include the particular university setting in which it is located to the expectations of funding agencies that provide support for major components to the expectations of consumers, service providing agencies and the Developmental Disabilities Council. The process of management must achieve a balance between these competing expectations even as support is drawn from them. The review team should determine the programs "ownership, " the controlling parties, how influence is exerted and by whom, and the techniques by which it is channeled.

            In a similar manner, relationships with community-based programs such as state service agencies, Developmental Disabilities Councils and local constituency groups should be reviewed. The nature of these relationships is best demonstrated by the commitment of resources, expectations for assistance and participation on planning and governance committees. The site review team should review how the UAP is perceived by service providers and consumers. What impact has the UAP had? What is the UAP known for? What does it do for them (or what do they think it should do for them)? What part do consumers play in influencing the UAP work scope? Are activities conducted by the UAP seen as exemplary, state-of-the-art and in harmony with the service systems?

            Other issues the team may address include the management style of the UAP, its organizational model and mechanisms for planning, problem solving, conflict resolution and consensus building.

            The site team may analyze the diversity of funding, how funding is leveraged, the conditions or expectations of funding sources and the stability of the UAP's fiscal support. To determine the degree that fiscal issues promote or restrict the UAP from fulfilling its mission, the site review team should determine (a) if funding is adequate; and (b) if support is from one (or a few) source(s) or is appropriately diversified.

            Consumer Responsiveness. Over the years, the responsiveness of UAPs to consumer concerns and perspectives has grown more critical. Although consumer responsiveness has always been an important aspect of UAP operations, the last two reauthorizations of the Developmental Disabilities Assistance and Bill of Rights Act have increasingly emphasized this aspect. Site review teams should attempt to determine the extent to which the UAP demonstrates in its programs and its interactions with consumers its sensitivity to consumer input and perspectives. Is the UAP "consumer-friendly"? Is the dignity of consumers respected in staff language and behavior and in the UAP's written materials? To what extent are consumers active and valued participants on the UAP staff, committees, councils, boards and advisory bodies? Does the UAP provide necessary accommodations needed by consumers for full participation? To what extent are consumers actively involved in helping to set training, service and research priorities?

            Another critical dimension of consumer responsiveness has to do with the UAP's ability and willingness to deliver its services with cultural competence. "Cultural Competence means provision of services, supports or other assistance in a manner that is responsive to the beliefs, interpersonal styles, attitudes, language and behaviors of individuals receiving services and that has the greatest likelihood of ensuring their maximum participation in the program" (Federal Register, 1996, Section 1388.1).

            The site team needs to validate that the UAP is committed to individuals with disabilities, family members, staff, trainees and community members from diverse cultural backgrounds in all levels of its activities. This commitment includes an active attempt to recruit individuals from minority backgrounds into the field of developmental disabilities in order to address the growing needs of an increasingly diverse American society.

            The consumer member of the team can be particularly valuable in guiding the team's observations in this area.

            Interdisciplinary Training.The primary objective of UAPs is to provide interdisciplinary training to prepare personnel to serve persons with developmental disabilities. Interdisciplinary training, however, has been defined in a variety of ways. The site review team should determine how the UAP has defined and implemented interdisciplinary training. Answers to the following areas of concern may help ascertain the scope and extent of the UAP's interdisciplinary training program:

bullet  Home discipline of trainees;

bullet  Level of skill trainees have in serving individuals with disabilities before entering the program;

bullet  Curriculum and the interdisciplinary training experiences provided by the UAP (knowledge, skills and awareness level);

bullet  Financing of interdisciplinary training and availability of stipends, graduate assistantships, certified practica and field experience opportunities;

bullet  Stated goals and purposes of interdisciplinary training;

bullet  Degree to which UAP faculty are committed to interdisciplinary training and how it is carried out;

bullet  View of the UAP's interdisciplinary training activities by members of other academic departments;

bullet  Method by which the UAP monitors student progress through the interdisciplinary training program;

bullet  Follow-up data on trainees to determine the effectiveness of the training, the impact it has had on the system and if such data are used to revamp the training program;

bullet  How skills are imparted (e.g., through clinical experience, lectures, reading or other activities);

bullet  Type of settings in which interdisciplinary training takes place (clinical, educational, community-based, hospital, etc.);

bullet  Interdisciplinary training responsibility (one individual who directs the program, a committee within the UAP or the responsibility of different projects);

bullet  Degree to which trainees participate in problem identification, problem solving, group decision-making as well as clinical practices;

bullet  Availability of individualized interdisciplinary training plans;

bullet  Comprehensiveness of interdisciplinary training plans and the focus on specific skills and information;

bullet  Presence of a core interdisciplinary training curriculum with stated or optional experiences and activities;

bullet  Established methods of measuring students' skills and knowledge;

bullet  Spectrum of interdisciplinary representation among the faculty.

            Community Services and Supports. Community service and support activities are defined as community services (projects and direct services), community training and technical assistance. Over the years, UAPs have come increasingly to focus their efforts toward community assistance models and less on the provision of direct services. This newly defined category of "community service and supports' is an outgrowth of that evolution and focuses attention on the activities that UAPs provide which help to increase community capacity to more effectively support individuals with developmental disabilities and assist them in becoming more independent, productive and integrated into the full fabric of community life. In sum, community service and supports encompass those activities which UAPs provide to help communities become more inclusive, more supportive of and more welcoming to individuals with developmental disabilities.

            The review team needs to determine the extent to which UAP community service activities:

bullet  Use capacity building strategies to strengthen the capability of communities, systems and service providers;

bullet  Plan collaboratively, including the participation of individuals with developmental disabilities and their families;

bullet  Target a wide range of audiences, including individuals with disabilities, their families, service and support personnel, and community members;

bullet  Are structured in a manner that facilitates the participation of the targeted audiences;

bullet  Address the unique needs of individuals with developmental disabilities and their families from diverse cultural and ethnic groups who live in the UAP's catchment area.

bullet  Address life-span issues;

bullet  Are delivered at times and in places that are consistent with community standards;

bullet  Interact with and involve community members, agencies and organizations;

bullet  Respond to a local or universal need that reflects critical problems in the field of developmental disabilities or to an emerging critical problem which reflects a trend or anticipated development in the field.

            In 1996, direct services became an option within this category of mandated core functions and the following requirements only apply where direct services are offered. In the case of UAPs which deliver direct services, the team should determine how effective the program is in:

bullet  Integrating the services into community settings;

bullet  Involving interdisciplinary student trainees, professionals from various disciplines, service providers, families and/or administrators;

bullet  Including adults and elderly individuals with developmental disabilities;

bullet  Maintaining cooperative relationships with community service providers;

bullet  Scheduling service times and places consistent with community standards;

bullet  Delivering services which are in response to a universal need reflecting a critical problem in the field of developmental disabilities or to an emerging, critical problem.

          Teams should note state-of-the-art and innovative practices of the UAP. This may provide the team an opportunity to highlight successful practices and facilitate letting the full UAP network know about them. Such practices include services and projects that:

bullet  Facilitate and demonstrate independence for individuals, community integration and inclusion, productivity, and human rights;

bullet  Are highly beneficial to individuals with developmental disabilities and are accepted by various disciplines;

bullet  Are innovative, cost-effective and are evaluated according to accepted practices of scientific evaluation;

bullet  Demonstrate research methods used to test hypotheses, validate procedures and field test projects;

bullet  Are practices and models that are evaluated, packaged for replication, and disseminated through the information dissemination component.

            Dissemination. Dissemination is an important aspect of UAP programs and activities. Some UAPs have developed extensive dissemination plans and mechanisms to collect, evaluate and distribute information. In other UAPs, dissemination activities are expected to be incorporated into other components (training, service, technical assistance and research and evaluation). Dissemination may include newsletters, technical reports, publications in scholarly journals, and descriptive or awareness information. Priority should be given to activities that synthesize research and information to improve practices and quality of life for individuals with developmental disabilities.

            The site review team should determine the nature of the UAP's dissemination activities, how dissemination is defined, how it is monitored and the extent to which the UAP has attempted to evaluate the results and outcomes of such dissemination activities.

            The following are some things concerning dissemination the team may want to determine:

bullet  Whether there is a written, coordinated dissemination plan;

bullet  Who is responsible for dissemination activities;

bullet  Whether the UAP channels dissemination activities through an individual or a unit;

bullet  Payment for the material the UAP disseminates;

bullet  The extent to which the dissemination mechanism is used and is known (occurrence of repeat orders, amount of use, etc.);

bullet  If the dissemination procedure is evaluated for effectiveness;

bullet  What the method is for monitoring dissemination activities and products;

bullet  Whether dissemination activities are taking advantage of other UAP activities such as research, curriculum development and training programs that are developing materials;

bullet  If dissemination activities take advantage of other information generating resources at the university and within the UAP network.

            Research.Research, particularly applied research, has been an integral part of the UAP effort since the programs were first established in 1963 (Fifield et al., 1989). The 1983 ad hoc workshop recommended that UAPs be encouraged to conduct research and that research activities be summarized and reported to ADD (Fifield et al., 1989).

            Even though research is not included as a mandated core function of UAPs, the last set of program criteria recognizes its importance: "...the UAP refines its activities on the basis of evaluation results. As members of the university community, involvement in program-relevant research and development of new knowledge are important components of " (Federal Register, Sept. 30, 1996; 45 CFR Part 1388.1). For many UAPs, especially those associated with research centers or institutes, research activities represent a significant part of their overall program and are an important means of improving training of personnel and services for individuals with developmental disabilities. Efforts of UAPs to promote and conduct research are an important means to carry out training, service and dissemination activities.

            The efforts of a UAP in research and the relationship of research to other program components should therefore be reviewed by the team. The team may want to review the nature and extent of research activities in the UAP and how that research relates to the mandated UAP core functions (i.e., interdisciplinary preservice training, community services, and dissemination). The site review team may want to determine the following:

bullet  The relationship of research centers or institutes with the UAP;

bullet  Sources of research support;

bullet  Current research directions of the UAP;

bullet  Research reputation of the UAP as viewed by other academic units in the university;

bullet  Faculty responsibility for developing and pursuing research;

bullet  Method of disseminating research information;

bullet  Whether the research activities are taking advantage of other UAP activities such as training, clinical or community services programs.

[Previous] [Top]   [Next]


Site Review Schedule

            The Review Team Chairperson should work with the UAP Director to develop a schedule for the site visit. This schedule needs to be approved by ADD before it is finalized. After it is finalized, advance copies should be shared with everyone connected to the review process. It is the Chairperson's responsibility to make sure that each team member has a copy (it could be included in the Briefing Book which is supplied to each team member prior to the visit) and it is the Director's responsibility to make sure that it is distributed to all relevant persons at the UAP.

[Previous] [Top]   [Next]


Site Review Evaluation Final Report

            A written site review report is to be submitted by the review team as soon as possible after the visit is concluded. The report should address the purpose of the site review, procedures followed, findings and recommendations. The value of the report is often directly proportional to the time it takes to get it back to the program.

            Typically, a team will have much of the Final Report written (or at least outlined) before physically leaving the UAP site. Most teams would find it convenient and effective to carry with them a laptop computer with the report outline (see below and APPENDIX C) already loaded onto the machine. As the team meets together during its visit, many notes can then be put together forming the basis of both the Exit Interview and the Final Report. The final preparation, organization and submission of the report is generally the responsibility of the team Chairperson. Any writing or additional information-gathering assigned to the various team members following the visit should be clearly outlined with deadlines established before the team concludes the review to assure an efficient manner for finishing the report in a timely fashion.

            Once the Chairperson has received the team members' contributions and has edited the document, it generally will be sent out to the team members for a final review and agreement before submission. In the case of ADD-initiated or sanctioned reviews, the Final Report goes directly to ADD which then takes the report under advisement and issues its own report to the UAP. Thus, in this case, the review team does not send a report directly to the UAP; that is the responsibility of ADD. In the case of a UAP-initiated review, the UAP Director may also be involved in the final review process and the Final Report may be sent directly to the UAP.

            Site evaluation reports will vary in length and in detail, but all should cover certain common elements to ensure thoroughness and comparability of information across the network. A recommended format is given in APPENDIX C and the individual sections are explained below.

            Introduction. The introduction of the site review report should explain the nature and purpose of the review (i.e., ADD or UAP initiated). The purpose of the site review should include an explanation of the specific objectives the site review team was asked to address. For example, an ADD-sponsored site review might have as its primary objectives "to assist ADD in monitoring compliance with the UAP Program Criteria and to provide peer review, feedback and technical assistance" to the UAP being reviewed. A UAP-initiated site review might have as its primary objective technical assistance on a specific component or program area. Identifying the specific objectives provides the basis for conducting the site review and for determining the direction, impact and relevance of the Final Report.

            The introduction should also include a list of the team members and an explanation of their titles, affiliations and roles on the team as well as a list of persons interviewed on-site (staff members, university officials, Developmental Disabilities Council members, advisory council members, trainees, consumers, etc.).

            A final section of the introduction should explain the major activities accomplished prior to the site visit by the site review team (e..g., development of the schedule, review of the UAP Self-Assessment materials, etc.).

            Findings. The Final Report should include any relevant background information deemed necessary to the report (e.g., where the UAP is housed within the university; any uniqueness about its structure or leadership; etc.). It should also include a section describing the team's review and validation of the UAP Self-Assessment. This section may vary in length depending upon what needs to be reported, but could be as short as a few sentences. (e.g., "The UAP faculty and staff find themselves in compliance [or at a level of partial compliance and in progress toward full compliance] in all areas and with all individual criteria. The Site Review Team concurs [or does not concur] with that conclusion")

            The majority of this part of the Final Report should be given over to reporting the team's findings and Recommendations. A word about terminology is in order at this point. For purposes of comparability of reports, the following definitions should be used:

  1. Recommendations represent strong statements or conclusions drawn by the team and are meant to be taken particularly seriously by the entity receiving the report (i.e., ADD and/or the UAP).
  2. Suggestions are intended to be observations made by the team which are to be understood at a lower level of statement strength. That is, Recommendations are statements the team feels need to be addressed by the UAP under review while Suggestions are statements the team feels ADD or the UAP might wish to consider.

            In general, the "Findings" portion of the report should include a description of the strengths and weaknesses of the UAP as well as the unique features identified by the site review team. The findings should highlight outstanding, exemplary or unique solutions to problems and should draw attention to innovative or cutting-edge activities the UAP is conducting along with areas that need additional attention.

            The findings should lead directly to Suggestions and Recommendations, which usually can be embedded conveniently into the Findings narrative at the point of discussion and then later summarized in the final section of the report. Suggestions and Recommendations may be most helpful to a UAP if additional information is provided such as the rationale for the recommendation(s), possible avenues of assistance in addressing the recommendation(s), etc.

            The recommended format for the Final Report as given in APPENDIX C organizes the report of findings into five major subdivisions:

  1. Administration/Governance/Operations;
  2. Consumer Responsiveness;
  3. Interdisciplinary Training;
  4. Community Services and Technical Assistance; and
  5. Dissemination.
Each subdivision is further divided into three sections:
  1. Overall Impressions;
  2. Strengths; and
  3. Opportunities for Future Development.
This format provides an opportunity and a guide for addressing the major aspects of UAP operations from an ADD perspective: Overall UAP governance; Responsiveness to consumers and consumer issues; and the current UAP program criteria.

            Suggestions and Recommendations. The last segment of the Final Report should be a summary of the team's overall impressions, areas deemed to be strengths of the UAP and the Suggestions and Recommendations made throughout the report.

            Review teams should shape their Recommendations and Suggestions in such a way that would strengthen the continuation core grant application, i.e., that would provide positive technical assistant to the UAP (e.g., suggesting that data and information might better be organized and reported in another format; suggesting a plan to address areas needing future development; etc.).

            In the case of an ADD-initiated review, the Final Report is written specifically for the agency and the report is sent directly to ADD, not to the UAP. After receiving the advice of the review team, ADD will then fashion a report to the UAP taking the review team's observations and conclusions into account and under advisement. For UAP-initiated reviews that are intended to be completed at the request of the individual UAP for technical assistance, the report would be sent directly to the UAP.

 

 

[Previous] [Back to Table of Contents] [Top]