Academic Affairs Handbook, Section 2.03.05

Comprehensive Program Review

SUBJECT: Comprehensive Program Review
SOURCE: Board of Regents Policy 205.01; Memorandum from the Senior Vice Chancellor for Academics and Fiscal Affairs, 10/18/00

Note: References to student learning outcomes in this document have been highlighted in color.

Note: This policy and its accompanying procedures were developed by the Administrative Committee on Institutional Effectiveness and endorsed by the Administrative Committee on Academic Affairs on July 18, 2000

POLICY

205.01 COMPREHENSIVE PROGRAM REVIEW

1. Each University System institution shall conduct academic program review on a periodic basis. Consistent with efforts in institutional effectiveness and strategic planning, each University System institution shall develop procedures to evaluate the effectiveness of its academic programs through a systematic review of academic programs, to address the quality, viability, and productivity of efforts in teaching and learning, scholarship, and service as appropriate to the institution's mission. The review of academic programs shall involve analysis of both quantitative and qualitative data, and institutions must demonstrate that they make judgments about the future of academic programs within a culture of evidence.
    
2. The cycle of review for all undergraduate academic programs shall be no longer than seven years and for all graduate programs no longer than ten years. Programs accredited by external entities may substitute an external review for institutional program review, provided the external review meets University System and institutional requirements for program review. If an external accreditation entity?s review cycle for undergraduate programs is ten years, the ten-year review cycle may be used for that program only. No program review cycle at any level shall exceed ten years.
    
3. The Senior Vice Chancellor for Academics and Fiscal Affairs must approve each institution's plan for the conduct of a complete cycle of program review and may require changes in the plan, providing adequate time for the change to be implemented. Each institution shall conduct program review according to the terms of its approved plan, with annual updates and requests for changes to the plan as necessary.
    
4. Planning and conduct of academic program reviews shall be used for the progressive improvement and adjustment of programs in the context of the institution's strategic plan and in response to findings and recommendations of the reviews. Adjustment may include program enhancement, maintenance at the current level, reduction in scope, or, if fully justified, consolidation or termination. Actions taken as the result of reviews and strategic plans shall be documented as outlined in Board Policy 205.01.E below.
    
5. Each institution shall submit an annual program review report to the Senior Vice Chancellor for Academics and Fiscal Affairs, which shall include a list of academic programs reviewed and a summary of findings for programs reviewed during the previous year. The institution must summarize actions taken both as the result of current reviews and as follow-up to prior years' reviews. For each review, institutions must establish that the program has undergone review and is meeting rigorous standards. The report must identify (1) quality, viability, and productivity parameters measured, and (2) findings relative to internal standards, the institution's strategic plan, and, as appropriate, external benchmarks.
    
6. The Senior Vice Chancellor for Academics and Fiscal Affairs shall monitor annually a small number of performance indicators for academic programs and shall initiate dialogue with the Chief Academic Officer of the institution when programs do not meet the guidelines defined by the indicators. If further investigation justifies additional study, the institution may be asked to conduct an off-cycle review of such programs.

PROCEDURES

MODEL FOR PROGRAM REVIEW

Introduction
The goal of academic program review in the University System of Georgia is the improvement of programs based on information gathered and analyzed during a cyclical review process. In conducting program review, institutional faculty and administrators assess progress over time, analyze costs and benefits of programs, and make strategic decisions about program modification. The procedures outlined below offer guidelines for rigorous and meaningful review in a context of institutional freedom to design individual review processes, to employ elements of the academic audit model now in use in many institutions, and to select internal and external benchmarks. The procedures also outline how the System Office will annually monitor performance indicators, the results of which might trigger an earlier review than scheduled. The procedures acknowledge the difficulty of conducting reviews for a large and diverse array of programs across the University System but at the same time establish guidelines which are within the reach of all institutions.

Guidelines for Academic Program Review
The purpose of these guidelines is to establish a system for periodic review of academic programs within University System of Georgia institutions. It is understood that academic program review is one component of an overall institutional effectiveness plan; other components include strategic planning, assessment of student learning outcomes, and assessment of outcomes in administrative areas. It is recommended that these guidelines be reviewed periodically to determine their effectiveness in guiding program review within the University System.

Definitions

Normal Program Review Process

Nothing in the following guidelines precludes independent reviews of specific academic programs by the University System or any of its institutions.

Institutional Process

1. Each institution will submit an initial report (one-time submission, updated as needed) to the University System Office. The report will list the date of the last program review, if a review has been conducted, and the accrediting body for the program, if applicable. If the program must be reviewed by an outside entity for accreditation, the review cycle and other descriptive information should be provided.
   
   The initial report will summarize the academic program review plan and process adopted by the institution, outlining the procedures and methods to be used. An initial list of the programs to be reviewed during the institution's full program review cycle will be provided.
    
2. The program under review will conduct a self-study focusing on relevant data such as program admissions, student credit hours, number and percentage of graduates, cost/student credit hour, and resources committed to the program. The self-study should include defined expectations and a measurement of these expectations relative to internal standards and external benchmarks.
    
3. Appropriate input and evaluative elements of an academic program review include:
   
   Mission - program mission, relation to institution mission, relation to University System mission, needs of students, and demand for graduates
   
   Teaching and Learning, Research and Scholarship, and Service - evaluation of these program functions should include, but may not be limited to, the following kinds of elements:
   
   Students - percent and number of majors and graduates, percent and number of graduates passing professional and qualifying examinations, diversity, selectivity, skills at entry, service course enrollments, program applications compared to program capacity, credit-hour generation, and student learning, satisfaction and evidence of success in meeting student needs and learning outcomes.
   
   Faculty and Staff - numbers (part- and full-time), costs, student-faculty ratio, average class size, faculty productivity, maintenance of an adequate faculty cohort for program success, diversity, credentials, and professional development.
   
   Facilities - space (adequacy and condition), cost, technology labs, equipment, library, and other indicators of adequacy of campus infrastructure to support the program.
   
   Curriculum - coherence, currency, relevance to program learning outcomes and student needs, course sequencing or frequency of course offerings, and enrollment patterns.
   
   Other Learning and Service Activities - advising, tutoring, internships, service learning, practica, study abroad, and career planning and placement.
   
   Research and Scholarship - faculty and student involvement, productivity, reputation, level of financial support, mentoring and development opportunities for new faculty.
   
   Service - projects completed and outcomes (program, division/school/college, institution, community and/or region levels) and contributions to mission.
    
4. The program review process must provide for an analysis of the self-study and of the program by a study group of external faculty and outside evaluators, if appropriate and possible. If internal reviewers are used, every attempt must be made to ensure that the review is as objective as possible. As appropriate, academic programs which are professionally accredited may use the self-study and external review processes of reaffirmation to satisfy the academic program review requirements, provided these guidelines are followed.
    
   1. The program review process must provide for input into and analysis of the review by the faculty governance and administrative bodies.
       
   2. The program review process must include a commitment by faculty of the program and appropriate administrators to act upon the findings and recommendations of the review. Each campus will regularly report on unit progress in implementing review recommendations.
       
   3. As appropriate, performance indicators such as the following should be addressed in all program reviews.
      
      Dedicated Resources (Human, physical, fiscal)
      Faculty Qualifications. The program meets all regional accreditation requirements for faculty qualifications. It is suggested that the program meet discipline-based accreditation requirements, whether or not special accreditation is sought.
      
      Faculty/student ratio. The program reports the faculty/student ratio and demonstrates that it is adequate and efficient for its mission. (Suggested: The faculty/student ratio meets the standard recommended by accrediting organizations.)
      
      Instructional technology. The program demonstrates that instructional technology is current and is adequate for the institution's mission.
      
      Facilities and non-instructional technical support. The program demonstrates that its physical facilities and non-instructional technical support are current and adequate for the program's mission.
      
      Learning resources for faculty and students. The program demonstrates that the library and other information resources are current and adequate for the mission and that they meet the requirements of the relevant accreditation body.¹
      
      Budget Resources. There is evidence that the program has financial resources necessary to support its teaching, research, and outreach efforts as appropriate to program and institutional mission.
      
      Program, Learning, and Service Outcomes
      Faculty/graduate ratio. The program's faculty/graduate ratio is computed and judged to be appropriate to the program's mission.
      
      Success of graduates. The program's graduates find appropriate employment or meet post-graduation educational goals in light of market trends and the needs of the state.
      
      Scholarly contribution. The program's scholarly productivity is appropriate to the mission of the program and the institution.
      
      Community service and outreach. The program's activity in community service and outreach is appropriate to the mission of the program and institution.
      
      Retention rates. The program demonstrates that retention rates are appropriate for the program and the student population.
      
      Student learning outcomes. Graduates of the program demonstrate that the knowledge, skills, attitudes, and behaviors of general education have been maintained and advanced. Graduates of the program exhibit mastery of their discipline.
      
      Processes
      Curriculum review. The program demonstrates that periodic review of the curriculum (focusing on coherence, level, and comparison with similar programs) is carried out, based on assessment of learning outcomes and other types of feedback, such as practice in the field.
      
      Design of learning experiences. Learning experiences are designed to reinforce the general attributes of a college graduate through recognized good practices such as the following:
      
      
      High expectations of students
      Coherence in learning
      Synthesizing experiences
      Integrating education and experience
      Active learning
      Ongoing practice of learned skills
      Prompt feedback to students
      Collaborative learning
      Significant time on task
      Respect for diverse talents and ways of learning
      
      
      Attrition rates. The program monitors attrition rates in light of similar rates for comparator institutions, with particular attention to sub-populations of the student body. The program demonstrates that it has a process in place to monitor and promote student progress.
       
5. Each program review plan must adhere to these Guidelines. In addition, institutional program review plans must meet the requirements of the Commission on Colleges of the Southern Association of Colleges and Schools Criteria, Section 3.1 (in part):

   The institution must develop guidelines and procedures to evaluate educational effectiveness, including the quality of student learning and of research and service. This evaluation must encompass educational goals at all academic levels and research and service functions of the institution. The evaluation of academic programs should involve gathering and analyzing both quantitative and qualitative data that demonstrate student achievement. Measures to evaluate academic programs and general education may include the following: evaluation of instructional delivery; adequacy of facilities and equipment; standardized tests; analysis of theses, portfolios, and recitals; completion rates; results of admissions tests for students applying to graduate or licensing examinations; evaluations by employers; follow-up studies of alumni; and performance of student transfers at receiving institutions. The institution must evaluate its success with respect to student achievement in relation to purpose, including as appropriate, consideration of course completion, state licensing examinations, and job placement rates.

6. Each institution will submit an annual report on program reviews as outlined in Section "E" of the program review policy. (Board Policy 205.01 E)

University System Office Process

The University System Office will review and approve the program review schedule for the current year and the report on programs reviewed the previous year. The University System Office will report to the Board of Regents as appropriate.

Triggered Program Review Process

System Office Process

1. The System Office will monitor annually a small number of indicators or "triggers" listed below in order to identify programs which may require additional study. The monitoring process is intended to begin a conversation between the University System Office and the campus about the health of the program(s) in question. Data already reported by the institutions to the System will be used to monitor the indicators². Indicators will be reviewed periodically and revised as necessary. While institutions are responsible for cyclical review of doctoral programs, there are no triggers for doctoral program review at this time.
   
   
   1. Graduate Programs
      
      1. Graduate enrollment in the major (average over the past three years) is less than 10.
         
      2. Graduate degrees awarded in the major (average over the past three years) is less than five.
   2. Bachelor's Programs
      
      1. Pass rates on licensure examinations is more than 10 percent below the state or regional average, if applicable.
      2. Enrollment in the major (average over the past three years) is less than 15.
      3. Degrees awarded in the major (average over the past three years) is less than 10.
   3. Associate Transfer Programs
      
      1. Less than half of associate-seeking students are still enrolled, have transferred to another System institution, have transferred to a career program, or have earned an associate degree within four years of matriculation.
      2. Transfers have a 2.0 grade point average or above at the receiving institution one year after transfer.
   4. Career Programs: Certificates and Associates³
      
      1. Enrollment in the program (average over the past three years) is less than 10.
      2. Degrees or certificates awarded (average over the past three years) is fewer than five.
      3. Pass rates on licensure examinations is more than 10 percent below the state or regional average, if applicable.
      
      
2. When performance falls below the indicators described above, the University System Office will begin a dialogue with the institution as outlined below:
    
   1. The University System Office will ask the Chief Academic Officer of the home institution about the state of the programs in question.
   2. Based on the institution's response, the University System Office and the Chief Academic Officer may agree to add that program to the current or next year's schedule of programs it will review, thus altering the institution's overall plan for the review cycle.
   3. If a full review of such a targeted program is conducted, a separate report on the results of that program review will be submitted to the Senior Vice Chancellor for Academic Affairs.
   
   
3. The annual review of key indicators by the University System Office may also reveal questions about program effectiveness in one or more discipline areas across institutions and programs. When general questions about programs in a particular discipline arise, the following will occur:
   
   
   1. A general review of the programs in that specific discipline at all USG institutions may be requested.
   2. A separate report from each institution on the results of the targeted program review may be requested by the Senior Vice Chancellor for Academic Affairs.
   
   
4. In the event that either Section "B" or "C" has been invoked, the institution's schedule of programs to be reviewed that year will include the programs specified by the University System Office in addition to those scheduled by the institution.

Institutional Process

When contacted about programs which fall below the indicators monitored by the University System Office, the institution will analyze the relevant data and provide reasons for the indicated results. Most questions will be resolved at this level; examples include programs which do not require course offerings beyond those already in place for high-demand programs with strong performance, unique programs, programs for which there is a statewide or regional need, and programs central to the mission of the institution. When the institution finds cause to examine the program further, the Chief Academic Officer and the University System Office will agree to add the program to the current or following year's cycle of program reviews.

Footnotes:

1. Or other national/regional organization standards/guidelines as appropriate if specialized accreditation is not available or appropriate for the program.
2. In addition to the indicators listed in this section, loss of special purpose accreditation or placement on probationary status will trigger an immediate review.
3. These triggers apply to certificates of at least one year and career associate programs. They do not apply to programs offered in cooperation with DTAE colleges or to DTAE programs at Bainbridge College, Clayton College & State University, Coastal Georgia Community College, or Dalton State College.

Last Updated: 10/30/2000