Introduction and Rationale
For the past several years, the administrative and support units of Georgia State University have been subject to two related but distinct assessment processes. The first, the Administrative and Support Unit Review (ASUR) process, was designed as a comprehensive long-term planning process that required the department under review to gather data on a number of operational performance questions, write a self-study for University review, and then submit an action plan to the Provost. The purpose of the process was to encourage a culture of continuous improvement within the administrative units of the University. Shortly after we began development of the program, SACS added to their accreditation standards a requirement for administrative units to engage in regular planning and assessment. ASUR was based on a 7-year cycle, and the University completed the first cycle in FY10.
The second process used to assess administrative departments, the monitoring of Institutional Effectiveness Outcomes, was developed in 2004 in response to additional SACS and Board of Regents requirements. The explicit focus of this requirement is the continuous use of data in the development of improvement strategies. In this annual process, all administrative departments identify the intended outcomes of their activity, and measure their progress at meeting those outcomes. Results from each fiscal year are entered into an electronic tracking system which can generate reports for senior management to use for planning and decision-making. The outcomes tracking process was designed to complement the ASUR process in that the performance data generated from the system could inform the department’s ASUR self-study.
As the completion of the first ASUR cycle approached, the Associate Provost for Institutional Effectiveness solicited feedback from the University community on both processes. In response to the comments and suggestions provided by the leadership of the University, the Administrative and Support Unit Review Committee, the departments that were reviewed in the first cycle, and the on-site SACS accreditation team, the Office of the Associate Provost for Institutional Effectiveness spearheaded an effort to revise and improve the processes the University uses to assess its administrative units.
The goals of the project were to make the assessment of administrative departments more meaningful and less time consuming for the departments while continuing to meet accreditation standards. The new Institutional Effectiveness Review (IER) process:
fully integrates the two assessment processes into a single smooth process;
analyzes performance at the Division of subdivision level, rather than at the departmental level;
provides an Executive Summary that goes forward to the community;
focuses on those elements critical to the impact of a unit; Mission, Functional Responsibilities, Outcomes, and Strategies for Improvement
shortens the overall review cycle; and
increases the amount of targeted facilitation and support available to the units under review.
Explanation of Modifications
Integration. Most departments considered both of the original University assessment processes to be outside their normal realm of work responsibilities. Refocusing the ASUR self-study for the divisions/subdivisions on the elements that their departments report on annually will decrease the onerousness of assessment and increase its usefulness. Since the monitoring of outcomes occurs every year, completing a more focused comprehensive review at 6 years might not seem so alien in terms of workflow.
Unit of Analysis. Because the previous ASUR process focused on individual (budgetary) units as the units of analysis, ASUR assessment was often fragmented—the vice presidents, associate provosts, and assistant vice presidents received detailed information on only one of their departments at a time, and that information was seldom synthesized at the divisional or subdivisional level. This fragmentation was even more apparent in the process to monitor outcomes (where we have approximately 100 reporting units compared to 70 for the ASUR process). Although the annual data were reported for all departments at the same time (at the close of the fiscal year), those data were seldom synthesized at the divisional or subdivisional level.
Defining the unit of analysis for ASU Review as the division or subdivision (where several departments are organized under an assistant vice president) allows for an integrated analysis at that level. In addition, intended outcomes (impacts) can be defined at the division or subdivision level as well as at the departmental level. If divisions/subdivisions elect to use this feature, they can ensure explicit alignment of outcomes with the University Strategic Plan and institutional priorities.
The new units of analysis will be more comparable in size, complexity, and significance to the University than are individual departments. In the first ASUR cycle, department size varied from one staff member to 100+, and departmental budgets (personnel excluded) ranged from a few hundred dollars to $1,000,000+. Each department had to answer the same review questions and provide the same data (climate survey data, peer institutional data, etc.). And each of those 70 departments accounted for the same amount of time and effort from the Associate Provost for Institutional Effectiveness, the ASUR Committee, and the Planning and Development Committee. The new process will result in more effective prioritization of these resources.
Administrative Functions within the Colleges and Divisions. In the original ASUR process, administrative functions in the colleges were, at times, included in the review of their respective centralized counterparts (for example, the college advisement offices were assessed with the Student Advisement Center). There were numerous issues related to this cumbersome process. The most problematic was that the fragmented process (with various administrative functions within each college being assessed in different years) made it difficult for a college to make systematic improvements based on the findings of the review, and nearly impossible for anyone to modify a function’s performance across colleges.
To make the process of assessment and improvement more feasible for these functions, all administrative functions within a college (or division) will be assessed in the same manner as other administrative departments. The colleges’ administrative functions will monitor their outcomes annually and undergo an ASU Review of their self-study every six years. Each college will determine the unit of analysis that best fits their operations—either one “unit” that includes all of their administrative functions, or four separate units (human resources and business; research and technology; student services; and public relations and development).
College administrative functions have not yet been required to identify and monitor their outcomes. This process will require that these functions, like centralized administrative departments and academic programs, develop intended outcomes and monitor and report their performance.
Executive Summary. With the previous assessment processes, there was the perception among departments that senior management (vice presidents, associate provosts, and assistant vice presidents) did not attend to these reports. Since the unit of analysis will now be at the division or subdivision level, it is natural that the vice president, associate provost, or assistant vice president prepare and submit either a single comprehensive report for their division or subdivision or an executive summary for the ASU Review (see below). The executive summary will make the reporting more meaningful for the university community as well (Administrative Council and the Senate Committee on Planning and Development).
Focus on Impact. To ease in the preparation of the assessment reports, the scope of the ASUR self-study report will be narrowed. The primary focus will be on the impact the division/subdivision has on the institution and its stakeholders and customers. Detailed discussion about services, organizational structure and resources will not be part of the formal report. It is assumed that if a division/subdivision is not meeting its targets with regard to its intended outcomes, the unit will then begin to look to those details in their problem analysis.
This changes the focus of long-term assessment from what activities a department engages in (and wants to engage in) to what impact the division/subdivision has, what impact it wants to have, what strategies are needed to get there, and how the individual departments within the division/subdivision will work together to implement those strategies.
The change in focus allows units to better track the effect of their interventions. (So, for example, instead of measuring the success of implementation of a new retention program by whether or not it was eventually implemented, the unit can measure and report the actual impact of that program on the retention rate of the students who engage in the program.)
Review Cycle. Senior leadership and departmental management agreed that seven years is too long for effective planning at the unit level, given the rapid changes in the environment (both inside and outside the University). Many departments undergoing review have indicated that after three years, the action plans developed from the ASUR process are useless.
The new cycle schedule and process allow the units to formally articulate goals annually, provide for more flexibility in operations, encourage mid-stream corrections where necessary, and maintain accountability to the institution and its constituents. As is currently practiced, units will continuously monitor their outcomes and report on them annually through WEAVEonline. In addition, more focus will be placed on the annual goals and the development and monitoring of the annual action plans to aid the divisions in their planning and budgeting processes.
Three years prior to a division’s/subdivision’s ASUR review, a 3-year summary report will be developed by the Office of Institutional Effectiveness and reviewed by the ASUR Committee to ensure that the units will be well-positioned for their comprehensive review.
The division/subdivision will submit a comprehensive self-study for review every six years. The division/subdivision comprehensive report will also include an organizational climate survey and a peer comparison, and a summative review by a University committee with input from the University Senate.
Facilitation. Since our ASU Review focuses on outcomes at the division/subdivision level, additional work will need to be done initially to integrate the assessment plans of the divisions/subdivisions with their constituent departments. The individual departments will have to be able to identify how their activities affect the outcomes of the division/subdivision. Furthermore, it will be imperative that the outcomes that the departments within the divisions and subdivisions articulate are comprehensive, and that each has effective measures and data collection in place to evaluate those outcomes.
In the transition years, facilitation will be re-incorporated to ensure that the units can build the assessment infrastructure, and have continuing guidance along the way. Annual outcomes will be reviewed and units inadequately defining or monitoring outcomes will receive targeted intervention.