[an error occurred while processing this directive]
The vice-presidents will review the departmental reports to create a divisional summary. In addition to looking for common themes and items that require divisional or university-wide support, the VPs will provide feedback to each department using this rubric.
If you have additional questions or have problems with the AIER system, please contact BJ Miller.
You may want to review the Technology Introduction page before you start working with the AIER system.
Fields at the top of the start tab allow you to change who is authorized to log in and edit the report, but not all of those people may be active in writing the report in any given year. The Report Author(s) field is used to indicate who actually worked on the report this year.
The mission or purpose statement answers the question “What does my department do?” It should be broad enough to cover all of the important aspects of the department’s mission, yet focused enough to clearly be identified with the department.
The distinctives section of the AIER should clarify how the program connects and contributes to EMU's vision and mission. It may help to ask, “How is this program different because it's at EMU?” Several areas you may want to consider include:
Functions at the bottom of the start tab allow you to attach a file and to publish the report when it is complete. The preferred format for the attachment is PDF. If you have multiple files, you can save each of them to PDF format and then merge the PDF files. Contact the HelpDesk or Institutional Research staff if you have questions about the process.
This section provides departments an opportunity to summarize the significant achievements of the department, external factors affecting the department, changes in staffing or resource needs, significant changes in demand or focus, and their impact on the overall productivity of the department. You may want to come back to this section once you have completed the rest of the report.
The plan for improvement you submitted last year will appear in this section for you. Please comment on your efforts at improvement over the previous academic year. Be specific about what strategies you implemented, how you measured your success, and whether you met the targets you set for improvement.
The two outcomes sections of the report have the same basic structure:
The outcome sections of the AIER include all intended outcomes important to the department mission. Learning outcomes refer to the specific knowledge, skills or developmental attributes that students develop through their college experience. Some administrative departments will not have learning outcomes. Operational outcomes define the desired impact of the department’s services on various internal and external audiences. All departments will have operational outcomes and we have a few suggestions for academic programs.
Examples of Outcome Statements
Customer satisfaction (students, alumni, employees, donors, etc.) provides an important perspective on program quality. In previous years, the satisfaction data collected through national and campus-wide surveys was included in the Quality section of the report. However, recognizing that departments should state and prioritize their own operational outcomes, the survey data section of the AIER has been removed. Please continue to report on the operational outcomes that are important to your department. Survey data that may serve as relevant measures for your outcomes are available on the IR website.
View the guide to survey data for help in interpreting the data. Contact the institutional research staff if you require further analysis.
A set of measures for each outcome should collectively cover all important aspects of the outcome from different points of view. Include both direct and indirect measures when possible. Measures may change more frequently than outcomes, but should not change each assessment cycle. The AIER process should include an evaluation of the quality and effectiveness of assessment instruments.
For each measure:
Each measure should include a benchmark or other criteria used to interpret the results of that measure. A benchmark is:
It may be helpful to think of benchmarking as similar to stating a hypothesis. A benchmark may be based on past performance, some agreed-upon standard of performance, or the performance level of some other group. Most importantly, the benchmark should not be set or changed arbitrarily.
Finding appropriate comparison groups as an aid in setting benchmarks is one of the most difficult challenges of assessment. The following resources may be of help:
How frequently should you collect and analyze data for a particular measure? It depends on the nature of the outcome, the type of measure, and many other factors. Here are a few points to consider:
Examples of Measures and Criteria or Benchmarks
For each measure, you will collect data and evaluate that data to determine if you are satisfying the corresponding outcome.
As noted above, it is perfectly reasonable to collect data on one schedule (e.g. yearly) and perform a complete analysis of the results less frequently (e.g. every 3 years).
Extended examples of the Outcome-Measure-Results-Analysis process.
The Resources section of the AIER gives you an opportunity to discuss the adequacy of the resources you have available to accomplish the work of your department. If the work of your department has changed or expanded significantly, discuss those changes and any adjustments you have made. If additional resources are required, explain what these resources might be, and how you would use them to better accomplish the mission of your department.
Specific sections within the Resources area include: faculty, staff, facilities, equipment, and library. There is also a space labeled "Other" in which to address resources not covered in the previous sections.
The Demand section asks you to examine the demand for your department’s services. The specific fields vary by type of program.
You’ll want to look at demand generated by both internal clients (students, faculty, and other staff, trustees), as well as by external clients (governing bodies, alumni, community groups and other clients). Discuss any changes in demand and your prediction for growth in demand for specific types of services.
The demand for academic programs is measured by the number of students enrolled in the program. You can copy in last year’s analysis of enrollment trends and update it based on new enrollment values and then note any new or changed factors affecting enrollment. You should also determine if the ideal program enrollment values are still set appropriately.
Note that undergraduate departments offer courses specifically for students majoring or minoring in their programs, but also offer courses in support of our general education curriculum and programs in other departments. Space is provided to comment on both types of enrollment.
In the cost section we are looking at the costs associated with the services provided by the department.
The principal driver of costs for academic departments is faculty salaries. The student-to-faculty ratio provides a method to quantify the amount of faculty work it takes to provide programs on a per student basis.
The “Summary data” link provides a table with the student FTE, faculty FTE, and the student-to-faculty ratios. The “Course Enrollment Detail” link includes data on enrollment in each course offered by your department over the last year, and “Faculty Load Detail” is a summary of the faculty loading for your department. See the footnotes for each table for an explanation of how faculty FTE and student FTE are calculated.
The student FTE and the total program cost are used to calculate the cost per full-time equivalent student. The data for this cost measure are found on the Financial Summary-click the link to the Summary data beside the Program costs.
The Financial Summary is divided into four tables. The top table, Student and Faculty Data, includes the student credit hours and student FTE data. Please note that the data do include summer courses. The other three tables of the Financial Summary show cost, revenue and (income) loss data for each of the last three years. Please note that numbers shown in parentheses are positive values.
The rows of the tables are identified with line numbers shown in the right-hand column; if you scroll down below the tables, you will find footnotes explaining how the amounts for each line are calculated. The second table, Summary Data for AIER Reports, includes the cost per student FTE : Line 1 is the revenue per student. Line 2 is the cost per student, the second measure in the AIER cost section. Line 3 provides you the (income) or loss per student FTE.
The third cost measure subtracts the direct expenses of the program from the program income, and calculates the dollars available to cover overhead expenses. This remainder, expressed as a percent of total program income, is referred to as the percent contribution to overhead (line 4). The “Summary Data for AIER Reports” is based on data from the operating budget, and does not include any revenue or cost from your restricted accounts.
In this section, you are asked to discuss the factors that contributed to your program’s success or failure to meet your contribution to overhead target. The trend data should be useful in helping you to see what is happening to income and expenses over time.
The Operating Account Costs and Revenue Data table may help you understand the trends you see in the Summary Data for AIER Reports table. The Campus Web Financial Data (lines 5-8), summarizes the activity in your program’s operating accounts and should match up with the total revenue and cost for your accounts on Campus Web. You can look up the detail for 2005-06 for each of your accounts on the Campus Web.
The Overall Program Financial Data section is designed primarily to incorporate allocated tuition net revenue for the undergraduate departments. The section proceeds through a series of steps to calculate an overall program (income) loss. Note line 13 does give you a total for program revenue.
Data on restricted accounts is in the bottom table and includes contributions, endowment income, grant income and costs for some program activities. Because programs use restricted accounts in different ways, you will need to include a description of the activity in the restricted accounts and their impact on the finances of your program.
In the cost effectiveness section we are looking at the amount of work produced by the department, and the costs associated with the resources required to accomplish that work. The principal driver of direct costs is salaries, so productivity is often expressed in units per staff FTE and will vary by department. Measures of costs are often expressed in costs per unit of service. Examples for both types of measures are given in the pink explanation box. If you have difficulty determining the staff FTE for your department, contact the Human Resources Office.
General questions about how to measure productivity or costs can be discussed with your supervisor. Professional associations can also be good sources of information on standard ways of measuring cost effectiveness and productivity. Longitudinal data as well as peer comparisons from survey data can be very useful in putting your department’s productivity and cost effectiveness numbers into perspective. Defining who your peers are can be a fairly complex process. The IR Office has done some work on defining our peers that might be helpful to you, so let us know if you have questions. Click on the Financial Summary Report link for assistance in evaluating the trend in departmental costs.
The Financial Summary includes two tables: one summarizing activity in accounts within the operating budget, and the other summarizing revenue and cost in the restricted accounts. Three years of data are included in the summary. Each row in the tables is identified with a line number shown in the right-hand column; if you scroll down below the tables, you will find footnotes explaining how the amounts for each line are calculated. The Campus Web Financial Data section summarizes the activity in your department’s operating accounts and should match up with the total revenue and cost for your accounts on Campus Web. You can look up the detail for 2005-06 for each of your accounts on the Campus Web. The department (income) loss figure is the bottom line, taking into account all revenues and costs. If your bottom line is a positive number, it will be in parentheses. In your report you will want to note changes over the three years in lines 5-8 and, where possible, identify the reasons for those changes.
The Overall Program Financial Data section proceeds through a series of steps to calculate an overall program (income) loss. The main purpose of this section is to bring the allocated tuition net revenue into the calculations for the undergraduate departments. It gives you a total department revenue line that adds any tuition or fees your department receives to the other department revenue, and program cost is added to give the overall program (income) loss (line 15).
Data on restricted accounts is in the bottom table and includes contributions, endowment and grant income and costs for some program activities. Because the programs use restricted accounts in different ways, you will need to include an analysis of the effect of activity in the restricted accounts and its impact on the finances of your department. More information on the revenue and cost included in each of your restricted accounts is available in the Financial Detail Report. The link to this report is in the pink explanation box on the Cost Effectiveness page in the AIER form.
The focus of the improvement plan is on using the results of your assessment activities and analysis of other data to improve your programs and the services you provide.
Improvement Plan action items should:
Multi-year action items can result when there is a need for further information and analysis before deciding on a specific course of action. For any long-term activity, ensure that a series of targets are established so that you can be certain progress is being made each year.
You may have action items that are not the direct result of outcome measures. When this happens please provide a brief description of how you identified the need for the action item.
Next year you will be asked to review progress in implementing this plan in the “Progress from Previous Year” section of the AIER.
If you have additional questions or have problems with the AIER system, please contact BJ Miller.