[an error occurred while processing this directive]

Example Results and Analysis of Outcome Measures

Note: All outcomes, instruments, criteria, and data are fictional and any resemblance to any department’s actual assessment plan is unintentional.

Student Learning Outcomes

Outcome: Foreign language majors will increase their levels of speaking and listening proficiency in a foreign language.

Measure (direct)

Students take the Foreign Language Skills Test (FLST) in their chosen language on two occasions: upon declaring the major and during the last semester of their senior year. This oral exam is administered by a faculty member using the same interview protocol with each student. Data are collected year-round and analyzed every other spring.

The exam session is tape-recorded so that the FLST may be scored at a later time using a rubric based on the ACTFL proficiency guidelines. Two faculty members score each exam, and a third faculty member is consulted if scores in any category differ by more than one point. Students receive two scores (speaking and listening), calculated as the average across the two faculty members.

Criteria

Results and Analysis

Twelve graduating seniors took the FLST during the past two years: nine in Spanish and three in French. Scores showed a significant increase from the pre-test occasion, and all but one student performed at least at the Advanced level. Detailed results are presented below.

 Pre-test
NoviceIntermediateAdvancedSuperiorAverage ScoreSD
Speaking36302.00.74
Listening45301.92.79
 Post-test
NoviceIntermediateAdvancedSuperiorAverage ScoreSD
Speaking01833.17.58
Listening01743.25.62

Speaking scores increased by an average of 1.17 points, which translates to a very large effect size of d = 2.7. Similarly, listening scores increased by an average of 1.33 points, which translates to a very large effect size of d = 3.2. We are satisfied with this level of increase in proficiency.

A detailed analysis of the exam scores for the student who did not reach the Advanced proficiency level revealed areas of weakness in grammar, pronunciation, and comprehension. Since this student’s result does not seem to indicate a systematic or prevalent issue with our program, no curricular or pedagogical changes are under consideration at this time. We will, however, continue to monitor student performance in these areas.

Outcome: Upon completion of the general education program, students will demonstrate effective oral and written communication skills.

Measures (direct)

Students present a persuasive speech as the final assignment in COMM 250. The assignment and scoring rubric are standardized for all sections of the course, and the course is required for all students. Speaking times are scheduled so that two faculty members may be present to score each speech, and a third faculty member is consulted if scores in any rubric category differ by more than one point. Data are collected in fall and spring semesters and are analyzed in the spring.

Criteria

Students complete three writing assignments in CWRIT 300 – a business letter, a letter to the editor of the student newspaper, and a research paper. The assignments and scoring rubrics are standardized across all sections of the course, and the course is required for all students. Grading is “blind” in that names are concealed and faculty members do not grade the work of their own students. Data are collected in fall and spring semesters and are analyzed in the spring.

Criteria

Results and Analysis

This report presents results for 186 students completing COMM 250 and CWRIT 300 in academic year 2006-07. The charts show the distribution of scores and the table presents average scores by entering cohort year.

Histograms showing distributions of scores
 Entering Cohort Year
2003-042004-052005-062006-07Transfer-inOverall
NAvg.NAvg.NAvg.NAvg.NAvg.ScoreSD
Speech1120.43122.911424.5319.82724.123.63.1
Bus. Ltr.512.610813.73214.2712.93414.113.83.6
Ltr. to Ed.521.010820.63222.3718.63420.620.84.7
Paper526.210831.43230.5724.63428.430.37.8

Overall average scores were above the minimum requirements for all three assignments. For three of the four assignments (speech, business letter, and letter to the editor), approximately 90% of students scored at or above the minimum required score. One-third of students, however, scored below the minimum score on the research paper. Students who took CWRIT 300 in their first or fourth year and transfer-in students scored below average. Several interventions have been discussed: requiring a prerequisite for CWRIT 300, advising students against taking the course prior to their third year, and instituting a CWRIT 300 “readiness test” with corresponding remediation where needed. This is the first time we have looked at the data in this way, and there are small sample sizes for most of the cohorts. Therefore, we will collect more data to inform any final decisions about intervention.

These assignments have been used in these courses for the past four years. Trend data are presented in the chart below, where the solid lines show changes in average scores over time for all four assignments and the dashed line represents the minimum required score for the research paper.

Line chart of the trends on the four scores

The below-average scores on the research paper during the first two years were attributed to inadequate information literacy skills. Students relied heavily on Google and popular, rather than scholarly, writings. In cooperation with the library, class sessions were designed so that students would have the opportunity to improve these skills, and these sessions were implemented in academic year 2005-06. There was a corresponding increase in overall scores on the assignment. Since a third of our students are still scoring below the minimum, however, further analysis and intervention is needed.

Operational Outcomes

Outcome: Foreign language majors and foreign language faculty will interact in ways that promote learning and personal growth.

Measure (indirect)

Students take the Foreign Language Skills Test (FLST) in their chosen language on two occasions: upon declaring the major and during the last semester of their senior year. This oral exam is administered by a faculty member using the same interview protocol with each student. Data are collected year-round and analyzed every other spring.

During the second testing occasion, three interview questions addressing student-faculty interactions are included.

Criteria

Results and Analysis

Twelve graduating seniors took the FLST during the past two years: nine in Spanish and three in French. The majority reported that they frequently interacted with faculty outside of class time. Of those who reported that such interactions did not take place, all but one had a job or family situation that prevented them from spending extra time on campus. Overwhelmingly, students described our faculty as engaged, available, and supportive.

The following quotations (translated into English) are typical of the impact student-faculty interactions had on the learning and personal growth of these students:

“Knowing that I’m welcome in my professor’s office makes me feel less stressed out and more confident about going for help.”
“My professor helped me find information and people to contact about my future career.”
“Once, when I made a bad grade on a test, my professor met with me and helped me understand what I did wrong.”
“I was, like, really confused about whether to go to graduate school or to get a job right after graduation. A couple of my professors really helped me, like, sort that out and think about the pros and cons of whatever decision I make.”
“I published an article about Guatemalan pottery with one of my professors. That was an awesome experience!”