Tracy Lauder
Can you assess the accumulated knowledge of a broad-based mass communications curriculum in the senior year? We have been implementing, with varied success, a senior comprehensive exam as part of our annual institutional assessment report. Going into our fourth year, we are still fine-tuning the process and sometimes question our results. Using such a tool has brought up a range of departmental issues and questions.
Process
Our comprehensive exam contains 100 multiple-choice questions, approximately 14 questions per each of the seven mandatory core courses in the major. These courses include skills courses (Beginning Design; Beginning Electronic Media; and Writing, Reporting and Editing I and II) and theoretical courses (Mass Media and Society, Persuasive Communication, and Media Law and Ethics). Instructors submit and agree on questions for the exam and then the exam is administered in the Media Law and Ethics course, which also serves as our capstone senior course in the final semester. (In this course, final senior Web portfolios are submitted and factored into each student’s course grade and are also used in departmental assessment.) As an incentive for students to review for the exam, the score is counted somewhere in the course grade, even if the entire exam is not directly tied to the course content.
Results
Our stated goal was to have 70 percent of students make a 75 percent or above on the comprehensive exam. Results were exceedingly bad the first two years. In 2009, only 16 percent of the students scored 75 percent or better; however, 42 percent of the class made at least a 70 percent on the exam. In 2010, we lowered the minimum score to 70 percent, yet only one student scored above a 70 on the exam; 43 percent scored between 50 and 60 percent, while 43 percent scored between 60 and 70 percent. We suspected that the low scores were due to
- issues with the exam itself, and
- students not taking the exam seriously.
The third year of implementation we met our goal: 73 percent of the graduating class scored 70 percent or higher on the exam. We attributed this success to
- tweaking the exam (rewriting questions, in some cases), and
- providing a grade incentive by factoring the comprehensive exam grade into the final course grade so that the students would be motivated to review for the test.
Challenges
Our greatest obstacle was selecting 14 key questions per course, particularly when different instructors may teach different sections of the course. For example, due to the vast amounts of content in the survey course, each instructor tends to emphasize different things, and, naturally, teaches to his or her strengths and interests. In fact, at our school, faculty often use different textbooks for the intro course. While one instructor tends to focus on First Amendment issues and media history, another instructor focuses on media literacy, while another emphasizes a critical/cultural studies approach, with more focus on international communication.
Depending on which section of the intro course a student took, the questions might or might not have been relevant. (Similar challenges resulted when adjuncts filled in during sabbatical leaves.) Although the department faculty strived to pull questions that they all believed were important from the introductory course, it sometimes was difficult to reach a consensus, and some felt academic freedom sometimes was at stake.
Another key challenge resulted from a curriculum change when a core course changed from a history/theory-based course to a skills-based course. What questions should we have included on the comprehensive exam, given that in any cohort, students might have taken different versions of the class? We ended up “throwing out” questions on the exams of students who had taken alternative versions. This kind of scrutiny to courses might take several cohorts to work its way out.
Additionally, as the focus of our curriculum and individual courses shifts in a rapidly changing discipline, instructors fine-tune courses, shift content to make courses timely and relevant, and, perhaps, shift key questions and central themes, too. This makes consistent testing in the comprehensive exam very challenging.
Finally, not all accumulated knowledge can be assessed with multiple-choice testing, and not all students showcase what they know in objective test formats. Standardized tests are not always the best measure of knowledge, which is why some programs in other disciplines use a combination of objective, essay, and oral exams. We have not ruled out these approaches for the future.
Benefits
The process of implementing a comprehensive senior exam accomplished several good things. First, with the pressure of providing a direct measure of assessment for our institutional report mandated by the our college’s Office of Institutional Research and reported to SACS (Southern Association of Colleges and Schools) for accreditation purposes, an objective exam proved to be a viable option. Although it has taken time to build and revise the exam, it takes minimal time to administer and grade, compared to other assessment measures, such as the portfolio review (which we also conduct), oral exams, and essays.
Second, it forced our department to review, as a faculty, the course content in all core courses and reflect on the key concepts we hoped every senior should be taking away from the program. We were able to discover areas of overlap from course to course and hear how each instructor was tackling important skills and concepts.
Finally, the senior comprehensive exam also forced the students to spend some time reviewing. (This year, we plan to provide a simple study guide to aid students in preparing for the exam.) According to our department chair, Teresa Keller: “Whether or not our comprehensive exam is a perfect reflection of what our students know, it does force them to sit down, even for one night, and look at all they’ve learned. They revisit every core course, and they recognize the knowledge base that they have built is great.”
Although our department may have resolved some logistical issues in using the exam as an assessment tool, the revision of the comprehensive exam will remain necessary nearly every year due to the changing nature of the mass media industry and the resulting course revisions and new course development.
Tracy Lauder is an associate professor at Emory & Henry College.
No comments yet.