Assessment of Collaboration Outcomes

English 1102 Summer, 2004 Pilot

Mary Hocks and Marti Singer, Department of English

During the summer semester of 2004, four instructors teaching English 1102 agreed to collect data regarding two types of collaboration conducted during their freshman composition courses. One type of collaboration common to all sections of ENG 1102 includes peer review of student written essays. The other type involved a group project whereby students collaborated on a single written piece. Given the short summer semester, we only directly assessed the group projects. Instructors used their own syllabus to include the collaborative aspects for this pilot. While each of the projects is different in the four sections of English 1102, they all address the same outcomes for collaboration and writing. Included in this report are sample assignments and syllabi, assessment rubrics, and an summary of student responses about their group collaboration projects and activities. We conclude with our plans for the 04-05 academic year.

1. Sample Assignments: The assignments included in this pilot focus on a theme or a genre of writing that requires students to produce a product created collaboratively that also demonstrates skills outlined in the learning outcomes designed for English 1102. Each project required collaborative group work throughout the semester that culminated in a written text and an oral presentation to the class. Many oral presentations included a visual component. For these specific sections, themes ranged from the study of the argument about zoos, to the use of colloquial language, to a “multigenre essay” using various print and electronic media. One section used a group community service learning project (representative syllabi and assignments are attached).

2. Assessment Procedures: While the assessments varied among the sections, a basic survey questionnaire was given to most of the students. The questionnaire was given to three sections of ENGL1102 where a total of 57 students answered at least six of the 10 questions. A copy of the questionnaire is attached. Another 14 students in a fourth section responded with a frank analysis of their experience with collaboration in general in this project. We analyzed data from a grand total of 71 students.

3. Preliminary Results: The results of the questionnaire indicate that over 80% of the students who participated in this pilot had a good grasp of the meaning of group collaboration and its distinction from co-authorship of one document; the same percent of students felt that they participated fully in the collaborative project. Three-Fourths of the students (54 students) from all four sections preferred smaller groups of two or three members. Two sections required four to five members in each group and some students (18 students) explicitly reported this larger group size as ideal for sharing work in a project; however, these larger groups also experienced some typical problems. These problems included personality clashes, slackers, communication problems, attendance problems, and not enough time or feedback due largely to the summer seven-week session.

All students who commented on their learning indicated that they learned more as a result of collaborative work—both about multiple perspectives on the topic and about how to work together on writing as a group. In other words, some groups felt they succeeded in the production of the written project and presentation, but felt like they failed in the collaboration of their individual efforts. Others felt just the opposite—that they collaborated well but the finished product was not adequate. The results of question 10 about visual presentation were very non-uniform and even contradictory. While some students valued the opportunity to learn PowerPoint, for example, others needed to know how to integrate the visual communication pieces with the paper.

In the holistic written assessments, nine out of 14 responded positively with descriptive comments such as, “It helped me to get to know my peers” and negative comments closely resembled the problems culled from the survey responses. For example, a student wrote, “collaborative learning is never a good experience. Meeting up with other group members outside of class time is a hassle.”

4. Plans for 2004-05: During this academic year, the English Department has recruited 7 instructors to focus on assessing collaborative outcomes in their English 1102 classes. These sections will work with both collaborative learning through peer review and collaboration in the form of group discussions about articles. Some sections may also include assignments where pairs of students co-author a paper or presentation. Given the logistical problems of group projects reported by students in the summer pilot, we have suggested that groups consist of two students who co-author a paper rather than the larger teams. In addition, this year we have broadened the scope of what we identify as collaboration to include group discussions of texts in class. These groups will consist of 3 to 5 members who collaborate on responses to texts. Based on a revised version of the assessment questionnaire developed in the pilot, all classes will use the same end-of-course survey for the collaboration.

While our pilot assessment analyzes the collaborative projects, it does not report on peer review. For this year, we will assess specifically peer review and consider questions such as: How does it help students? How much? We’ve attached a new rubric for responding to articles and student writing that can be used to orchestrate group discussions and peer reviews. This group of instructors will develop assessment procedures that use the attached rubric to serve two purposes: provide feedback to peers about their writing and provide feedback that will allow for revision of the assessment tool. In addition, these instructors will ask students to complete a pre-survey during the first few weeks of class that assesses their previous experience with and opinions about collaboration. This benchmark will make the results of the final questionnaire more specific and help us in revising the course.