Unit Name:  COMMUNICATION

Prepared By:  DAVID CHESHIER (Assoc Chair), 19 August 2005

Academic Year:  2004-2005

 

The following summarizes faculty-directed assessment activity relating to five degree programs administered by the department of communication (undergraduate majors in speech, journalism, and film, an MA program that includes tracks in film/video production, human communication & social influence, and mass communication, and a two-track PhD program with emphases in Public Communication and Moving Image Studies). 

 

The 2004-2005 academic year was the first in which full-scale data collection was undertaken after several semesters of area faculty planning and pilot data projects.  Assessment of the undergraduate majors was organized mainly through the department's Undergraduate Studies Committee, now chaired by Dr. Doug Barthlow.  That committee's membership is organized so that all departmental degree programs are represented.  Accordingly, Dr. Greg Lisby and Barthlow oversaw area faculty work relating to journalism, Dr. Ly Bolia organized work connected to film, and Dr. Jaye Atkinson supervised assessment for the speech major.  At the graduate level, assessment was organized by the department's two graduate directors, Dr. Greg Smith and Dr. Michael Bruner, working with their colleagues on the department's Graduate Studies Committee (Smith, Bruner, James Darsey, Kathy Fuller-Seeley, and Ted Friedman).  Both committees have planned area faculty meetings for the FA05 semester to review 04-05 activity and recommend improvements for the 05-06 academic year.

 

 I. Assessment Procedures

III. Changes to Procedures or Curriculum Based on Assessment

 II. Achievement of Departmental Objectives for Students    

A. Undergraduate-- 
     B. Graduate

IV. Changes in Department’s Assessment Goals

 


I.  Assessment Procedures

The department of communication organized significant assessment activity in the 2004-2005 academic year.  Learning outcomes were identified in all departmental majors in SP04, pilot data collected in SU05, and data fully collected and evaluated in both FA04 and SP05. Those results are reported separately in what follows, by major program (film, journalism, speech, graduate).  The theater major is being discontinued and a proposal to reconstitute it as a BIS concentration is under revision; for that reason data is not reported for theater. 

In each of the following sections, the process generated recommendations for continuing improvement which are being reviewed by area faculty to strengthen content instruction in communication consistent with the university's overall specification of learning outcomes.  Similar outcomes are reported below for the department's graduate programs.  The process was overseen by members of the department's undergraduate committee (Miller, Lisby, Bolia, Atkinson, Barthlow) and by the department's graduate directors (Smith and Bruner).  Assessment instruments were developed in coordination with the Provost's office and the university Center for Teaching and Learning, with a specific view to developing results relevant to disciplinary norms and national accreditation standards.

 

FILM major

FILM area faculty identified three significant learning outcomes.  Those goals are:

 

 

 

 

Assessment was administered in Film 1010, Film 2700, and Film 4750 in FA04 and SP05 to determine the extent to which students were able to demonstrate a fundamental understanding vs. in-depth knowledge of various principles of Cinema history, theory, and criticism.  The difference between scores in three courses was used to assess the extent to which students have gained knowledge of principles sought. These data are being used by Film faculty to guide possible revision of individual course content, overall revision of Film curriculum, and/or consideration of course prerequisites.

 

JOURNALISM major

Journalism faculty proceeded by matching instruction in its degree tracks (including broadcasting & public relations) with institutional general education goals.  While not all goals are specifically addressed in each of the major's classes, most are.  The journalism assessment strategy for each of the connected GenEd goals included the following:

 

 

 

In addition, major specific goals and learning outcomes were identified for the undergraduate Journalism major.  These goals will be specifically accessed in AY05-06.  These goals include:  (1) Students will use communication technology and they will be able to adapt to a changing technological environment; (2) Students will communicate effectively in written, oral, visual and/or digital forms; (3) Students will be critical and ethical communicators and consumers of communication, and (4) students will understand the development of communication industries in their political, legal, social and economic environments.  The plan is to gather scores on 10-question assessment developed by Journalism faculty.  Assessment instruments will be administered in Jour 1000 and again in Jour 3070 each term to determine extent to which students demonstrate a fundamental vs. in-depth knowledge of various principles of journalism/mass communication theory.  Measured differences between scores in two courses will be used to assess extent to which students have gained knowledge of principles sought. Data will be used by Journalism faculty to guide possible revision of individual course content, overall revision of journalism mass/communication curriculum, and/or consideration of prerequisites 

 

 

SPEECH major

Speech area faculty identified three major goals for students majoring in the program, to be acquired regardless of the specific major courses taken (which are organized into two tracks or major concentrations).  These include:

For each goal an assessment instrument was created and classes identified so that speech majors would be captured and data collected.  The results discussion which follows elaborates the details of these instruments more fully.

 

GRADUATE PROGRAMS in communication (M.A., Ph.D.)

Four professional development outcomes were identified as central to the M.A. curriculum, and five to the doctoral program in communication.  Faculty assessed attainment of these outcomes by reviewing submitted thesis and dissertation projects, conducting year-end faculty evaluations of all doctoral students, and review of completed comprehensive examinations.

 

The department's M.A. programs are organized to achieve the following competencies:  (1) Significant mastery of research methods relating to knowledge generation in the communication arts, with specific expertise generated in the methodological approach best suited to the research interests of a particular student; (2) An understanding of the key concepts of communication scholarship; (3) Written and oral communication competency in conveying difficult communication concepts clearly, and in such a way that knowledge generation is facilitated; (4) Detailed knowledge development of a specific area of communication inquiry, such that a student will be able to join high-level advanced conversations and debates relating to a particular area of social importance.

 

The department's doctoral programs seek to inculcate the following competencies:  (1) Significant mastery of research methods relating to knowledge generation in the communication arts, with specific expertise generated in the methodological approach best suited to the research interests of a particular student and at the most disciplinarily advanced level; (2) Pedagogical proficiency, which is to say the student should be able to organize disciplinary findings and areas of inquiry in such a way as can be understood by the field's larger publics (students, audiences, intelligent and interested observers of communication phenomena, etc.); (3) An understanding of the key concepts of communication scholarship, with particular attention to advanced theoretical work undertaken by scholars in the cognate areas connected to communication research; (3) Written and oral communication competency in conveying difficult communication concepts clearly, and in such a way that knowledge generation is facilitated such that students with the doctorate will be competent and comfortable reporting their research in the field's leading academic journals; (4) Detailed knowledge development of a specific area of communication inquiry, such that a student will be able to join high-level advanced conversations and debates relating to a particular area of social importance; (5) Professional development competencies, including a detailed knowledge of the ways in communication scholarship is produced and disseminated, and evidence of high-level participation in such activities.

 


II.  Achievement of Departmental Objectives for Students

 

The following summarizes the extent to which assessment goals were met in each program of study:

 

FILM

 

Final exams in Film 1010, Film Aesthetics and Analysis, Film 2700, History of Motion Pictures, and Film 4750, Film Theory and Criticism were evaluated based on a questionnaire consisting of 10 categories of principles and concepts of cinema. For each category a rating of 1-4 (1 being the lowest and 4 the highest) was given.  

 

The overall average (2 sections of 3 courses Film 1010, Film 2700, and Film 4750 for spring semester) of all exams were as follows: 

 

Film 1010 - 1.84

Film 2700 -  2.04

Film 4750 - 2.865 

 

These results show that at a fundamental level students in the Film major grasp a basic understanding of cinema in Film 1010 and are able to retain this knowledge over the course of their studies.  When evaluated in film 2700 and 4750, they demonstrate that they are advancing in their knowledge of Cinema.  Film faculty are eager to see that students are gaining a basic understanding of concepts and principles of cinema, that they are retaining and building on what they learned in Film 1010, and retaining and building on what they learned in Film 2700 and Film 1010. 

 

 

JOURNALISM

 

The overall AY0405 average (3 sections each for both fall and spring terms) of all term papers submitted to Jour 3060 is: 75.05. The overall AY0405 average (3 sections for fall semester and 2 sections for spring semester) of all papers submitted to Jour 3070 is: 81. The overall AY0405 average of all papers submitted in both courses is: 78.02. 

 

Students in the Journalism major (including broadcasting and public relations) meet the General Education goals the major addresses. Specifically, students write proficiently (Goal #1 - however, it must be noted that oral communication is not a component of Jour 3060 or Jour 3070 and, thus, was not measured), think critically (Goal #3), are able to understand and discuss contemporary issues (Goal #4), and use technology (Goal #5, specifically, word processing using computers) effectively. 

 

Preliminary data were collected to assess departmental learning outcomes in SP05.  The overall AY0405 average for Jour 1000 is: 40.8. The overall AY0405 average for Jour 3070 is: 57.6.  Students in the Journalism major (including broadcasting and public relations) demonstrate a greater depth of knowledge of various principles of journalism/mass communication theory (Goal #3) by a factor of 16.8, after having taken Jour 3070. With respect to the fourth goal, the overall AY0405 average for Jour 1000 is: 39.9. The overall AY0405 average for Jour 3060 is: 50.2.  Students in the Journalism major (including broadcasting and public relations) demonstrate a greater depth of knowledge of various principles of journalism/mass communication law (Goal #4) by a factor of 10.2, after having taken Jour 3060.

 

 

SPCH

 

Speech area faculty created assessment instruments that related to a series of major-designated content goals.  Aspects of all three goals for speech majors were assessed in AY 2004-2005, as well as outcomes relating to institutional general education goals.  Plans are underway to refine the assessment process, which will continue in AY 2005-2006, though both the objectives assessed and the procedures used will be clarified/more focused in the upcoming year.   Results are specified by goal in the following:

Overall, speech majors are performing well on a majority of the objectives set for them. Lower performance in some knowledge areas may be a function of irrelevant questions (i.e., information not actually taught in the courses) being included on the assessment exam and of the two-track division of the speech major (i.e., objectives cover both tracks of the major but students only take one track).

 

GRAD:  Master's Degree Program

 

Students enrolled in the department of communication's Master's Degree programs were judged to be satisfactorily reaching desired learning outcome levels, specified as follows:

 

 

 

 

 

 

GRAD:  Ph.D. Degree Program

 

 

 

 

 

 

 

 

III.  Changes to Procedures or Curriculum Based on Assessment

 

In the 05-06 academic year, the department has planned several activities designed to significantly improve the assessment process.  All address the need to make continued procedural improvements, and no curricular modifications are expected in the immediate future (apart from the longer term reviews which will start in 05-06 and which are explained below).  In each case planned activity is occurring in response to degree program-specific recommendations elaborated above.  Specifically:

 

 

 

 

 

 

o       Based on review of the Journalism major's goals and objectives by Harry Dangel and the General Education Committee, the "pre-test" of students enrolled in Jour 1000 was eliminated in December 2004, at the end of the fall semester. The committee stated that the Journalism faculty should not assess the extent to which students have progressed toward a certain goal but whether students have achieved goals set for them. The Journalism faculty should now determine what level of achievement it expects of students in the undergraduate Journalism major and assess that.  To facilitate this change (above) and to reduce the amounts of additional measurements taken and data collected during assessment, it is recommended that the 10-question assessments developed by Journalism faculty for Jour 3060 and Jour 3070 be discarded and, instead, that a new measure based on in-class student test performance (e.g., the average of all student test grades in both Jour 3060 and Jour 3070 each term) be used to assess Journalism goals #3 and #4. As with the other goals discussed above, if assigned grades on tests are used to determine whether undergraduate students meet these goals, an overall average of 70 (out of 100) or better is desired.

 

o       The biggest dilemma still facing the Journalism major (including broadcasting and public relations) when dealing with the new assessment requirement is that the major does not now have an exit course in which its goals would normally (and fairly easily) be assessed. The Journalism faculty plans to address this shortcoming, and will consider making a course such as Jour 4800, Mass Media & Society, its exit course again – or some other course focusing on communication ethics.  The Journalism faculty would also do well to consider the statement of Lee C. Bollinger, president of Columbia University, who on April 15, 2003, published a thoughtful discussion of "The Future of Journalism Education." It is available online at: http://www.columbia.edu/cu/president/communications%20files/futurejournalismeducation.htm (last accessed May 24, 2005). From this, the Journalism faculty may find that we are placing too much emphasis on communication theory (especially in our requirement that it be a core class) and not enough on communication ethics in the undergraduate Journalism major.

 

o       The department's Undergraduate Studies Committee might benefit from a reorganization that would more fully empower it to handle General Education and undergraduate portions of the annual assessment report. The committee chair/Director of Undergraduate Studies must take a proactive and inclusive approach to assessment. Individual committee members must be charged with collecting specific data each semester from faculty teaching courses whose materials will be used in assessment by certain deadlines and with developing specific portions of the annual report, which can then be collated and submitted to the department chair.  Because faculty must be brought fully into the assessment process and given clear responsibilities and specific deadlines for the collection of specific types of data and for its interpretation and written analysis, regularly scheduled area meetings of faculty are planned to evaluate findings and make recommendations for change in goals, assessment procedures and curriculum.

 

 

 

 

IV.  Changes in Department's Assessment Goals

 

No specific changes in identified curricular content or learning outcomes are planned for the 05-06 academic year.  Rather, the department's attention is focused on improving data collection efforts and the instruments by which these goals are measured, as detailed above.  The only specific anticipated change in assessment goals and outcomes is that the Undergraduate and Graduate Studies committees have been asked to review University general education goals and make another determination of whether our majors connect to each of those goals in assessable ways.  In some cases earlier review only identified limited connections to specific outcomes, or were unable to specify particular seminars where general education skill outcomes could be targeted and improved.