Unit Name: COMMUNICATION
Prepared By: DAVID CHESHIER (Assoc Chair), 19 August 2005
Academic Year: 2004-2005
The following summarizes faculty-directed
assessment activity relating to five
degree programs administered by the department of communication (undergraduate majors in speech, journalism, and film, an MA program that includes tracks in film/video production, human communication & social influence,
and mass communication, and a
two-track PhD program with emphases
in Public Communication and Moving Image Studies).
The 2004-2005 academic year was the first in
which full-scale data collection was undertaken after several semesters of area
faculty planning and pilot data projects.
Assessment of the undergraduate majors was organized mainly through the
department's Undergraduate Studies Committee, now chaired by Dr. Doug
Barthlow. That committee's membership
is organized so that all departmental degree programs are represented. Accordingly, Dr. Greg Lisby and Barthlow
oversaw area faculty work relating to journalism, Dr. Ly Bolia organized work
connected to film, and Dr. Jaye Atkinson supervised assessment for the speech
major. At the graduate level,
assessment was organized by the department's two graduate directors, Dr. Greg
Smith and Dr. Michael Bruner, working with their colleagues on the department's
Graduate Studies Committee (Smith, Bruner, James Darsey, Kathy Fuller-Seeley,
and Ted Friedman). Both committees have
planned area faculty meetings for the FA05 semester to review 04-05 activity and
recommend improvements for the 05-06 academic year.
|
III. Changes to
Procedures or Curriculum Based on Assessment |
|
|
II. Achievement
of Departmental Objectives for Students A. Undergraduate--
|
The department of communication organized
significant assessment activity in the 2004-2005 academic year. Learning outcomes were identified in all
departmental majors in SP04, pilot data collected in SU05, and data fully
collected and evaluated in both FA04 and SP05. Those results are reported
separately in what follows, by major program (film, journalism, speech,
graduate). The theater major is being
discontinued and a proposal to reconstitute it as a BIS concentration is under
revision; for that reason data is not reported for theater.
In each of the following sections, the process
generated recommendations for continuing improvement which are being reviewed
by area faculty to strengthen content instruction in communication consistent
with the university's overall specification of learning outcomes. Similar outcomes are reported below for the
department's graduate programs. The
process was overseen by members of the department's undergraduate committee
(Miller, Lisby, Bolia, Atkinson, Barthlow) and by the department's graduate
directors (Smith and Bruner).
Assessment instruments were developed in coordination with the Provost's
office and the university Center for Teaching and Learning, with a specific
view to developing results relevant to disciplinary norms and national
accreditation standards.
FILM major
FILM area faculty identified three significant
learning outcomes. Those goals are:
Assessment was administered in Film 1010, Film
2700, and Film 4750 in FA04 and SP05 to determine the extent to which students
were able to demonstrate a fundamental understanding vs. in-depth knowledge of
various principles of Cinema history, theory, and criticism. The difference
between scores in three courses was used to assess the extent to which students
have gained knowledge of principles sought. These data are being used by
Film faculty to guide possible revision of individual course content, overall
revision of Film curriculum, and/or consideration of course prerequisites.
JOURNALISM major
Journalism faculty proceeded by matching
instruction in its degree tracks (including broadcasting & public
relations) with institutional general education goals. While not all goals are specifically
addressed in each of the major's classes, most are. The journalism assessment strategy for each of the connected
GenEd goals included the following:
In addition, major specific goals and learning
outcomes were identified for the undergraduate Journalism major. These goals will be specifically accessed in
AY05-06. These goals include: (1) Students will use communication
technology and they will be able to adapt to a changing technological
environment; (2) Students will communicate effectively in written, oral, visual
and/or digital forms; (3) Students will be critical and ethical communicators
and consumers of communication, and (4) students will understand the
development of communication industries in their political, legal, social and
economic environments. The plan is to
gather scores on 10-question assessment developed by Journalism faculty. Assessment instruments will be administered
in Jour 1000 and again in Jour 3070 each term to determine extent to which
students demonstrate a fundamental vs. in-depth knowledge of various principles
of journalism/mass communication theory.
Measured differences between scores in two courses will be used to
assess extent to which students have gained knowledge of principles sought.
Data will be used by Journalism faculty to guide possible revision of
individual course content, overall revision of journalism mass/communication
curriculum, and/or consideration of prerequisites
SPEECH major
Speech area faculty identified three major goals
for students majoring in the program, to be acquired regardless of the specific
major courses taken (which are organized into two tracks or major
concentrations). These include:
For each goal an assessment instrument was
created and classes identified so that speech majors would be captured and data
collected. The results discussion which
follows elaborates the details of these instruments more fully.
GRADUATE PROGRAMS in communication (M.A., Ph.D.)
Four professional development outcomes were
identified as central to the M.A. curriculum, and five to the doctoral program
in communication. Faculty assessed
attainment of these outcomes by reviewing submitted thesis and dissertation
projects, conducting year-end faculty evaluations of all doctoral students, and
review of completed comprehensive examinations.
The department's M.A. programs are organized to
achieve the following competencies: (1)
Significant mastery of research methods relating to knowledge generation in the
communication arts, with specific expertise generated in the methodological
approach best suited to the research interests of a particular student; (2) An
understanding of the key concepts of communication scholarship; (3) Written and
oral communication competency in conveying difficult communication concepts
clearly, and in such a way that knowledge generation is facilitated; (4)
Detailed knowledge development of a specific area of communication inquiry,
such that a student will be able to join high-level advanced conversations and
debates relating to a particular area of social importance.
The department's doctoral programs seek to
inculcate the following competencies:
(1) Significant mastery of research methods relating to knowledge
generation in the communication arts, with specific expertise generated in the
methodological approach best suited to the research interests of a particular
student and at the most disciplinarily advanced level; (2) Pedagogical
proficiency, which is to say the student should be able to organize
disciplinary findings and areas of inquiry in such a way as can be understood
by the field's larger publics (students, audiences, intelligent and interested
observers of communication phenomena, etc.); (3) An understanding of the key
concepts of communication scholarship, with particular attention to advanced
theoretical work undertaken by scholars in the cognate areas connected to
communication research; (3) Written and oral communication competency in
conveying difficult communication concepts clearly, and in such a way that
knowledge generation is facilitated such that students with the doctorate will
be competent and comfortable reporting their research in the field's leading
academic journals; (4) Detailed knowledge development of a specific area of
communication inquiry, such that a student will be able to join high-level
advanced conversations and debates relating to a particular area of social
importance; (5) Professional development competencies, including a detailed
knowledge of the ways in communication scholarship is produced and
disseminated, and evidence of high-level participation in such activities.
II. Achievement of
Departmental Objectives for Students
The
following summarizes the extent to which assessment goals were met in each
program of study:
Final exams in Film 1010, Film Aesthetics and
Analysis, Film 2700, History of Motion Pictures, and Film 4750, Film Theory and
Criticism were evaluated based on a questionnaire consisting of 10 categories
of principles and concepts of cinema. For each category a rating of 1-4 (1
being the lowest and 4 the highest) was given.
The overall average (2 sections of 3 courses
Film 1010, Film 2700, and Film 4750 for spring semester) of all exams were as
follows:
Film 1010 - 1.84
Film 2700 - 2.04
Film 4750 - 2.865
These results show that at a fundamental level
students in the Film major grasp a basic understanding of cinema in Film 1010
and are able to retain this knowledge over the course of their studies.
When evaluated in film 2700 and 4750, they demonstrate that they are advancing
in their knowledge of Cinema. Film faculty are eager to see that students
are gaining a basic understanding of concepts and principles of cinema, that
they are retaining and building on what they learned in Film 1010, and
retaining and building on what they learned in Film 2700 and Film 1010.
JOURNALISM
The overall AY0405 average (3 sections each for
both fall and spring terms) of all term papers submitted to Jour 3060 is:
75.05. The overall AY0405 average (3 sections for fall semester and 2 sections
for spring semester) of all papers submitted to Jour 3070 is: 81. The overall
AY0405 average of all papers submitted in both courses is: 78.02.
Students in the Journalism major (including
broadcasting and public relations) meet the General Education goals the major
addresses. Specifically, students write proficiently (Goal #1 - however, it
must be noted that oral communication is not a component of Jour 3060 or Jour
3070 and, thus, was not measured), think critically (Goal #3), are able to
understand and discuss contemporary issues (Goal #4), and use technology (Goal
#5, specifically, word processing using computers) effectively.
Preliminary data were collected to assess
departmental learning outcomes in SP05.
The overall AY0405 average for Jour 1000 is: 40.8. The overall AY0405
average for Jour 3070 is: 57.6. Students in the Journalism major
(including broadcasting and public relations) demonstrate a greater depth of
knowledge of various principles of journalism/mass communication theory (Goal
#3) by a factor of 16.8, after having taken Jour 3070. With respect to the
fourth goal, the overall AY0405 average for Jour 1000 is: 39.9. The overall
AY0405 average for Jour 3060 is: 50.2. Students in the Journalism major
(including broadcasting and public relations) demonstrate a greater depth of
knowledge of various principles of journalism/mass communication law (Goal #4)
by a factor of 10.2, after having taken Jour 3060.
SPCH
Speech area faculty created assessment
instruments that related to a series of major-designated content goals. Aspects of all three goals for speech majors
were assessed in AY 2004-2005, as well as outcomes relating to institutional
general education goals. Plans are underway to refine the assessment
process, which will continue in AY 2005-2006, though both the objectives
assessed and the procedures used will be clarified/more focused in the upcoming
year. Results are specified by
goal in the following:
Overall, speech majors are performing well on a
majority of the objectives set for them. Lower performance in some knowledge
areas may be a function of irrelevant questions (i.e., information not actually
taught in the courses) being included on the assessment exam and of the
two-track division of the speech major (i.e., objectives cover both tracks of
the major but students only take one track).
Students enrolled in the department of
communication's Master's Degree programs were judged to be satisfactorily
reaching desired learning outcome levels, specified as follows:
GRAD: Ph.D. Degree Program
III. Changes to Procedures or
Curriculum Based on Assessment
In
the 05-06 academic year, the department has planned several activities designed
to significantly improve the assessment process. All address the need to make continued procedural improvements,
and no curricular modifications are expected in the immediate future (apart
from the longer term reviews which will start in 05-06 and which are explained
below). In each case planned activity
is occurring in response to degree program-specific recommendations elaborated
above. Specifically:
o Based on review of the
Journalism major's goals and objectives by Harry Dangel and the General
Education Committee, the "pre-test" of students enrolled in Jour 1000
was eliminated in December 2004, at the end of the fall semester. The committee
stated that the Journalism faculty should not assess the extent to which
students have progressed toward a certain goal but whether students have
achieved goals set for them. The Journalism faculty should now determine what
level of achievement it expects of students in the undergraduate Journalism
major and assess that. To facilitate
this change (above) and to reduce the amounts of additional measurements taken
and data collected during assessment, it is recommended that the 10-question
assessments developed by Journalism faculty for Jour 3060 and Jour 3070 be
discarded and, instead, that a new measure based on in-class student test
performance (e.g., the average of all student test grades in both Jour 3060 and
Jour 3070 each term) be used to assess Journalism goals #3 and #4. As with the
other goals discussed above, if assigned grades on tests are used to determine
whether undergraduate students meet these goals, an overall average of 70 (out
of 100) or better is desired.
o The biggest dilemma
still facing the Journalism major (including broadcasting and public relations)
when dealing with the new assessment requirement is that the major does not now
have an exit course in which its goals would normally (and fairly easily) be
assessed. The Journalism faculty plans to address this shortcoming, and will
consider making a course such as Jour 4800, Mass Media & Society, its exit
course again – or some other course focusing on communication ethics. The Journalism faculty would also do well to
consider the statement of Lee C. Bollinger, president of Columbia University,
who on April 15, 2003, published a thoughtful discussion of "The Future of
Journalism Education." It is available online at: http://www.columbia.edu/cu/president/communications%20files/futurejournalismeducation.htm
(last accessed May 24, 2005). From this, the Journalism faculty may find that
we are placing too much emphasis on communication theory (especially in our
requirement that it be a core class) and not enough on communication ethics in
the undergraduate Journalism major.
o The department's
Undergraduate Studies Committee might benefit from a reorganization that would
more fully empower it to handle General Education and undergraduate portions of
the annual assessment report. The committee chair/Director of Undergraduate
Studies must take a proactive and inclusive approach to assessment. Individual
committee members must be charged with collecting specific data each semester
from faculty teaching courses whose materials will be used in assessment by
certain deadlines and with developing specific portions of the annual report,
which can then be collated and submitted to the department chair. Because faculty must be brought fully into
the assessment process and given clear responsibilities and specific deadlines
for the collection of specific types of data and for its interpretation and
written analysis, regularly scheduled area meetings of faculty are planned to
evaluate findings and make recommendations for change in goals, assessment
procedures and curriculum.
IV. Changes in Department's
Assessment Goals
No
specific changes in identified curricular content or learning outcomes are
planned for the 05-06 academic year.
Rather, the department's attention is focused on improving data
collection efforts and the instruments by which these goals are measured, as
detailed above. The only specific
anticipated change in assessment goals and outcomes is that the Undergraduate
and Graduate Studies committees have been asked to review University general
education goals and make another determination of whether our majors connect to
each of those goals in assessable ways.
In some cases earlier review only identified limited connections to
specific outcomes, or were unable to specify particular seminars where general
education skill outcomes could be targeted and improved.