Student Learning Outcomes Assessment Plan

Oral Communication Competency as Developed in the GSU Core Curriculum

PRELIMINARY DRAFT

 I.  Description of the Outcomes that Define the Goal

Georgia State University’s General Education Goals, approved by the Undergraduate Council (1/30/04) and the University Senate (2/13/04) commit the university to developing oral communication competencies in its undergraduates.  Goal I.2 says students should “communicate effectively using appropriate oral or signed conventions and formats.”

This goal is responsive to an increasing national awareness that oral communication skills are vital for vocational success, and also central to inculcating civic and basic like skill competencies.  The evidence for such a claim is abundant and it is not necessary to systematically introduce it here, but evidence continues to accumulate.  According to a 2000 survey of 4000 human resources professionals, 71% cited solid communication skills as vital to employee retention.[1]  A recent study of business schools conducted by the Association to Advance Collegiate Schools of Business (a group that accredits American MBA courses) found that MBA graduates think that the ability communicate effectively with another person is the single most useful skill in their career, but that only six percent of business schools are even “moderately effective” in teaching that skill.[2]  The Public Forum Institute has reported that so-called “soft skills,” such as interpersonal communication, problem solving, and critical thinking are now more valued by employers than “hard skills.”[3]  Communication competency is also valuable to the university’s educational mission across all academic majors.  A 2001 report by the National Research Center on English Learning and Achievement (CELA) of the U.S. Department of Education highlights research that convincingly links training in high quality classroom discussion and oral communication skills with subject matter achievement in those same classrooms.[4]

Although oral communication proficiency is often built into course requirements by individual faculty, the core curriculum makes no explicit place for the targeted enhancement of such skills apart from SPCH 1000 (Human Communication), an elective option in Area B.  Along with PHIL 2410 and other Perspectives options, all of which are two-credit hour classes, the area is organized to foster communication and critical thinking competencies.

The communication discipline has given considerable attention to oral communication competency, especially since accreditation pressure (mainly from the North Central and Southern agencies) has highlighted the issue, and by now a wide range of resources speak to the philosophical and practical questions arising in communication assessment.  A 1994 summer conference of the National Communication Association was dedicated to the assessment issue, and its proceedings are widely cited in the field.  What follows in this section is largely derived from NCA Task Force publications relating to communication assessment.

“Oral communication competency,” as typically understood, involves the inculcation of a wide range of interconnecting skills.  Skilled communicators must be mindful of the ethical and evaluative standards of their culture, must adapt their messages in a context of multicultural difference, and are increasingly required to possess technological competencies.  Because these skills are often the central focus of disciplinary training in the major, most work relating communication competency to general education requirements focuses on the development of speaking and listening skills.

With respect to these two outcomes, the National Communication Association began developing disciplinary standards with appointment of a Task Force on Sophomore Level Exit Competency, named in the aftermath of the association’s 1982 El Pomar Conference.  The Task Force findings were then reviewed by the NCA Educational Policy Board Publication Board, and Administrative Committee.  They were then published in an NCA publication (Quianthy 1990).

The specific competencies identified by NCA researchers include the following (all adapted from Quianthy and available online at www.natcom.org):

The full range of competencies listed here demarcate a set too large to be adequately assessed in the summer pilot project.  We plan to cull out measures which connect to the experience of preparing and delivering a public speech (which is built into the assignment structure for SPCH 1000).  These measures include determinations of student competency in such performance areas as:  successfully gaining the audience’s attention, justifying a topic, establishing credibility, logical presentation style, successful use of supporting materials, and delivery measures such as those relating to eye contact, natural gesturing, and strong vocal projection. 

II.  Description of the Ways in Which Courses to be Assessed Contribute to this Goal

Academic units identified a total of 21 courses in the university core curriculum which potentially contribute to the general education oral communication requirement.   These include:

                ECON 2100         The Global Economy

ENGL 1101         English Composition I

                ENGL 1102         English Composition II

                FORL 1002          Elementary Language (coded by language)

                FORL 2001          Intermediate Language I (coded by language)

                FORL 2002          Intermediate Language II (coded by language)

                GEOL 1121-2      Introductory Geosciences

                GEOL 2001          Geologic Resources & Environment

                HIST 1111            Survey of World History to 1500

                HIST 1112            Survey of World History since 1500

                HIST 2110            Survey of U.S. History

                MATH 1070        Elementary Statistics

                MATH 1101        Introduction to Mathematical Modeling

                MATH 2211        Calculus of One Variable I

                MATH 2212        Calculus of One Variable II

                MATH 2215        Multivariate Calculus

                MATH 2420        Discrete Mathematics

                POLS 2401           Global Issues

                SPCH 1000          Human Communication

                THEA 2040         Introduction to the Theater

The committee undertook a syllabus review so that a determination could be made about the extent to which oral communication activities are centrally featured in the class curriculum.  In most cases we discovered that while particular instructors may integrate presentational activities into their grading, the overall course content was not designed to focus on oral skill development.  Where presentations appeared in course syllabi, it was often unclear whether or not they were graded with an eye to assessing communication competency (often the grading metric specified was purely content driven).  In other cases there was no apparent connection to oral communication development whatsoever.  In yet other cases, such as several of the English courses, oral communication competency appears to be built into the overall core curriculum architecture, but such courses are more likely to be used as vehicles for surveying written communication competency.  For the purposes of pilot data collection, which is what we intend for summer 2004, it did not seem prudent to impose a double burden on instructors.

The course where the closest connection to the General Education goal can be made is SPCH 1000, which aims to introduce students to the disciplinary work done by communication researchers (understood very broadly), and to provide some very limited oral performance opportunities.  Because the class is a two credit hour course, thus limiting weekly contact hours, performance is limited to a major speech which all students are required to give.  The standard SPCH 1000 syllabus thus has students take three tests and give one major graded speech.  On a case by case basis instructors build in other less formal occasions for oral presentation (e.g., some teachers have students give a short impromptu introduction speech).  The major speech is preceded by roughly three or four weeks of intensive lecturing on public speaking, with an emphasis on practical topics such as speech organization, research, etc., and the speech grade is 25% of the overall potential points a student can earn in the class.

As the standard syllabus language says, “SPCH 1000 is a broad-based course designed to give students an understanding of the basic theoretical principles of human communication.  Areas to be covered are communication basics, public speaking, persuasion, rhetoric, interpersonal communication, business communication, visual communication, and political communication.”  For the most part these topics are reviewed in lecture format, and the tests are objective (typically multiple choice in format).

SPCH 1000 enrolls a significant number of GSU undergraduates.  In a typical annual cycle approximately 2,500 students enroll (this accounts for roughly one-third of Area B enrollments, with the PHIL and PERS courses each attracting about an additional third apiece).  The course curriculum is standardized and closely supervised:  a lecturer (Dr. Steve Braden) is assigned to oversee the course, and annual orientations introduce instructional staff to the details of course administration.   The course is taught in two formats.  Large lecture classes which meet once a week, supplemented by once a week small-section breakouts (where students undertake performance activities) account for roughly 15% of overall credit hour generation.  All other students taking the class are taught in small standalone sections which are capped to enroll no more than 35 students (and are often smaller based on room size constraints).

Although SPCH 1000 does obviously connect to the enhancement of oral communication competency, it should be noted that this is not its major purpose, and the structural limitations (section size, staffing, and limited contact hours) would make it difficult to envision SPCH 1000 functioning as a major or exclusive vehicle for the enhancement of performance skills.  Another issue to be addressed is the potential problem of self-selection by students:  Because SPCH 1000 is one option in a menu, the students most apprehensive of public presentation (and thus in most need of attention and training) are the most likely to opt out of SPCH 1000 in favor of other classes where their fears are less likely to be activated.

Still, SPCH 1000 does enable the collection of pilot data, and because the course is mainly taken by first and second year students (from FY 2002-2004, 79.4% of enrolled students were frosh or sophomore standing, with 54.1% first years) data collected there can serve as benchmark data for later efforts undertaken in many majors to inculcate discipline-specific presentational expertise.[5]

III.  Description of the Assessment Methods

We plan to collect pilot data by use of three instruments, all of which are in common use.  Two are focused surveys which ask students to report their level of performance apprehension.  Both measures have been in wide use for decades and by now have accumulated considerable evidence of reliability.  Because these measures take little class time to administer, they can establish some benchmark pre- and post-test indication of apprehension and its reduction over the course of a term. 

The major measure we plan to put into use is an adapted version of the Competent Speaker, a university-level assessment instrument designed by the NCA, which has been adapted to also serve as the speech evaluation form in SPCH 1000.

The Competent Speaker instrument was developed by S. P. Morreale, M. R. Moore, et al., and was published in 1993.  The instrument was validity tested in several ways.  For instance, a panel of eleven speech communication educators was involved in the final editing, and negative correlations have been reported between this instrument and the six public speaking-related items on the Personal Report of Communication Apprehension (which is to say, more competent public speakers also have lower apprehension, a finding consistent with communication research and common sense).  A reviewer for NCA who independently analyzed available oral competency instruments rated this one highly, although work remains to develop items that better assess “thinking, reasoning, and evidence as substantial components of speech preparation and presentation” (Ellen Hay, Large Scale Assessment of Oral Communication:  K-12 and Higher Education, 2E, NCA, p. 52).  Hay calls attention to the usefulness of this instrument for “institutions that offer multiple sections of the public speaking course because it allows for standardized evaluation of the speeches.”

The Competent Speaker is standardized and has been cited for its high levels of inter-coder reliability.  Course instructors will be trained in the use of the instrument in a standard introductory session, and training materials accompany the instrument.  The goal is to combine the standard national instrument for oral communication with the course specific speech evaluation form, which we have accomplished fairly easily since the criteria for effective speech making tend to dovetail with the Competent Speaker measures.

Because SPCH 1000 only involves students in one public speaking event, even well planned data collection, predicated on commonly used assessment instruments, will not likely tell us very much about whether the GSU core produces or strengthens oral communication competency.  But in combination with similar assessments underway in the majors, some comparisons will be enabled between first-year and senior achievements.

IV.  Description of the Data Collection and Analysis Procedures

The committee will administer these instruments in summer session sections of SPCH 1000, after training instructors in their use.  The data will be aggregated and reported to the larger assessment committee, and make recommendations for future expansion of these efforts.

Given the number of students who move through the SPCH 1000 class, it may be advisable down the road to supplement these activities with random sampled videotaping and external assessment of oral communication performance.  In addition, it might be advisable to add the use of other instruments which are more closely focused on other aspects of oral performance, including assessment of listening skills.

References

Bassett, R.E., Whittington, N., and Staton-Spicer, A.  “The Basics in Speaking and Listening for High School Graduates:  What Should be Assessed?”  Communication Education 27 (1978): 293-303.

Christ, W.G. (eds.).  Assessing Communication Education:  A Handbook for Media, Speech, and Theater Educators (Hillsdale, N.J.:  Lawrence Erlbaum, 1994).

Jones, E.A.  Essential Skills in Writing, Speech and Listening, and Critical Thinking for College Graduates:  Perspectives of Faculty, Employers, and Policymakers (University Park, Penn.:  National Center for Postsecondary Teaching, Learning, and Assessment, 1994).

–––.  National Assessment of College Student Learning:  Identifying College Graduates’ Essential Skills in Writing, Speech and Listening, and Critical Thinking; Final Project Report [NCES Publication No. 95-001] (Washington, D.C.: U.S. Department of Education, Office of Educational Research and Improvement.

Morreale, S. and Brooks, M. (eds.). NCA Summer Conference Proceedings and Prepared Remarks:  Assessing College Student Competency in Speech Communication (Annandale, Va.: National Communication Association, 1994).

Quianthy, R.L.  Communication is Life:  Essential College Sophomore Speaking and Listening Competencies (Annandale, Va.:  National Communication Association, 1990).

Westphal-Johnson, N., and Fitzpatrick, M.A.  “The Role of Communication and Writing Intensive Courses in General Education:  A Five Year Case Study of the University of Wisconsin-Madison.”  JGE:  The Journal of General Education 51.2 (2002): 73-102.