Department of Marketing

Robinson College of Business

Georgia State University

September 30, 2005

Annual Student Learning Outcomes Assessment Report

Undergraduate Marketing Program (BBA)

Marketing Master of Science Program

Academic Period: Spring 2005

I. Assessment Procedures

III. Changes to Procedures or Curriculum Based on Assessment

II. Achievement of Departmental Student Learning Objectives

IV.Changes in Department’s Assessment Goals

The undergraduate program of the Department of Marketing of the Robinson College of Business is assessed using both performance on the ETS Major Field Test and performance relative to the objectives stated in the Marketing Department’s Assessment Plan approved by the Department in 2002. The approved assessment plan, is attached at the end of this report.

ETS Major Field Test Performance:

Undergraduate students pursuing a degree in business administration from the Robinson College of Business at Georgia State University are required to pass BUSA 4980, the capstone course, to receive their bachelor’s degree. As part of that course, students take the Major Field Test, a standardized test created and scored by the Educational Testing Service in Princeton, NJ. The Major Field Tests are undergraduate outcomes assessments designed to measure the basic knowledge and understanding achieved by students in major fields of study. The results of the Major Field Test enable academic departments to better assess and refine curricula, gauge the progress of students compared to others in the program and those in similar programs at schools throughout the country. The Major Field Test is given to BBA students at more than 500 colleges and universities annually.

The disciplines represented on the Major Field Test are accounting, economics, management, quantitative business analysis, finance, marketing, the legal and social environment, and international issues. Knowledge of each discipline is measured using 10-15 questions. The Major Field Test for Business is made up of 120 questions designed to measure BBA student knowledge and ability to apply significant facts, concepts, theories, and analytical methods in key business disciplines. Every semester, scores are reported by major and all majors are evaluated on the knowledge of their own discipline as well as each of the other disciplines.

Georgia State University undergraduate marketing major percentile scores on marketing items are reported below for both semesters during the traditional academic year. The test is not administered during the summer semester. Georgia State marketing majors’ performance on the test has been collected from fall semester 2000 through spring semester 2005 with the exception of fall semester 2002 when ETS encountered problems with computing the percentile scores. The percentile scores reported below indicate the performance of GSU marketing majors on marketing questions. They are compared to other graduating seniors at other schools and at GSU who took the ETS Major Field Test over the previous three years. GSU senior marketing major scores on the marketing questions included in the Major Field Test are as follows:

Semester Marketing Majors’ Percentile Score on ETS Marketing Questions

ETS Major Field Test Conclusion:

The above data indicate that GSU marketing majors consistently perform well on the Major Field Test for Business/Marketing, confirming their excellent knowledge of basic marketing concepts.

Performance on Assessment Plan Objectives:

This assessment findings reported below are those taken from the assessment plan that was approved in 2002 and which is attached at the end of this report. The data is based on evaluation of student work product in MKT 4900: Marketing Problems, the marketing capstone course. Three different sections of the Marketing Problems course and two faculty members that taught the course are represented. The marketing capstone course provides an excellent opportunity for program assessment because of its requirement that students apply critical thinking skills to the solution of marketing problems, using the knowledge-base they have amassed from previous marketing courses as well as other courses in their undergraduate business curriculum.

Procedures Used: Course materials from Marketing Problems, MK 4900, were assembled for review by requesting the two marketing faculty, Drs. Barksdale and Nasser submit student work product for evaluation. Four Marketing faculty members. Drs. Ritu Lotia, Carol White, Bronis Verhage, and Carolyn Curasi agreed to evaluate the student work product.

Two different types of written assignments were chosen for the evaluation: individual case final exam write-ups and group projects. The group projects are of two types because the instructors use two slightly different formats. Both types of group projects are hands-on and applied and students work with real-world organizations.

The first type of project is a sponsored project. Sponsored projects are those that have a large, recognized sponsoring organization such as the FBI or Chevrolet and have a specific set of marketing objectives the students seek to accomplish. Sponsored projects are class-wide projects. Every member of the class works together as a marketing agency to accomplish the sponsor’s objectives. Students research the client’s problem, design a marketing program based on primary and secondary marketing research, implement their program, and evaluate their campaign’s results. The second type of group project used in the Marketing Problems class is a marketing plans. Marketing plan projects focus on the creation of a comprehensive marketing plan for a real client. Marketing plan projects are conducted in small groups by four or five students working together to create a viable marketing plan.

This report is confined to the written assignments developed during spring semester 2005. The evaluators, Drs. Lohtia, Verhage, Curasi, and White, were each asked to submit their evaluations to Dr Carol White for her to compute the average score for each scoring dimension. Each of the assessors was to submit a total of eleven scoring forms indicating that they had evaluated two sponsored projects, two marketing plans and seven case exam write-ups.

Separate scoring templates were created for the three different types of written assignment; one for evaluating case exam write-ups, a second for evaluating sponsored projects and a third scoring form for evaluating marketing plan projects. Each type of assignment was evaluated on slightly different dimensions, however, the scale used for scoring each dimension was the same across all three forms. The scale used is a five-point scale. The scale anchors are: 1. fails to meet the objective, 2. meets the objective at times, 3. meets objective, 4. exceeds objective at times, and 5. consistently exceeds the objective.

PROGRAM OBJECTIVE I

Students will demonstrate an ability to critically analyze a firm’s marketing problems and formulate effective marketing solutions in the key decision areas of price, promotion, product and distribution.

Assessment Methods: Performance on assigned marketing cases in MKT 4900.

Expected Outcomes: The student should be able to:

I.1 - identify actual marketing problems.

Average Score: 3.1

I.2 - explore alternative marketing solutions.

Average Score: 2.77

I.3 - evaluate the pros and cons of the alternative solutions.

Average Score: 2.61

I.4 - apply qualitative and quantitative data relevant to the problem.

Average Score: 2.55

I.5 - develop a recommendation that offers a coherent decision.

Average Score: 2.94

I.6 - support their decisions with qualitative and quantitative data.

Average Score: 2.77

Average Score: 2.79

Analysis and Recommendations: The results for the case analysis scores suggest that the students come close to meeting the objectives for each of the dimensions but that there is room for improvement. One improvement which will be examined for future assessment efforts is to make the instructions to those asked to assess the work product more explicit and detailed to insure greater consistency and more timely evaluations.

With respect to the scores on this first iteration of our assessment of the undergraduate program in marketing, the use of case exams as one element of the process suggests the faculty teaching case classes need to develop a clear set of objectives for the students. Student performance on two of the dimensions exceed the other scores. For example, students meet the objective for identifying marketing problems (3.11) and for developing a recommendation that offers a coherent decision (2.94). The reason that these two scores are higher than those of the other dimensions likely reflects the instructions given for the exam write-up which emphasized that the students focus on clearly specifying the problem in the case clearly and on providing a coherent recommendation. While the other dimensions are important in the case analysis process, they were not emphasized in the instructions for the exam and the students, therefore, did not address them in detail.

Caution is necessary in placing too much store in these initial findings because of sampling issues associated with the assignment that the faculty assessors were given. They were asked to choose seven cases at random from a group of 21. Given that assessors kept the cases for several days, it is likely that the same seven cases were graded by most of the faculty rather than allowing for a representative sampling which was the goal.

The Marketing Department’s Undergraduate Committee has just begun to examine the undergraduate curriculum in marketing. This data will be useful in suggesting possible changes to the pedagogy. For example, cases are not widely used in the undergraduate business curriculum. Some of the scoring results may be a consequence of the student’s lack of familiarity with the case method.

PROGRAM OBJECTIVE II

Students will understand the basic principles of strategic marketing planning.

Assessment Methods

Performance on assignments in the marketing capstone course either through:

Execution of a comprehensive marketing plan, or
Participation in a comprehensive sponsored marketing project for a marketing organization.

Assessment Method A: Marketing Plan

Expected Outcomes: The student should be able to:

II.A1 - develop a situation analysis using primary and/or secondary sources.

Average Score: 3.25

II.A2 - develop a product positioning.

Average Score: 3.25

II.A3 - identify viable target market segments.

Average Score: 3.625

II.A4 - establish measurable marketing objectives.

Average Score: 3.125

II.A5 - describe a sound marketing strategy through the application of the four P’s.

Average Score: 3.25

II.A6 - establish a feasible budget.

Average Score: 2.625

II.A7 recommend an evaluative method for assessing results.

Average Score: 2.625

Average Score: 3.107

Analysis and Recommendations:

The average score across all the various dimensions of the marketing plan project suggest that students are meeting the objectives. The assessors’ scores suggest that students were able to do an adequate job on five of the seven dimensions. The projects that were evaluated did not score as well on the dimension of establishing feasible budgets and the students’ recommendation of an evaluative method for assessing results. Overall, these scores, while preliminary and based on a sample of only two projects evaluated by four faculty, indicate that students are meeting the objectives although there is room for improvement on two dimensions which can likely be addressed quite simply by providing more instructions regarding budgeting and evaluating results in both the written assignment and in class.

One final caveat is that it is important that the faculty that evaluate the marketing plans be coached on the fact that the marketing plan projects are the product of small groups of four to five students and are not likely to be as elaborate as the sponsored project reports which are the product of an entire class. While there is no evidence that the faculty that evaluated the marketing plans were biased by the sponsored projects, it is something that should be addressed clearly in instructions on future evaluations.

Assessment Method B: Sponsored Marketing Project

Expected Outcomes: The student should be able to:

II.B1 - employ primary research on behalf of the sponsoring organization.

Average Score: 4.5

Analysis and Recommendations:

II.B2 - apply the four P’s (produce, place, promotion, price)

Average Score: 4.0

Analysis and Recommendations:

II.B3 - apply appropriate promotion tools (advertising, PR, sales promotion, direct marketing, personal selling).

Average Score: 4.16

Analysis and Recommendations:

II.B4 - demonstrate their understanding of the relationship between marketing decision making and sound financial principles.

Average Score: 4.0

Analysis and Recommendations:

II.B5 – executes the proposed strategy and evaluates success using primary research.

Average Score: 4.0

Analysis and Recommendations:

II.B6 – develop a feasible budget.

Average Score: 4.0

Analysis and Recommendations:

Average Score: 4.11

Analysis and Recommendations:

Sponsored projects do quite well overall. Students are consistently rated as exceeding the objective. Sponsored projects are mammoth class-wide efforts that require a marketing research project to develop a baseline as well as the creation of an actual marketing campaign, the implementation of the campaign as well as the evaluation of the campaign’s success.

Sponsored projects and marketing plan projects are both problematic as they recruiting real clients and extensive client cooperation to be successful.

PROGRAM OBJECTIVE III

Students will be able to develop primary and secondary research projects and interpret, evaluation, and synthesize the results for purposes of marketing decision making.

Assessment Methods: This objective is specific to the Marketing Research course and the Buyer Behavior course. We were unable to evaluate these during this round.

Expected Outcome 1.1

The student should be able to analyze, interpret, apply, and communicate findings from primary and/or secondary research.

Analysis and Recommendations:

ASSESSMENT OUTCOMES: OVERALL CONCLUSIONS

Our initial findings suggest that the undergraduate marketing students perform quite well since the scores on the dimensions that are defined in the program objectives in the initial plan all fall at about the midpoint of the scale indicating we are meeting the objective for that dimension.

Our student’s objective knowledge is sound but their applied knowledge is somewhat uneven. Their performance on case exams is not as strong as we would like it to be. This is likely because most students entering MK 4900, Marketing Problems have never encountered substantial cases before in any of their previous courses in the undergraduate curriculum in business. As a result, the pedagogy is completely new to them. Earlier exposure to case studies would help address this issue.

While there is room for improvement on specific dimensions, those improvements should be relatively straightforward now that they have been identified. Providing clear instructions to students regarding the format for the case exam along with instructions as to what areas need to be addressed would address the concerns we uncovered with respect to case analysis. Providing more information regarding the aspects of the marketing plan that will be evaluated such as budgeting and performance evaluation is also suggested by the evaluators’ scores.

There are some very bright spots in our findings. Specifically, our students perform extremely well on basic marketing knowledge as measured by the Major Field Test administered by ETS. Our undergraduate majors score at about the 90^th percentile level compared to other students in other majors at GSU and at other colleges and universities. Another important strength appears to be our class projects. Both the marketing plans and the sponsored projects were evaluated as meeting or exceeding the objectives overall. The sponsored projects did especially well scoring an average of 4.11 out of 5 averaged across all the dimensions.

Finally, we should not place too much credence in interpreting these initial results. The findings are for a single semester. There is not much data to draw sweeping generalizations. The real benefits will come from continuous improvement; using these initial results to improve both the evaluation process itself and to improve what takes place in the classroom as well. Through successive evaluations of future semesters’ work, patterns will emerge providing reliable insights.

Assessment: Plan and Results
Program	Marketing
Program mission	The undergraduate marketing curriculum is designed to provide students with the unique knowledge and skills to pursue a career in the field of marketing. The program 1.) gives students an understanding of the social, economic, legal and ethical aspects of the marketing environments in which organizations operate; and 2.) provides students with the analytical skills and technical competence necessary to formulate strategic, integrated solutions to marketing problems. The undergraduate marketing faculty is committed to delivering its services to students in an atmosphere that stimulates intellectual curiosity and fosters a desire for life-long learning.
Date revised	February 18, 2002
Assessment Plan			Assessment Results
Mission statement element	Assessment objective	Method of assessment and sampling approach; date assessment administered	Results for the objective	Change in assessment results (from the previous report) for the assessment objective	What will be done differently in the future to improve the program and its assessment
Analytical skills and technical competence necessary to formulate strategic, integrated solutions to marketing problems	Students will demonstrate an ability to critically analyze a firm’s marketing problems and formulate effective marketing solutions in the key decision areas of price, promotion, product and distribution.	Annual review of projects, case analyses, and/or exams in the core courses with particular attention to MKT 4900, the capstone course in the undergraduate marketing curriculum
Analytical skills and technical competence necessary to formulate strategic, integrated solutions to marketing problems	Students will understand the basic principles of buyer behavior (including market segmentation, consumer and industrial decision-making processes, cross-cultural differences, and factors affecting customer satisfaction), and they will be able to apply this knowledge in solving marketing problems	Annual review of projects, case analyses, and or exams in the core courses, with particular attention to MKT 4100-Buyer Behavior and MKT 4900 Marketing Problems.
Analytical skills and technical competence necessary to formulate strategic, integrated solutions to marketing problems	Students will be able to develop primary and secondary research projects and interpret, evaluate, and synthesize the results for purposes of marketing decision-making.	Annual review of projects, case analyses, and/or exams in the core courses, with particular attention to MKT 4200-Marketing Research and MKT 4900-Marketing Problems.

Copyright © 2001 Center for Teaching and Learning (CTL)/Center for Teaching and Learning with Technology (TLTC), Georgia State University. All rights reserved.