Assessment of the Bachelor of Business Administration (BBA) Degree Program

J. Mack Robinson College of Business

Georgia State University

2005

 

Assessment of the BBA program consists of assessing the achievement of the student learning outcomes related to (a) the core business curriculum and (b) specific majors within the BBA program.  This report summarizes the assessment results for the BBA core business curriculum

 

Assessment Techniques

 

Several assessment methods are utilized to measure our success in achieving the desired student learning outcomes.  The primary assessment methods are (a) the Educational Testing Service (ETS) Business Test, (b) the EBI Undergraduate Business Student Satisfaction Survey, and (c) assessment surveys administered in two of our business core courses.

 

1.                  The ETS “Business Test” is a major field exam that is intended to be administered to graduating seniors to test core knowledge in particular business subject matter areas.  The Robinson College of Business has been administering the ETS Business Test to all seniors completing their capstone course during the spring and fall semesters since the 1990s.  This exam looks at student performance in eight core subject areas that are standard to all AACSB accredited programs.  The student and institution scores are ranked with those of other students and institutions that take the same version of the test.  Although the other test takers are not peers of the Robinson College for the most part, they do provide a common background against which scores can be compared over the semesters.  The results for students in the five major disciplines at the Robinson College are presented in aggregate scores here.  They show steady and consistent performance of our students, and over the years show a general increase in the percentile scores that ETS is able to determine.

 

2.                  The EBI student satisfaction survey is administered to all seniors enrolled in BUSA 4980, the College’s capstone course, in the spring semester.  In Spring 2005, 77% of those students (301/193) responded.  A large number of multiple choice questions are asked of students on a wide-ranging set of issues concerning their experience at Georgia State University and in the Robinson College.  Most of these raw scores are combined into factor scores on fifteen different topic areas.

 

Both the raw scores and the factors are then ranked against other institutions and against the Robinson College’s results over time.  The other institutions against which these scores are compared are in three categories.  First there are six selected institutions knows as the “Select 6” that the Undergraduate Program Council sees as “strong peer” institutions.  For Spring 2005 these institutions were the University of Virginia, Arizona State University, The University of Georgia, The University of Maryland, Purdue University and Boston University.

 

The second comparative group is made up of all the participating schools in the same Carnegie Classification as Georgia State University.  In Spring 2005 there were a total of 50 institutions, including Georgia State, in this Carnegie class.  The third and final comparative group is made up of the entirety of the participating schools.  In Spring 2005 a total of 157 institutions including Georgia State participated.

 

Demographic differences between the Robinson College and all three of these comparative groups exist.  In gender, the Robinson College respondents were 55% female, while the three comparative groups were between 45% and 47% female.  The Robinson College is 24% African American, compared to 3% in the Select 6, 5% in the Carnegie class peers, and 6% at all institutions.  Asian Americans made up 19% of the Robinson College respondents, 13% in the Select 6 schools, 10% in the Carnegie class schools, and 8% of all institutions’ respondents.  The self reported SAT/ACT scores of Robinson College students are significantly below those of the Select 6 and Carnegie class institutions, and comparable to all institutions as a whole.

 

3.         Student projects completed in the capstone Strategic Management course of the J. Mack Robinson College of Business were assessed as part of the overall undergraduate assessment.  Instructors in each of the sections observed student oral presentations and read student reports.  All of these student projects concerned the integrated business simulation that is required in all sections.  For this assessment a total of 43 different groups were assessed.  They were assessed in the learning outcomes that related to communication skills, team work and computer usage. 

 

Instructors rated each team on a Likert-type scale after observing the team’s presentation and written report.  In addition each instructor made qualitative comments on the items for their students as a whole.  Many of these were common across instructors.  The most representative comments and those that are most likely to be addressable are listed after the results reports.

 

Expected Learning Outcomes

 

1.         Students will demonstrate effective communication skills

 

A.        Two questions on the EBI survey look at the extent to which the business program enhanced the student’s presentation and writing skills.  These two questions’ raw results are then combined into a factor score.  All questions are asked on a seven point scale with an option for the student to indicate “Not Applicable” or to omit answering the question.

 

The Factor into which the two questions are combined also includes the question used in Item 4 concerning the ability to work in teams and is titled by EBI as:

 

“Factor 12: Learning Outcomes: Effective Communication and Team Work”

 

The two questions that feed into this factor that are relevant to communication skills are:

 

Question 51 Learning Outcomes – To what extent did the Business program enhance your: Presentation skills

Question 52 Learning Outcomes – To what extent did the Business program enhance your: Writing skills

 

In the spring of 2005 the following results were reported for the Robinson College BBA degree recipients.

 

Item

Robinson College

“Select Six” Schools

GSU

Rank

With 6

50 Carnegie Class Institutions

All 157 Institutions

N

Mean

Std

Dev

Wgtd

Mean

Std.

Dev.

Diff in Means

Wgtd

Mean

Std.

Dev.

Diff in Means

GSU

Rank

Wgtd

Mean

Std.

Dev.

Diff in Means

GSU

Rank

F 12

296

5.43

1.20

5.30

1.17

0.13

3rd

5.23

1.14

0.20

13th

5.31

1.12

 0.12

64th

Q51

296

5.70

1.28

5.66

1.28

0.04

3rd

5.52

1.28

0.18

16th

5.56

1.26

 0.14

54th

Q52

295

5.20

1.42

4.59

1.58

0.61

2nd

4.70

1.51

0.50

9th

4.88

1.46

 0.32

45th

 

The following table shows the scores that Robinson College students have obtained over the last three years in terms of the mean scores on these factors and the questions.  The EBI Survey also reports the standard deviation of the scores for 2004 but not for 2003.  These questions were not asked in years prior to 2003.  The differences reported are all versus the 2005 results.

 

Item

2005’s Data

2004’s Data

2003’s Data

 

N

Mean

Std Dev

N

Mean

Std Dev

Difference

Mean

Difference

Factor 12

296

5.43

1.20

291

5.56

1.13

-0.13

5.33

 0.10

Quest 51

296

5.70

1.28

291

5.84

1.16

-0.14

5.62

 0.08

Quest 52

295

5.20

1.42

290

5.39

1.30

-0.19

5.23

-0.03

 

B.                 The following results were obtained from the student survey in the BusA 4980 Strategic Management capstone course in the Fall Semester 2005.

 

BBA Program Objectives

BBA Expected Outcomes

Student will demonstrate effective communication skills

Synthesizing, arranging and presenting complex material competently in both written and oral form and adapting presentations to specific audiences and purposes

Assessment Criteria

Students on this team were able to synthesize, arrange and present the material for the final simulation-based written report.

 

Strongly                                                                                                                  Strongly

Disagree                                                                                                                  Agree

 

    1                    2                    3                    4                   5                  6                  7

 

Responses

 

    0                    0                    0                    5                   20                17                1

          

 

 

 

Students on this team were able to synthesize, arrange and present the material for the final simulation-based oral presentation.

 

Strongly                                                                                                                  Strongly

Disagree                                                                                                                  Agree

 

    1                    2                    3                    4                   5                  6                  7

 

Responses

 

    0                    0                    2                    9                   16                 7                  9

 

 

Students on this team were able to present their material in a manner consistent with the audience that they were addressing and for the purpose that was set out in the assignment.

 

Strongly                                                                                                                  Strongly

Disagree                                                                                                                  Agree

 

    1                    2                    3                    4                   5                  6                  7

 

Responses

 

    0                    0                    4                    1                   23                 6                 9

 

 

 

 

2.                  Students will effectively use computer technology

 

A.        Two questions on the EBI survey look at the extent to which the business program enhanced the student’s ability to use and manage technology respectively.  These two questions’ raw results are then combined into a factor score.  All questions are asked on a seven point scale with an option for the student to indicate “Not Applicable” or to omit answering the question.

 

The Factor into which the two questions are combined is titled by EBI as:

 

 “Factor 13: Learning Outcomes: Use and Manage Technology”

 

The two questions that feed into this actor are:

 

Question 54 Learning Outcomes – To what extent did the Business program enhance your: Ability to use technology

Question 55 Learning Outcomes – To what extent did the Business program enhance your: Ability to manage technology

 

In the spring of 2005 the following results were reported for the Robinson College BBA degree recipients.

 

Item

Robinson College

“Select Six” Schools

GSU

Rank

with 6

50 Carnegie Class Institutions

All 157 Institutions

N

Mean

Std

Dev

Wgtd

Mean

Std.

Dev.

Diff in Means

Wgtd

Mean

Std.

Dev.

Diff in Means

GSU

Rank

Wgtd

Mean

Std.

Dev.

Diff in Means

GSU

Rank

F 13

296

5.15

1.50

4.99

1.40

0.16

1st

5.06

1.40

0.09

21st

5.13

1.38

 0.02

86th

Q54

296

5.17

1.53

5.08

1.45

0.09

3rd

5.15

1.44

0.02

27th

5.22

1.41

-0.05

102nd

Q55

293

5.12

1.58

4.90

1.46

0.22

1st

4.98

1.48

0.14

19th

5.04

1.44

 0.08

75th

 

The following table shows the scores that Robinson College students have obtained over the last three years in terms of the mean scores on these factors and the questions.  The EBI Survey also reports the standard deviation of the scores for 2004 but not for 2003.  These questions were not asked in years prior to 2003.  The differences reported are all versus the 2005 results.

 

 

Item

2005’s Data

2004’s Data

2003’s Data

 

N

Mean

Std Dev

N

Mean

Std Dev

Difference

Mean

Difference

Factor 13

296

5.15

1.50

291

5.25

1.44

-0.10

5.24

-0.09

Quest 54

296

5.17

1.53

291

5.33

1.51

-0.16

5.33

-0.16

Quest 55

293

5.12

1.58

287

5.15

1.48

-0.03

5.13

-0.01

 

B.                 The following results were obtained from the BusA 4980 student survey.

 

BBA Program Objectives

BBA Expected Outcomes

Student will effectively use computer technology

Exhibiting competency with computer software (word processing, spreadsheets, graphics).

Assessment Criteria

Students on this team exhibited an effective and appropriate use of computer software skills (word-processing, spreadsheets, graphics, etc.) in their final written project.

 

Strongly                                                                                                                  Strongly

Disagree                                                                                                                  Agree

 

    1                    2                    3                    4                   5                  6                  7

 

Responses

 

    0                    0                    0                    2                   4                  28               9

 

 

 

Students on this team exhibited an effective and appropriate use of computer software skills (word-processing, spreadsheets, graphics, etc.) in their final oral presentation.

 

Strongly                                                                                                                  Strongly

Disagree                                                                                                                  Agree

 

    1                    2                    3                    4                   5                  6                  7

 

Responses

 

    0                    0                    0                   11                   11                 12              9

 

 


  1. Students will demonstrate analytical skills in solving business problems.

 

Four questions on the EBI survey look at the extent to which the business program enhanced the student’s critical thinking and problem solving skills.  These four questions’ raw results are then combined into a factor score.  All questions are asked on a seven point scale with an option for the student to indicate “Not Applicable” or to omit answering the question.

 

The Factor into which the four questions are combined is titled by EBI as:

 

“Factor 15: Learning Outcomes: Critical Thinking and Problem Solving”