Assessment of the Bachelor of Business Administration (BBA) Degree Program

Assessment of the Bachelor of Business Administration (BBA) Degree Program

J. Mack Robinson College of Business

Georgia State University

2005

Assessment of the BBA program consists of assessing the achievement of the student learning outcomes related to (a) the core business curriculum and (b) specific majors within the BBA program. This report summarizes the assessment results for the BBA core business curriculum

Assessment Techniques

Several assessment methods are utilized to measure our success in achieving the desired student learning outcomes. The primary assessment methods are (a) the Educational Testing Service (ETS) Business Test, (b) the EBI Undergraduate Business Student Satisfaction Survey, and (c) assessment surveys administered in two of our business core courses.

1. The ETS “Business Test” is a major field exam that is intended to be administered to graduating seniors to test core knowledge in particular business subject matter areas. The Robinson College of Business has been administering the ETS Business Test to all seniors completing their capstone course during the spring and fall semesters since the 1990s. This exam looks at student performance in eight core subject areas that are standard to all AACSB accredited programs. The student and institution scores are ranked with those of other students and institutions that take the same version of the test. Although the other test takers are not peers of the Robinson College for the most part, they do provide a common background against which scores can be compared over the semesters. The results for students in the five major disciplines at the Robinson College are presented in aggregate scores here. They show steady and consistent performance of our students, and over the years show a general increase in the percentile scores that ETS is able to determine.

2. The EBI student satisfaction survey is administered to all seniors enrolled in BUSA 4980, the College’s capstone course, in the spring semester. In Spring 2005, 77% of those students (301/193) responded. A large number of multiple choice questions are asked of students on a wide-ranging set of issues concerning their experience at Georgia State University and in the Robinson College. Most of these raw scores are combined into factor scores on fifteen different topic areas.

Both the raw scores and the factors are then ranked against other institutions and against the Robinson College’s results over time. The other institutions against which these scores are compared are in three categories. First there are six selected institutions knows as the “Select 6” that the Undergraduate Program Council sees as “strong peer” institutions. For Spring 2005 these institutions were the University of Virginia, Arizona State University, The University of Georgia, The University of Maryland, Purdue University and Boston University.

The second comparative group is made up of all the participating schools in the same Carnegie Classification as Georgia State University. In Spring 2005 there were a total of 50 institutions, including Georgia State, in this Carnegie class. The third and final comparative group is made up of the entirety of the participating schools. In Spring 2005 a total of 157 institutions including Georgia State participated.

Demographic differences between the Robinson College and all three of these comparative groups exist. In gender, the Robinson College respondents were 55% female, while the three comparative groups were between 45% and 47% female. The Robinson College is 24% African American, compared to 3% in the Select 6, 5% in the Carnegie class peers, and 6% at all institutions. Asian Americans made up 19% of the Robinson College respondents, 13% in the Select 6 schools, 10% in the Carnegie class schools, and 8% of all institutions’ respondents. The self reported SAT/ACT scores of Robinson College students are significantly below those of the Select 6 and Carnegie class institutions, and comparable to all institutions as a whole.

3. Student projects completed in the capstone Strategic Management course of the J. Mack Robinson College of Business were assessed as part of the overall undergraduate assessment. Instructors in each of the sections observed student oral presentations and read student reports. All of these student projects concerned the integrated business simulation that is required in all sections. For this assessment a total of 43 different groups were assessed. They were assessed in the learning outcomes that related to communication skills, team work and computer usage.

Instructors rated each team on a Likert-type scale after observing the team’s presentation and written report. In addition each instructor made qualitative comments on the items for their students as a whole. Many of these were common across instructors. The most representative comments and those that are most likely to be addressable are listed after the results reports.

Expected Learning Outcomes

1. Students will demonstrate effective communication skills

A. Two questions on the EBI survey look at the extent to which the business program enhanced the student’s presentation and writing skills. These two questions’ raw results are then combined into a factor score. All questions are asked on a seven point scale with an option for the student to indicate “Not Applicable” or to omit answering the question.

The Factor into which the two questions are combined also includes the question used in Item 4 concerning the ability to work in teams and is titled by EBI as:

“Factor 12: Learning Outcomes: Effective Communication and Team Work”

The two questions that feed into this factor that are relevant to communication skills are:

Question 51 Learning Outcomes – To what extent did the Business program enhance your: Presentation skills

Question 52 Learning Outcomes – To what extent did the Business program enhance your: Writing skills

In the spring of 2005 the following results were reported for the Robinson College BBA degree recipients.

Item	Robinson College			“Select Six” Schools			GSU Rank With 6	50 Carnegie Class Institutions				All 157 Institutions
Item	N	Mean	Std Dev	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank With 6	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank
F 12	296	5.43	1.20	5.30	1.17	0.13	3^rd	5.23	1.14	0.20	13^th	5.31	1.12	0.12	64^th
Q51	296	5.70	1.28	5.66	1.28	0.04	3^rd	5.52	1.28	0.18	16^th	5.56	1.26	0.14	54^th
Q52	295	5.20	1.42	4.59	1.58	0.61	2^nd	4.70	1.51	0.50	9^th	4.88	1.46	0.32	45^th

The following table shows the scores that Robinson College students have obtained over the last three years in terms of the mean scores on these factors and the questions. The EBI Survey also reports the standard deviation of the scores for 2004 but not for 2003. These questions were not asked in years prior to 2003. The differences reported are all versus the 2005 results.

Item	2005’s Data			2004’s Data				2003’s Data
	N	Mean	Std Dev	N	Mean	Std Dev	Difference	Mean	Difference
Factor 12	296	5.43	1.20	291	5.56	1.13	-0.13	5.33	0.10
Quest 51	296	5.70	1.28	291	5.84	1.16	-0.14	5.62	0.08
Quest 52	295	5.20	1.42	290	5.39	1.30	-0.19	5.23	-0.03

B. The following results were obtained from the student survey in the BusA 4980 Strategic Management capstone course in the Fall Semester 2005.

BBA Program Objectives	BBA Expected Outcomes
Student will demonstrate effective communication skills	Synthesizing, arranging and presenting complex material competently in both written and oral form and adapting presentations to specific audiences and purposes
Assessment Criteria
Students on this team were able to synthesize, arrange and present the material for the final simulation-based written report. Strongly Strongly Disagree Agree 1 2 3 4 5 6 7 Responses 0 0 0 5 20 17 1

Students on this team were able to synthesize, arrange and present the material for the final simulation-based oral presentation.

Strongly Strongly

Disagree Agree

1 2 3 4 5 6 7

Responses

0 0 2 9 16 7 9

Students on this team were able to present their material in a manner consistent with the audience that they were addressing and for the purpose that was set out in the assignment.

Strongly Strongly

Disagree Agree

1 2 3 4 5 6 7

Responses

0 0 4 1 23 6 9

2. Students will effectively use computer technology

A. Two questions on the EBI survey look at the extent to which the business program enhanced the student’s ability to use and manage technology respectively. These two questions’ raw results are then combined into a factor score. All questions are asked on a seven point scale with an option for the student to indicate “Not Applicable” or to omit answering the question.

The Factor into which the two questions are combined is titled by EBI as:

“Factor 13: Learning Outcomes: Use and Manage Technology”

The two questions that feed into this actor are:

Question 54 Learning Outcomes – To what extent did the Business program enhance your: Ability to use technology

Question 55 Learning Outcomes – To what extent did the Business program enhance your: Ability to manage technology

In the spring of 2005 the following results were reported for the Robinson College BBA degree recipients.

Item	Robinson College			“Select Six” Schools			GSU Rank with 6	50 Carnegie Class Institutions				All 157 Institutions
Item	N	Mean	Std Dev	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank with 6	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank
F 13	296	5.15	1.50	4.99	1.40	0.16	1^st	5.06	1.40	0.09	21^st	5.13	1.38	0.02	86^th
Q54	296	5.17	1.53	5.08	1.45	0.09	3^rd	5.15	1.44	0.02	27^th	5.22	1.41	-0.05	102^nd
Q55	293	5.12	1.58	4.90	1.46	0.22	1^st	4.98	1.48	0.14	19^th	5.04	1.44	0.08	75^th

Item	2005’s Data			2004’s Data				2003’s Data
	N	Mean	Std Dev	N	Mean	Std Dev	Difference	Mean	Difference
Factor 13	296	5.15	1.50	291	5.25	1.44	-0.10	5.24	-0.09
Quest 54	296	5.17	1.53	291	5.33	1.51	-0.16	5.33	-0.16
Quest 55	293	5.12	1.58	287	5.15	1.48	-0.03	5.13	-0.01

B. The following results were obtained from the BusA 4980 student survey.

BBA Program Objectives	BBA Expected Outcomes
Student will effectively use computer technology	Exhibiting competency with computer software (word processing, spreadsheets, graphics).
Assessment Criteria
Students on this team exhibited an effective and appropriate use of computer software skills (word-processing, spreadsheets, graphics, etc.) in their final written project. Strongly Strongly Disagree Agree 1 2 3 4 5 6 7 Responses 0 0 0 2 4 28 9

Students on this team exhibited an effective and appropriate use of computer software skills (word-processing, spreadsheets, graphics, etc.) in their final oral presentation.

Strongly Strongly

Disagree Agree

1 2 3 4 5 6 7

Responses

0 0 0 11 11 12 9

Students will demonstrate analytical skills in solving business problems.

Four questions on the EBI survey look at the extent to which the business program enhanced the student’s critical thinking and problem solving skills. These four questions’ raw results are then combined into a factor score. All questions are asked on a seven point scale with an option for the student to indicate “Not Applicable” or to omit answering the question.

The Factor into which the four questions are combined is titled by EBI as:

“Factor 15: Learning Outcomes: Critical Thinking and Problem Solving”

The four questions that feed into this factor that are relevant to communication skills are:

Question 58 Learning Outcomes – To what extent did the Business program enhance your: Ability to think critically

Question 59 Learning Outcomes – To what extent did the Business program enhance your: Ability to define problems

Question 60 Learning Outcomes – To what extent did the Business program enhance your: Ability to solve problems

Question 61 Learning Outcomes – To what extent did the Business program enhance your: Ability to analyze and interpret data

In the spring of 2005 the following results were reported for the Robinson College BBA degree recipients.

Item	Robinson College			“Select Six” Schools			GSU Rank With 6	50 Carnegie Class Institutions				All 157 Institutions
Item	N	Mean	Std Dev	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank With 6	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank
F 15	296	5.73	1.16	5.56	1.14	0.19	3^rd	5.53	1.11	0.20	13^th	5.57	1.09	0.16	51^st
Q58	296	5.74	1.23	5.49	1.29	0.25	3^rd	5.50	1.26	0.24	11^th	5.56	1.23	0.18	47^th
Q59	296	5.71	1.21	5.51	1.23	0.20	3^rd	5.51	1.20	0.20	11^th	5.55	1.18	0.16	46^th
Q60	296	5.73	1.21	5.56	1.20	0.17	3^rd	5.55	1.20	0.18	12^th	5.58	1.18	0.15	52^nd
Q61	296	5.73	1.27	5.61	1.21	0.12	3^rd	5.56	1.21	0.17	13^th	5.59	1.20	0.14	51^st

Item	2005’s Data			2004’s Data				2003’s Data
	N	Mean	Std Dev	N	Mean	Std Dev	Difference	Mean	Difference
Factor 15	296	5.73	1.16	291	5.80	1.08	-0.07	5.64	0.09
Quest 58	296	5.74	1.23	290	5.74	1.30	0.00	5.60	0.14
Quest 59	296	5.71	1.21	291	5.82	1.10	-0.11	5.64	0.07
Quest 60	296	5.73	1.21	291	5.80	1.14	-0.07	5.64	0.09
Quest 61	296	5.73	1.27	291	5.82	1.15	-0.09	5.68	0.05

B. The following table indicates the level of achievement in core business disciplines as measured by the ETS Business Test. This test indicates that RCB graduates are at the 80% percentile of business schools that utilize this assessment technique.

Seniors in BUSA 4980 ETS Scores and Percentiles by Semesters for Five Large Majors Combined
Assessment Indicators	Spring 2004 GSU N=329	Fall 2003 GSU N=275	Spring 2003 GSU I N=275	Fall 2002 GSU N=263	Spring 2002 GSU N=301	Fall 2001 GSU N=244	Spring 2001 GSU N=284	Fall 2000 GSU N=268
Accounting	52.9	50.9	52.5	50.5	50.8	49.2	52.6	51.1
	0.9	0.9	0.9	1.0	0.9	1.0	1.0	0.9
	85	80	84.0	62	50	72	64
Economics	46.9	46.4	46.8	43	41.1	39.7	41.2	40.8
	0.8	0.8	0.9	0.9	0.7	0.8	0.8	0.8
	60	60	71	0	52	43	53	50
Management	62.8	64.2	62.1	60.6	55.3	53.9	55.9	55.7
	0.9	0.9	0.9	1.0	0.8	0.9	0.8	0.8
	70	75	64	64	53	68	65
Quantitative Business Analysis	66.0	65.6	65.2	63.7	52.7	52.1	52.9	51.9
	0.8	0.9	0.9	0.9	0.7	0.8	0.7	0.7
	85	85	87	72	67	73	65
Finance	43.1	40.8	41.4	38.8	41.0	39.4	42.0	40.3
	1.0	1.0	1.1	1.1	1.0	1.0	1.1	1.0
	80	75	79	66	58	73	63
Marketing	54.1	54.0	53.1	51.3	47.6	49.3	50.7	49.2
	0.9	0.9	0.9	1.0	0.8	0.8	0.8	0.9
	85	85	82	46	58	67	57
Legal and Social Environment	56.7	55.2	55.7	52.8	43.5	42.5	42.3	43.1
	1.0	1.0	1.1	1.0	1.0	1.0	1.0	1.0
	85	70	76	65	57	56	61
International Issues	50.8	49.6	48.9	44.6	45.9	43.7	44.7	44.6
	1.0	75.0	1.0	0.9	0.9	1.0	1.0	1.0
	80	70	79	55	40	48	45
TOTAL	159.4	158.8	158.5	156.2	155.9	154.6	156.8	155.6
St. Dev.	13.1	12.3	12.5	13	11.8	11.5	13	12.1
St. Error	0.07	0.70	0.75	0.80	0.68	72	0.78	0.74
Instit. %ial	80	80	81	62	54	69	59

Students will function effectively as a team member.

One question on the survey looks at the extent to which the business program enhanced the student’s ability to work in a team. This question’s raw result is then combined into a factor score. All questions are asked on a seven point scale with an option for the student to indicate “Not Applicable” or to omit answering the question.

The Factor into which the question is combined also includes the questions used in Item 1 concerning the ability to communicate and is titled by EBI as:

“Factor 12: Learning Outcomes: Effective Communication and Team Work”

The question that feed into this factor that is relevant to working in teams is:

Question 53 Learning Outcomes – To what extent did the Business program enhance your: Ability to work in teams

In the spring of 2005 the following results were reported for the Robinson College BBA degree recipients.

Item	Robinson College			“Select Six” Schools			GSU Rank With 6	50 Carnegie Class Institutions				All 157 Institutions
Item	N	Mean	Std Dev	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank With 6	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank	Wgtd Mean	Std. Dev.	Diff in Means	GSU Rank
F 12	296	5.43	1.20	5.30	1.17	0.13	3^rd	5.23	1.14	0.20	13^th	5.31	1.12	0.12	64^th
Q53	295	5.41	1.46	5.64	1.43	-0.23	3^rd	5.48	1.37	-0.07	28^th	5.49	1.36	-0.08	100^th

Item	2005’s Data			2004’s Data				2003’s Data
	N	Mean	Std Dev	N	Mean	Std Dev	Difference	Mean	Difference
Factor 12	296	5.43	1.20	291	5.56	1.13	-0.13	5.33	0.10
Quest 53	295	5.41	1.46	289	5.47	1.53	-0.06	5.15	0.26

B. The results of the BusA 4980 survey also revealed an ability to function well in a team environment.

BBA Program Objectives	BBA Expected Outcomes
Students will function effectively as a team member.	Exhibiting the ability to interact effectively as a group or team member.
Assessment Criteria
Students on this team showed during the semester an ability to function as a team in executing their roles in the simulation and in producing the final project deliverables. Strongly Strongly Disagree Agree 1 2 3 4 5 6 7 Responses 0 0 3 6 6 22 6

Students will demonstrate an appreciation for learning as a lifelong process.

In the Spring of 2005 some RCB-specific questions were added to the EBI survey. Several of these questions dealt with graduates’ expectations relating to the future and their desire to continue the learning process.

Georgia State University

EBI Undergraduate Business Exit Assessment

Institution Specific Questions

Question

Mean

Std Dev

CQ01. After you receive your degree, which of these statements would best describe your employment situation?

Answers	N	Response Rate

I will have a new full time job

120

41.00%

I will have the same full-time job

16.00%

I will have a new part time job

3.00%

I will have the same part-time job

5.00%

I am seeking, but do not have employment

29.00%

I will continue as a full-time student

6.00%

I have no plans for work or school

2.00%

2.91

1.95

296

CQ02. Please supply the annual base salary range for any full time job you are going to have upon graduation, or select one of the alternative answers.

Answers	N	Response Rate

Less than $20,000 per year

4.00%

Between $20,000 and $29,999 per year

10.00%

Between $30,000 and $39,999 per year

27.00%

Between $40,000 and $49,999 per year

17.00%

Over $50,000 per year

13.00%

I am still seeking full time employment

23.00%

I will not have or seek a full time job

7.00%

4.23

1.62

294

CQ03. As you think about your future after graduation, which of these statements best describes your view toward further formal education?

Answers

Response Rate

I will not pursue any further education

8.00%

I am unsure about further education

26.00%

I will take more undergraduate classes

3.00%

I will likely seek an MBA at GSU

109

37.00%

I will likely seek an MBA elsewhere

16.00%

I will likely seek a non-MBA masters

6.00%

I will likely attend Law School

5.00%

3.67

1.57

293

CQ04. If English is your native language, please indicate how many credit hours you took of any foreign language during your undergraduate program, whether at GSU or at another school. If English is not your native language, please mark number 7.

Answers

Response Rate

I did not study any foreign language.

133

46.00%

I took 1-3 semester hours

13.00%

I took 4-6 semester hours

15.00%

I took 7-9 semester hours

6.00%

I took 10-12 semester hours

2.00%

I took 13 or more semester hours

2.00%

English is not my native language

17.00%

2.77

2.22

289

CQ05. If English is not your native language, please indicate how many credit hours you took in a language other than your native language (a 3rd language) during your undergraduate program, whether at GSU or elsewhere. If English is your native language, please mark number 7.

Answers

Response Rate

I did not take any 3rd language classes

24.00%

I took 1-3 semester hours

5.00%

I took 4-6 semester hours

4.00%

I took 7-9 semester hours

4.00%

I took 10-12 semester hours

1.00%

I took more than 13 semester hours

1.00%

English is my native language

168

62.00%

5.01

2.64

273

CQ06. How many times in the last one/two year(s) have you met with an advisor in the Undergraduate Office of Academic Assistance?

Answers

Response Rate

None

12.00%

28.00%

21.00%

14.00%

4.00%

more than 5

9.00%

3.59

1.67

294

CQ07. How many times in the last one/two year(s) have you used the facilities of the University's Placement Services on the Main Campus.

Answers

Response Rate

None, because I have a job already

23.00%

None, because I was not interested

18.00%

16.00%

2-3

13.00%

4-5

2.00%

More than 5

6.00%

Didn't know it existed

22.00%

3.57

2.24

295

CQ08. How many times in the last one/two year(s) have you used the facilities of the Robinson College's Placement Services at 35 Broad Street.

Answers	N	Response Rate

None, because I have a job already

21.00%

None, because I was not interested

16.00%

6.00%

2-3

6.00%

4-5

1.00%

More than 5

2.00%

Didn't know it existed

144

49.00%

4.50

2.61

296

CQ09. During your time at Georgia State University did you participate in an internship program that was related to your career, regardless of whether this internships was obtained through Georgia State or not?

Answers

Response Rate

Yes

101

34.00%

195

66.00%

1.66

0.47

296

Georgia State University

(OrderID: 9818)

Action Plan

The RCB is encouraged by the assessment results that indicate achievement in realizing the desired learning outcomes for BBA graduates. The one change suggested by this assessment process is the addition of more assessment in the BComm 3950 business communications course. This required course for BBA majors provides an excellent opportunity to more systematically assess the oral and written communication skills of BBA students.