Draft FAQs 9-30-04

Why are we doing Learning Outcomes Assessment? We have always assessed student learning outcomes—examinations, course projects, papers, internships, etc. The thing that is different about this process is that we are looking at the evidence of whether our students are learning what want them to as an element of program assessment and revision.

Is the Assessment of Learning Outcomes totally SACS-driven? (What are the SACS criteria?) Although the Southern Association of Colleges and Schools (SACS) requires all accredited institutions to conduct an institutional assessment of student learning, it is not the only group that requires the process. The Georgia Board of Regents and Georgia State University Senate policy also require programs to assess student learning.

What does “assessment” mean in Learning Outcomes Assessment? In higher education the term assessment traditionally has two components: gathering information in a prescribed manner (i.e., measurement) and interpreting that information to make decisions about policy, instruction, or assessment practices (i.e., evaluation).

What should we be assessing? We should be assessing what we as faculty have proclaimed as the major goals of our instructional programs, namely the knowledge, skills and values that students should master within our programs. These major outcomes would be aligned with course syllabus objectives.

Why are we assessing General Education Goals? Isn’t that the function of the core curriculum? The General Education Goals (or Gen Ed Learning Outcomes) were passed by the University Senate for all undergraduate students and will be assessed in the core by the General Education Assessment Committee for the Core of the Senate’s APACE Committee.

However, because the general outcomes are central to our work in our various majors, we assess them there as well. For example, although we expect all GSU graduates to effectively communicate in writing, use technology, or demonstrate critical thinking skills, etc., we recognize that the way these skills are expressed will vary across different programs. Therefore, each program defines and assesses how a graduate of their program will demonstrate competence in each of the seven GSU Learning Outcomes. These outcomes reflect the competencies that any and all GSU students should have upon graduation.

Why can’t we just report on whether or not students have passed our courses? Because our courses typically have multiple learning outcomes and objectives, we would lose the ability to assess whether our student have demonstrated mastery of a particular element of content or skill by just examining course grades. A major purpose of the assessment is to determine how we might improve our instructional programs, and so, we try to tailor the assessment process to the parts of courses the align with program outcomes, e.g., knowledge of elements of the content, ability to write clearly, ability to document and justify a conclusion, etc.

What is the difference between learning outcomes for a course and the student learning outcomes that we are assessing? Hopefully, none. The outcomes for courses and for programs should be aligned, although program outcomes will usually be more broadly stated than course outcomes.

Can we use test and course assignments that we already have in place for our assessment data? Certainly! Because we are already doing assessments of student learning when we assign grades, the embedded assessments we use in courses (tests, projects, papers, internships) become the core of learning outcomes assessment.

What is the timetable for assessment? When should the data be collected? Analyzed? Each program’s assessment report is due at the end of each academic year as part of the annual report to the Dean. Data should be collected during the year at the points when students are assessed within courses or, in the case of an external assessment (one not tied to a course), annually in time to include the interpretation of the results in the annual report.

Do we need to assess all of our students on all of our outcomes? That depends. If your program is relatively small, in order to be able to determine whether students have demonstrated mastery of each competency, it would be necessary to assess each student. In the case of larger programs, however, it is possible to select and assess a sample of students and draw conclusions about the program based on their performance.

What are some types of data that should be collected? The ideal data are direct assessment of students’ learning that might be drawn from capstone courses, course embedded assessments (e.g., examinations, projects, papers), externally administered examinations (e.g., accreditation examinations), internship performances (e.g., student teaching or clinical practice in nursing), and evidence from student work in portfolios. Indirect assessments are also helpful and would include student surveys, exit interviews, surveys from employers, and reports from external reviewers. An excellent guide is the Assessment Manual for the University of Wisconsin.

Do we need to use pre- and post-tests? No. While the results from pre-and post tests are valuable, they answer a different question—“How much did students learn?” The question to be answered with the assessment of student learning outcomes is simply, “Do students demonstrate that they know or do the program learning outcomes”? We are essentially measuring students' performance against the outcome (i.e., standard).

How can we manage and organize the data we collect? Because the results are expressed as simple tallies and percentages, data can be entered into a spreadsheet by each learning outcome in the program.

What kind of statistics would be acceptable in the annual report? Because the question we are answering is the extent to which our students have demonstrated that they know and can do the outcomes we have specified, most results will be in the form of tabulating the number and percentages of students (e.g., 45 students equaling 84% met outcome # 4).

Will assessment results be reported as aggregates only or by individual instructor? Results from assessment plans should show only program-wide results and never identify any individual faculty or students.

What happens if students don’t do very well on some of the learning outcomes? It is not our intent to show that all students have met all learning outcomes. Rather, we are demonstrating that we have a “culture of assessment” and that based on the systematic assessment of our students’ performance, we are working to improve the quality of student learning.

What does this effort mean in terms of faculty time? Ideally, the implementation of the assessment plan will be integrated into the flow of what a faculty member has been doing in class. The only difference is that data are aggregated by groups of students (e.g., those taking the final examination in XXX) rather than individually—as in grading. Sometimes, it will be necessary to capture a slice of an examination or project in order to have the assessment match the learning outcome (e.g., tallying performance on the subset of questions that required critical thinking or the part of a project that assessed communicating clearly in writing).

What does this effort mean in terms curricular design or syllabus changes? Again, in the best of circumstances, there would be no changes needed if our curriculum and syllabi match the learning outcomes specified by the program. However, there are likely to be instances in which faculty decide that some portion of an assessment in a course should be common for all sections in order to document an outcome. Likewise, there may be a need to negotiate where an outcome is covered in a program, if this has not already been decided.

We already do a lot of student outcome assessment for our accrediting organization. How is this different? The principle difference is that in addition to assessing student outcomes, you will also be discussing the results of those assessments as a faculty and making changes to improve those outcomes. In other words, you will not be providing evidence of a particular standard but rather using evidence to indicate where and how student outcomes can be improved.

What does the annual report look like? When is it due? The annual report of student learning outcomes asks for three things: a description of assessment procedures, a report on the extent to which students are demonstrating the program’s learning outcomes, and a description of any changes in the curriculum, assessment procedures, or student learning outcomes as a result of the assessment.

Who will read the annual report? The assessment report is sent to the Dean who reviews it and then submitted to the Provost as part of the annual report of the college/school. It becomes part of the University’s documentation of our work in assessing student learning and revising our procedures in order to improve.

How will the results be used? Departments will, of course, use the results from the assessment of student learning to decide whether students are demonstrating the mastery of content and skills they desire. It is assumed that nearly all programs will find some aspects of the instructional program they wish to improve—whether it’s the level of student learning in some area, the assessment process or the actual instructional goals. At the University level, the results will be reviewed to a) identify and disseminate best practices for improving student outcomes; b) discern patterns across programs that indicate a need for addressing outcome improvement at the university level rather than at the department level; and c) provide an accountability mechanism to support the culture of student outcomes improvement.

How will next year’s assessment be different from this year’s? The overall process will remain the same, but programs may certainly choose to refine the learning outcomes, assessment procedures, curriculum or instruction, or some combination of them.

How many years can we expect to do learning outcomes assessments? The process of documenting student learning outcomes for programs is now part of the fabric of out university. The process is a central part of the program review process, required by the Board of Regents, and linked to professional and institutional accreditation.

25. Do we need to assess all learning outcomes every year?