Department of Nutrition

Appendices to Annual Report: Annual Student Learning Outcomes Assessment Report

Unit Name: Department of Nutrition--23 BS Graduates; 10 internship students; and 5 MS graduates.

Prepared by: Jana Kicklighter, PhD, RD, Chair , Date: June 21, 2005

Academic Year: 2004 -2005

A web-based assessment program, called TrueOutcomes, was purchased with funding from FY 05 student technology fees. The program consists of three major components, including a curriculum matrix, an electronic portfolio and an evaluation module. During the 2004-2005 academic year, the department utilized TrueOutcomes to create a curriculum matrix for each of our three academic programs, including the Didactic Program in Dietetics (DPD), Dietetic Internship, and M.S. in Health Sciences with a concentration in nutrition. Faculty who teach courses in the various programs aligned their course objectives with the learning outcomes, which provided a visual list of courses and experiences that contribute to each learning outcome. Faculty participated in three workshops on TrueOutcomes during the 2004-2005 academic year and learned how to align their course objectives to learning outcomes, how to develop rubrics, and how to input assessment data, based on the rubrics. We will continue to develop rubrics for grading some of the course assignments and portfolios during the 2005-2006 academic year and the electronic portfolio and evaluation components of TrueOutcomes will be utilized. An assessment of each program follows, based on the assessment data collected during the 2004-2005 period.

 I. BS Degree (Didactic Program in Dietetics - DPD)

The DPD is an accredited program through the Commission on Accreditation of Dietetic Education of the American Dietetic Association. Eight learning outcomes, which incorporate the knowledge and skills requirements of the accrediting agency, were identified for the DPD and elements of assessment include a comprehensive examination, a portfolio, an exit questionnaire and an alumni survey administered at one year and three years after graduation. Performance of our graduates on the national examination for Registered Dietitians (RD) was also examined. During spring semester of 2005, 23 senior students took the comprehensive exam, 23 senior students completed the exit questionnaire, 7 alumni completed the one-year alumni survey and 6 alumni completed the three- year alumni survey. Of the 23 students who completed the comprehensive exam, 60% answered more than one- half of the items correctly, while 40% answered one-half or less of the questions correctly. However, the same exam that was given to dietetic internship students, who have already completed B.S. degree programs, was administered to these senior students and items in the exam were not aligned with the learning outcomes, both of which limit the utility of this assessment method. Also, the portfolio was not required for this class of senior students but will be required for students who are currently in the junior DPD class, so that they can collect relevant artifacts and complete their reflections throughout their junior and senior years. Therefore, portfolio data will be included in the next annual assessment report. For calendar year 2004, we had 9 out of 10 DPD graduates take and pass the Registration Examination for Dietitians (RD) on the first attempt, resulting in a 90% pass rate. The national pass rate on the exam for this period was 78%. Our DPD graduates scored above the national average on the total test, 29.23 compared to the national average of 26.88, as well as the subscore for foods and nutrition (17.00 compared to the national average of 15.07) and foodservice (16.15 compared to the national average of 14.64). Data from the exit and alumni surveys were analyzed using simple numbers and tallies and trends were identified across the data collected. Findings for each learning outcome are summarized below.

1. Communicate effectively: The University basic core goals I, communication, and II, collaboration, align with this learning outcome. On the exit questionnaire, 18 of the 23 students rated their preparation in this area as either 5 (outstanding, with 5 being the highest) or 4 (more than satisfactory); on the 1-year alumni survey 7 of the 7 alumni rated their preparation in this area as either outstanding or more than satisfactory while on the 3-year alumni survey 5 of the 6 alumni rated their preparation as either outstanding or more than satisfactory.
2. Demonstrate an understanding of the influence of chemical, microbiological, and physiological disciplines as they affect food and nutrition: On the exit questionnaire, 21 of the 23 students rated their preparation in this area as either 5 (outstanding, with 5 being the highest) or 4 (more than satisfactory); on the 1-year alumni survey 7 of the 7 alumni rated their preparation in this area as either outstanding or more than satisfactory while on the 3-year alumni survey 6 of the 6 alumni rated their preparation as either outstanding or more than satisfactory.
3. Integrate psychological, social and economic aspects of the environment and examine how they individually and collectively affect food and nutrition: On the exit questionnaire, 15 of the 23 students rated their preparation in this area as either 5 (outstanding, with 5 being the highest) or 4 (more than satisfactory); on the 1-year alumni survey 7 of the 7 alumni rated their preparation in this area as either more than satisfactory or satisfactory, while on the 3-year alumni survey 5 of the 6 alumni rated their preparation as either outstanding or more than satisfactory.
4. Utilize critical thinking skills in the interpretation and application of research methodologies: The University basic core goals III, critical thinking, and VI, technology, align with this learning outcome. On the exit questionnaire, 13 of the 23 students rated their preparation in this area as either 5 (outstanding, with 5 being the highest) or 4 (more than satisfactory) while 5 students rated it as satisfactory or average (3), 6 students rated it as less than satisfactory (2), and one student rated it as unsatisfactory (1); on the 1-year alumni survey 7 of the 7 alumni rated their preparation in this area as more than satisfactory or satisfactory, while on the 3-year alumni survey 3 of the 6 alumni rated their preparation as outstanding,1 rated it as more than satisfactory, 1 as satisfactory, and 1 as less than satisfactory.
5. Demonstrate an understanding of the science of food and food policy in promotion of a healthy lifestyle and pleasurable eating in diverse population groups: The University basic core goal IV, contemporary issues, aligns with this learning outcome. On the exit questionnaire, 21 of the 23 students rated their preparation in this area as either 5 (outstanding, with 5 being the highest) or 4 (more than satisfactory); on the 1-year alumni survey 7 of the 7 alumni rated their preparation in this area as either outstanding or more than satisfactory, while on the 3-year alumni survey 5 of the 6 alumni rated their preparation as outstanding.
6. Demonstrate an understanding of the role of nutrients and food in human health, disease prevention, health promotion, and medical nutrition therapy: The University basic core goal V, quantitative skills, aligns with this learning outcome. On the exit questionnaire, 22 of the 23 students rated their preparation in this area as either 5 (outstanding, with 5 being the highest) or 4 (more than satisfactory); on the 1-year alumni survey 7 of the 7 alumni rated their preparation in this area as either outstanding or more than satisfactory, while on the 3-year alumni survey 6 of the 6 alumni rated their preparation as either outstanding or more than satisfactory.
7. Apply knowledge of management principles and systems in planning, monitoring, and evaluating dietetic services and practice and implementing of quality improvement programs: On the exit questionnaire, 10 of the 23 students rated their preparation in this area as either 5 (outstanding, with 5 being the highest) or 4 (more than satisfactory) while 11 students rated it as satisfactory or average (3), and 2 students rated it as less than satisfactory (2); on the 1-year alumni survey 7 of the 7 alumni rated their preparation in this area as either more than satisfactory or satisfactory, while on the 3-year alumni survey 1 of the 6 alumni rated their preparation as outstanding, 4 as more than satisfactory, and 1 as satisfactory.
8. Describe the impact of laws, regulations, and costs on health care systems and food and nutrition programs: On the exit questionnaire, 8 of the 23 students rated their preparation in this area as either 5 (outstanding, with 5 being the highest) or 4 (more than satisfactory) while 11 students rated it as satisfactory or average (3), and 4 students rated it as less than satisfactory (2); on the 1-year alumni survey 3 of the 7 alumni rated their preparation in this area as more than satisfactory and 4 rated it as satisfactory, while on the 3-year alumni survey 1 of the 6 alumni rated their preparation as outstanding, 2 as more than satisfactory, and 3 as satisfactory.

A meeting of nutrition faculty devoted to examination of the assessment data was held on May 9, 2005. No changes were recommended in the learning outcomes.

The following changes to the DPD program and assessment methods were recommended by faculty:

· The Undergraduate Team will develop a computerized comprehensive exam using primarily knowledge and comprehension level test questions currently used by instructors in courses included in the DPD program curriculum. The exam will simulate the national credentialing examination to become a Registered Dietitian in terms of proportion, content domains and types of questions. Questions will be classified into content domains, based on the learning outcomes, and circulated to faculty for validation, so that student performance on the various learning outcomes can be determined.

· The two learning outcomes graduating students perceived as not being met at a satisfactory level were research and health care systems, while alumni rated the areas of health care systems and management as only satisfactory. Faculty will re-examine the curriculum matrix to insure adequate coverage of these learning outcomes and will integrate research skills through all of the major courses and heighten students’ awareness that they are using research skills in their various course projects and assignments. Emphasis will be placed on skills in searching and critiquing the literature and understanding simple statistics and different levels of scientific evidence. Faculty also recommended that we integrate health care systems into the Medical Nutrition Therapy and the Nutrition Intervention courses. A part-time instructor currently teaches the two management-related courses and the recommendation for strengthening this area was to work more closely with this instructor to insure that students are meeting the learning outcome.

· Portfolios will be required for all senior DPD students in the 2005-2006 academic year. The rubric for evaluating the portfolio will be aligned with the learning outcomes, both for the DPD and the university core; 100% of the students will receive a proficient score on the portfolio rubric.

II. Dietetic Internship Program

The Dietetic Internship program, which is a program accredited by the Commission on Accreditation of Dietetic Education of the American Dietetic Association, has eight learning outcomes that incorporate the competencies specified by the accreditation standards. Elements of assessment for this program include a computerized pretest-posttest examination, a portfolio, an exit questionnaire and an alumni survey administered at one year after graduation. Performance of our internship graduates on the national examination for Registered Dietitians (RD) was also examined. During fall and spring semesters of 2004-2005, 10 interns took the pretest-posttest exam, 10 completed the portfolio and exit questionnaire, and 11 alumni completed the one-year alumni survey. Interns completed a computerized exam similar to the exam administered to become a RD. The students’ mean score on the pretest was 56.3, compared to 72.6 on the posttest, indicating an improvement in test scores. Items in the exam were linked to the content areas of the Registration Examination outlined by the Commission on Dietetic Registration. These items, however, were not aligned specifically with the learning outcomes, which limits the utility of this assessment method. The Dietetic Internship Director used a rubric to evaluate each intern’s portfolio and determined that each intern met learning outcomes 1-8. We had 14 dietetic internship graduates take and pass the RD exam in 2004, resulting in a 100% pass rate. Our dietetic internship graduates also scored above the national average on the total exam (30.29 compared to the national average of 26.53), as well as the foods and nutrition (17.82 compared to the national average of 14.77) and foodservice (17.29 compared to the national average of 14.39) sections. Data from the exit and alumni surveys were analyzed using simple numbers and tallies and trends were identified across the data collected. Findings for each learning outcome are summarized below.

1. Utilize management principles and marketing strategies in dietetic service and practice: On the exit questionnaire, 1 of the 10 students indicated that the competency “perform marketing functions” was not met sufficiently; for the 11 alumni responding to the 1-year alumni survey, the mean score was 3.5 on a scale of 5 to 1 (5 being highest).

2. Demonstrate ability to apply legislative, regulatory, and professional guidelines to dietetic service and practice: On the exit questionnaire, 1 of the 10 students indicated that the competency “participate in legislative and public policy processes as they affect food, food security, nutrition and health care” was not met sufficiently; for the 11 alumni responding to the 1-year alumni survey, the mean score was 4.0 on a scale of 5 to 1 (5 being highest).

3. Collect data, assess nutrition status, determine nutrient requirements, develop and implement interventions for individuals and groups in a variety of clinical and community settings, and document in appropriate records: On the exit questionnaire, 1 of the 10 students indicated that the competencies “consult with organizations regarding food access for target populations” and “participate in waived point-of-care testing” were not met sufficiently; for the 11 alumni responding to the 1-year alumni survey, the mean score was 4.6 on a scale of 5 to 1 (5 being highest).

4. Use the computer and other methods of technology to enhance dietetic service and practice: On the exit exam, 2 of the 10 students indicated that the competency “participate in the use of mass media to promote food and nutrition” was not met sufficiently while 1 student indicated that the competency “participate in facility management, including equipment selection and design/redesign of work units” was not met sufficiently; for the 11 alumni responding to the 1-year alumni survey, the mean score was 4.5 on a scale of 5 to 1 (5 being highest).

5. Apply and utilize strategies for effective communication and education in dietetic service and practice: For the 11 alumni responding to the 1-year alumni survey, the mean score was 4.8 on a scale of 5 to 1 (5 being highest).

6. Monitor and evaluate food production, distribution, and operations for dietetic service and practice for effectiveness, efficiency, and cost-containment: 1 of the 10 students indicated that the competencies “participate in facility management, including equipment selection and design/redesign of work units” and “participate in applied sensory evaluation of food and nutrition products” were not met sufficiently; for the 11 alumni responding to the 1-year alumni survey, the mean score was 4.2 on a scale of 5 to 1 (5 being highest).

7. Demonstrate an ability to participate in and conduct program evaluations: For the 11 alumni responding to the 1-year alumni survey, the mean score was 4.4 on a scale of 5 to 1 (5 being highest).

8. Behave professionally and ethically in dietetic service and practice: For the 11 alumni responding to the 1-year alumni survey, the mean score was 4.9 on a scale of 5 to 1 (5 being highest).

A meeting of nutrition faculty devoted to examination of the assessment data was held on May 9, 2005. No changes were recommended in the learning outcomes.

The following changes to the internship program and assessment methods were recommended by faculty:

· The rubric used for evaluating the portfolios will include assessment of each specific learning outcome; 100% of the interns will receive a proficient score on the portfolio rubric.

· Items in the computerized comprehensive exam will be categorized based on the various learning outcomes to facilitate examination of student performance on each outcome; the mean score of the post-test will be >70%, which is equivalent to the passing score on the Registration Examination for Dietitians (RD Exam).

· A workshop on reimbursement will be added to the dietetic internship program to better address the competency on coding and billing of dietetics/nutrition services to obtain reimbursement for services.

· Interns will be given an additional assignment to write a media piece to address the competency of using the mass media to promote food and nutrition.

III. M.S. in Health Sciences with a Concentration in Nutrition

The M.S. program has 5 learning outcomes. Elements of assessment for this program include a thesis, master’s project or portfolio as a capstone project, an exit questionnaire and an alumni survey administered at one year and three years after graduation. During spring semester of 2005, 3 students completed electronic portfolios as their capstone project, 5 graduating students completed the exit questionnaire, 5 alumni completed the one-year alumni survey, and 7 alumni completed the three-year alumni survey. Data from 11 recent graduates (Spring 2004 – Spring 2005) analyzed by the Office of Institutional Research were also examined in relationship to the learning outcomes. Graduates rated their competency in each area, upon entry into and exit from the program, on a scale of 1 (very weak competency) to 5 (very strong competency). The course instructor for the Portfolio Seminar course and each graduate student’s advisor used a rubric to evaluate each student’s portfolio, although the rubric did not include an evaluation of each specific learning outcome. However, the three portfolio students’ perceptions of their experiences during the graduate program as related to the learning outcomes indicated too much emphasis on content related to research methods and lack of adequate coverage of applied and clinical nutrition content. Also, these students had difficulty using assignments from their Advanced Normal Nutrition course as evidence of meeting learning outcome # 4. Portfolio students also identified only one course in their curriculum for learning outcome #5, related to health policy, and students perceived that the assignments that could be used for documentation of this learning outcome were limited. In addition, a rubric was developed for evaluation of students’ master’s projects and theses, but not in time for evaluation of the graduate student who completed a master’s project spring semester. Data from the exit and alumni surveys were analyzed using simple numbers and tallies and trends were identified across the data collected. Findings for each learning outcome are summarized below.

1. Demonstrate entry-level competence in the design, interpretation and

ethnical conduct of research: On the exit questionnaire, 5 of the 5 students rated their preparation as a 1, very effective (with 1 being the highest); on the 1-year alumni survey, 4 of the 5 alumni rated this area as very effective and 5 of the 5 alumni rated it as very useful to their current jobs (1, with 1 being the highest); on the 3-year alumni survey, 6 of 7 alumni rated the area as very effective and 5 of 7 rated it as very useful to their current jobs and 2 of 7 rated it as somewhat useful; for ability to “organize and interpret data appropriately” the mean score for the 11 recent graduates was 2.73, upon entry into the program, and 4.64, upon exit from the program; for ability to “locate and organize information from multiple sources” the mean score upon entry was 2.82, versus 4.55 upon exit from the program.

2a. Demonstrate technical and scientific oral and written communication

skills: On the exit questionnaire, 5 of the 5 students rated their preparation as very effective; on the 1-year alumni survey, 4 of the 5 alumni rated the area as very effective and 5 of the 5 alumni rated it as very useful to their current jobs; on the 3-year survey, 4 of the 7 alumni rated the area as very effective, 3 rated it as somewhat effective; 5 of the 7 3-year alumni rated it as very useful while 2 of the 7 alumni rated it as somewhat useful; for ability to “write clearly and effectively,” the mean score for the 11 recent graduates was 3.18, upon entry into the program, and 4.36, upon exit from the program; for ability to “speak clearly and effectively” the mean score upon entry was 3.36, versus 4.18 upon exit from the program.

2b. Use current and emerging technologies for information and

communication to enhance the practice and delivery of nutrition care in a professional manner: On the exit questionnaire, 4 of the 5 students rated this area as very effective; on the 1-year alumni survey, 3 of the 5 alumni rated it as very effective and very useful, while 2 of the 5 rated it as somewhat effective and 3 of the 5 rated it as somewhat useful; on the 3-year alumni survey, 7 of the 7 alumni rated this area as very effective and 6 of the 7 rated it as very useful to their current jobs; for ability to “effectively use technology” the mean score for the 11 recent graduates was 3.36, upon entry into the program, and 4.73, upon exit from the program.

3. Design and evaluate nutrition care plans and interventions for health

promotion and disease prevention and management for individuals and subpopulation groups based on the efficacy and strength of scientific evidence: On the exit questionnaire, 3 of the 5 students rated this area as very effective, 1 rated it as somewhat effective and 1 rated it as not effective; on the 1-year alumni survey, 2 of the 5 alumni rated the area as very effective, while 2 rated it as somewhat effective and 1 as not effective; 4 of the 5 1-year alumni rated the area as very useful; for the 3-year alumni survey, 4 of the 7 alumni rated the area as very effective, 3 rated it as somewhat effective and 1 rated it as not effective; 6 of the 7 3-year alumni rated it as very useful to their current jobs.

4. Comprehend the interrelationships between macro- and micronutrient intakes as they impact human health in normal and disease states: On the exit questionnaire, 4 of the 5 alumni rated this area as very effective; for the 1-year alumni survey, 3 of the 5 alumni rated it as very effective and very useful, while 2 of the 5 rated it as somewhat effective and somewhat useful; for the 3-year alumni survey, 3 of the 7 students rated it as very effective, 3 of the 7 as somewhat effective, and 1 as not effective; 4 of the 7 3-year alumni rated the area as very useful, 2 as somewhat useful and 1 as not useful.

5. Evaluate contemporary principles of health policy in the U.S. and other countries to better understand the essential components of delivering health services: On the exit questionnaire, 2 of the 5 students rated this area as very effective, while 3 students rated it as somewhat effective; on the 1-year alumni survey, 2 of the 5 alumni rated it as very effective, 2 of the 5 as somewhat effective, and 1 of the 5 as not effective; 2 of 5 1-year alumni found the area very useful for their current jobs, while 1 of 5 rated it as somewhat useful and 2 of 5 as not useful; for the 3-year alumni survey, 1 of the 7 alumni rated the area as very effective and very useful, while 6 of the 7 rated it as somewhat effective and somewhat useful to their current jobs.

A meeting of nutrition faculty devoted to examination of the assessment data was held on May 9, 2005. The following changes to the graduate program and assessment methods were recommended by faculty:

· No changes were recommended in the 5 learning outcomes for the graduate program, although faculty recommended that portfolio students be able to add one additional, customized learning outcome, such as one related to leadership or their chosen specialty track of nutrition education/counseling, sports nutrition, or public health.

· Include assessment of each specific learning outcome in the rubric for evaluating portfolios; 100% of the students will receive a proficient score on the portfolio rubric.

· Include assessment of each specific learning outcome in the rubric for evaluating research thesis/master’s projects and have each student’s committee members use the rubric beginning summer, 2005; 100% of the students will receive a proficient score on the research/master’s project rubric.

· Address the repetitiveness and sequence of information in the three research-related courses in the curriculum as related to learning outcome #1: eliminate the statistics course as a separate course and integrate statistical concepts into a two course, applied research methods sequence, and require the development of a proposal for a research project. The first research methods course should include components of a research paper, critique of research articles and the literature, basic methods in qualitative and quantitative research, and analytical processes and tools.

· Strengthen the curriculum related to learning outcomes #3 and 4, addressing nutrition care and intervention for health promotion and disease prevention and management and the interrelationships between macro-and micronutrient intakes: separate Advanced Normal Nutrition into a two course sequence and revise the Nutrition Issues course to address cutting edge information and trends, such as biotechnology and genomes, rather than repeating concepts related to the undergraduate medical nutrition therapy courses; revise the assignments for Advanced Normal Nutrition to more closely align with the learning outcomes.

· Increase the emphasis on evaluation of principles of health policy, as related to learning outcome #5, by incorporating more on nutrition and health-related policies and standards of practice into the Nutrition Issues and Nutrition Intervention courses and revising some of the assignments for the Trends in Health Policies, Practices and Laws course.