ANNUAL STUDENT LEARNING OUTCOMES

ASSESSMENT REPORT

 

School of Music

 

Submitted by David Myers, Associate Director, and John Haberlen, Director

 

July 2005

 

Academic year:  2004-2005

 I. Assessment Procedures

III. Changes to Procedures or Curriculum Based on Assessment

 II. Achievement of Departmental Objectives for Students    

A. Undergraduate-- 
     B. Graduate

IV. Changes in Department’s Assessment Goals

 

 

 

I/II.  Assessment Procedures/Achievement of Departmental Objectives

for Students

           

            A. Outcomes assessed  - undergraduate

Consistent with its accreditation by the National Association of Schools of Music, at the bachelor’s level the School of Music sustained  ongoing, embedded assessment of eight learning outcomes along four dimensions:  performance; theory; history and literature; and pedagogy.  The bachelor’s level music graduate:  

 

1.         Perceives, analyzes and explains the theoretical structure of music in styles and genres from pre-Renaissance through contemporary eras, and demonstrates independent synthesis of this knowledge when listening, creating, and performing.

 

Relevant dimensions:  performance; theory; history and

            literature

 

Assessment elements:  end-of-semester performance juries; final examinations in history and theory; rising junior “qualifying exams;” junior and senior recitals

 

Data collected:  ratings, comments, and placement levels from end-of-semester performance juries; results of rising junior qualifying exams; evaluations of junior and senior recital juries and recitals, along with student-written program notes

 

Data analysis: 

            Performance assessment rubrics include note accuracy, interpretation, technical execution, tone quality, phrasing and articulation, and artistry.  In 2004-2005, 39 juniors and seniors applied to perform required recitals and 33 were approved via assessments conducted by faculty panels (juries).  Of the 33 who performed, faculty panels rated 27 recitals “outstanding” and 6 “good.”  “Rising  junior” qualifying exams were taken in Fall 2005 by 13 students 12 of these were admitted to upper-division standing and 1 was retained at the same level.  167 students performed end-of-semester juries in Fall 2004.  Of these, faculty panels advanced the sequential placements of 163 based on technical, repertoire, and general knowledge standards for each of 8 placement levels.  Placement-level distribution for SP 05:

            I:          5 students

            II:         48

            III:        13

            IV:       48 (highest level for music management)

            V:         11

            VI:       25 (highest level for music technology)

            VII:      4 (highest level for music education)

            VIII:     13 (highest level for performance)

 

The grade distribution for Mus 2450 (final semester of theory sequence) for 54 enrolled students indicated a mean of 2.9 (4-point scale), with 50 students meeting or exceeding the required proficiency of 2.0.  Mus 4810 (2nd semester of music history) showed a mean of 2.1 for 20 students, with 18 meeting the required proficiency level of 2.0 or above.

 

Relevant General Ed Goals:

            Communication I.1 and I.2.  Students’ public recital program notes indicate consistently high levels of effective written communication regarding composers, dates, titles, and theoretical information using standard music-writing conventions.  Though creativity and non-verbal performance, other than signing, are completely absent from the university’s learning outcomes, the strong performance ratings of students indicate significant achievement in communicating effectively via musical creativity and performance.

 

            Collaboration II.1.  The majority of musical performances require collaboration between soloists and accompanists, or among chamber musicians.  Strong performance evaluations, achieved via sequential rehearsals and coaching, reflect high levels of collaborative achievement.

 

            Critical Thinking III.2, III.4.  Music performance is a continuing and cyclical process of performance, critical review, and revision.  The high level of student performance indicates students’ achievement in utilizing information as a basis for formulating hypotheses and testing them against existing models.

 

            Contemporary Issues IV.2.  In analyzing and performing contemporary music and the work of contemporary composers, students effectively analyze global and international questions in the arts.

 

2.         Places Western and non-Western music in historical and cultural context

 

            Relevant dimensions:  History and literature 

 

            Assessment elements:  final examinations in music history and world music; program notes for public recitals

           

Data collected:  Accumulated program notes for all student recitals; final grades in music history and world music

 

            Data analysis:  Student-written recital program notes, which are submitted to a review/revision process prior to public dissemination, indicate consistently accurate information regarding historical and cultural context of performed works.  The grade distributions in Mus 4810 (see Outcome 1) and Mus 4820 (World Music) indicate that 28 of 30 total enrolled students met the expected proficiencies, with course means of 2.1 and 3.9 (4-point scale) respectively.

 

            Relevant General Ed. Goals:

                        Communication I.1.  Recital program notes indicate effective written communication and formats for public dissemination of information relating to performed works.

 

                        Contemporary Issues. IV.1.  In placing contemporary works in cultural context, students demonstrate effective analysis of current issues within the discipline.

 

                        Critical Thinking III.1, III.2, III.3.  Research underlying program notes includes collecting data, formulating questions, and evaluating various perspectives and evidence for consistent information and interpretations.  Quality of program notes indicates students do this effectively.

 

3.         Uses appropriate applications of technology in listening, creating, and performing music

 

            Relevant dimensions:  performance; theory

 

            Assessment elements:  final examinations in Computer Applications in Music; student projects utilizing technology

 

            Data collected: final grades in computer applications classes; ratings of music technology projects

 

            Data analysis:  In SP 2005, all 16 enrolled students met the proficiency for computer applications (Mus 4730), with a mean of 3.8 (4-point scale).  Music technology senior project reviews showed a high proficiency pass rate for all 11 students, with a mean of 3.8.

 

            Relevant General Ed. Goals:

                        Technology VI.I.  In meeting School of Music proficiencies, students demonstrate use of computers and technology appropriate to the discipline.

 

4.         Performs diverse repertoire with advanced levels of musicianship in large ensembles, small ensembles, and as a soloist

 

            Relevant dimensions:  performance; theory; history and literature

 

            Assessment elements:  public performances; semi-annual performance juries

 

            Data collected: concert programs; observation of public concerts; performance assessments via semester juries, recital juries, and recitals.

 

            Data analysis:  Repertoire from ensemble, chamber music, and recital programs demonstrates a comprehensive sample of composers, eras, genres, styles, and difficulty levels.  Chamber music and jazz combos (Mus 3110, 3120, 3130, 3140, 3150, 3160) show a proficiency grand mean of 3.8 (4-point scale).  15 students enrolled in junior and senior recital in SP 2005 all received 4.0 on a 4-point scale for recital performances.

 

5.         Using knowledge of instruments and the voice, composes and improvises music in imitation, in original works, and/or with non-traditional sounds

 

            Relevant dimensions:  performance; theory

 

            Assessment elements:  final projects in Basic Improvisation; projects in Instrumentation and Orchestration; public performances; composition portfolios

 

            Data collected:  grades in Basic Improvisation; grades in composition seminar

           

            Data analysis:  Mus 3010 (Basic Improvisation) indicates that all 16 enrolled students achieved or exceeded the minimum proficiency (2.0 on 4-point scale) for Sp 2005.  Mus 4820 (composition seminar) for SP 2005 indicates a mean of 3.9 for 10 enrolled students, with 9 students at 4.0 and 1 at 3.0.  

 

6.         Demonstrates functional keyboard and conducting proficiencies

 

            Relevant dimensions:  performance

 

            Assessment elements:  satisfactory completion of Piano Class IV or departmental piano proficiency; final examinations in basic and advanced conducting

 

            Data collected:  numbers of students satisfactorily completing proficiency through Piano IV; grades in basic and advanced conducting

 

            Data analysis:  Mus 2720 (4th course in piano sequence) satisfies the piano proficiency with a grade of C or higher.  16 of 18 students satisfied the proficiency, receiving As or Bs, with a class mean of 3.3.  Basic conducting grades yielded a mean of 3.7 for 9 students in SP 2005, indicating that all satisfied the requisite knowledge and skills.  12 advanced conducting students in SP 2005 satisfied the standards with grades of A and B, with a mean of 3.9.

 

            Relevant General Ed. Goals:

                        Communication I.2.  Conducting requires significant verbal and non-verbal communication skills.  The indicators of strong knowledge and skill in conducting classes concurrently indicate effective verbal and non-verbal communication. 

 

7.         Demonstrates skills of oral, written, and verbal presentation and teaching to support sharing music with lay audiences and other musicians

 

            Relevant dimensions: history and literature; pedagogy

 

            Assessment elements:  exit proficiencies in relevant pedagogy classes; projects in music history, music education, and music management; supervised practica/internships

 

            Data collected:  grades in instrumental techniques classes; grades in Music-in-Education seminar; music education student teaching evaluations;

 

            Data analysis: Sp 2005 instrumental techniques courses show a grand mean of 3.7 for 18 enrolled students, all of whom received As and Bs, thus satisfying skill and knowledge expectations.  Music-in-Education enrolled 17 students in SP 2005, with a mean of 3.7 and all students meeting or exceeding the minimum proficiency of 2.0 or higher.  Student teaching exit evaluations for 9 enrolled students showed a mean of 3.7 on a 4-point scale, with all students receiving Bs or better.

 

            Relevant General Ed. Goals:

                        Communication I.2.  Evaluations indicate that students communicate effectively via oral conventions. 

 

8.         Demonstrates knowledge, synthesis, skills, problem-solving, and application consistent with careers relevant to a selected concentration within the School of Music

            Relevant dimensions:  performance; theory; history and literature

 

            Assessment elements:  junior-level qualifying examination for all concentrations; culminating experiences, including recitals, student teaching, and internships; portfolios associated with capstone experiences; composition portfolios

 

Data collected: number of students admitted to upper-division music concentrations (see Outcome 1); grades in senior recitals (see Outcome 1); grades in student teaching (see Outcome 7); grades in internships  

 

                                    Data analysis:  In addition to analyses reported above,

internship evaluations in music management/recording yielded means of 3.7 for 9 enrolled students in SP 2005. 

 

Explanation (11/05)

 

The placement levels must be viewed as individual levels of progress across 8 semesters of a program (4 at the master's level.)..  A baseline admission standard based on the performance audition places a student on Level I.  Within Level I there can be a range of achievement, generally indicated by the letter grade given at the end of the semester.  A

student receives both a jury grade (given by a panel of instructors) and a progress grade (assigned by the individual lesson instructor) each semester. The composite grade entered on the student's record (jury performance and semester progress) is determined by the faculty panel in a given area, e.g., voice, brass, woodwinds, etc.  The jury grade is retained on the "jury form" in the student's individual file as documentation of achievement and the assigned level for each semester.  Over the course of a program, the student typically has 4 to 8 jury assessments retained in his or her file.

 

Obviously, the objective is for every student to grow from his or her entering performance level to the highest level possible over the course of his or her program.  Thus, entering freshmen are rarely placed beyond Level I since the goal is to advance them as far as possible throughout a 4-year program with the benefit of high-calibre instruction. 

 

Advancing their placement is thus a matter of progress based on a combination of established criteria and individual progress -- which, I am pleased to say -- emphasizes the importance of maximum growth as opposed to regression to the mean.  This is authentic, individualized, performance/data-based assessment that existed long before it became the vernacular in education generally.  A student who is retained at the

same level for two semesters (i.e., typically receives a C or  lower) has a third semester to advance to the next level or is advised out of the program for lack of progress.

 

An initial (entrance) placement beyond Level I means that a student demonstrates performance achievement standards exceeding the highest expected standard of a Level I placement.  The same would be true for any more advanced placement level.  At the semester jury, a student's level may actually be accelerated if s/he demonstrates progress/achievement beyond the highest possible standard for the next sequential level.  Standards are both technical (right notes, etc.) and expressive (artistic expression).  Based on a student's entering performance or progress in the program, the level placement may be moderated by the anticipated time necessary for a student to reach maximum potential as demonstrated through the performance of 1 or more solo public recitals.  All programs except music technology, music education, and music management require 2 public recitals -- one in the junior year (30 minutes) and one in the senior year (60 minutes) --  and 1 junior-level recital is highly recommended for music education students.  Graduate programs vary, but most performance majors must do both a chamber and solo recital of 1 hour each.  Composition students must do public recitals both in performance and in compositions.  Thus, placements must ensure adequate time for faculty to prepare students for these public performances.  Before approval to perform a public solo

recital, a student must have participated in "all-area" public recitals (i.e., play 1 or 2 numbers on a program of by multiple students).  In addition, a solo public recital requires a pre-performance "jury" by a faculty panel that occurs several weeks prior to the public performance -- students who are insufficiently prepared are not permitted to perform the recital at that time.

 

Course numbers are differentiated to some degree by placement levels -- for example, Placements 1 through IV are "lower division" (2000) and V through VIII are "upper division" (3000 or 4000 depending on program).  8000 is common at the graduate level, though 7000 is used in some programs.

 

Levels cannot in and of themselves be interpreted as "doing well" or "not doing well."  They are progress indicators -- a student who progresses sequentially through the levels semester by semester is meeting minimal standards, but may also be achieving at any point along the continuum within that level (grades of A, B, C, etc.).  As indicated above, if s/he exceeds the maximum criteria for that level, s/he may be assigned to a higher level.  Because performance expectations for music management and technology are not as rigorous as those in other areas, their levels are fewer.

 

Level III, which you asked about, would mean that a technology student is roughly at the midpoint of his or her ultimate minimum performance expectations -- a grade of A, B, C, etc. would indicate where s/he falls within that Level.  For a music management student, a level III placement means s/he is making satisfactory progress toward the culminating standard indicated at Level IV.  For a performance major, Level III would ordinarily mean progress toward the cumulative standards indicated by Level VIII -- however, if the student fails to satisfy Level IV criteria at the semester jury, s/he may be held at Level III

and would then receive a letter from me indicating the consequences of  failure to advance.  I a student fails to progress past the current level over 3 semesters, s/he is advised out of the degree program.

 

 

            B.  Outcomes assessed – graduate level

Consistent with its accreditation by the National Association of Schools of Music, at the master’s level the School of Music sustained  ongoing, embedded assessment of six learning outcomes.  The master’s level music graduate:

 

1.         Demonstrates advanced levels of repertoire knowledge, technique, artistry, and style appropriate to a diverse representation of composers, historical eras, performance practices, and interpretive guidelines

 

            Assessment elements:  end-of-semester juried performance assessments; solo and chamber public performances

 

            Data Collected:  jury assessments; recital jury assessments; recital evaluations

 

            Data Analysis:  Assessment rubrics for performance include note accuracy, rhythm, phrasing and articulation, interpretation, expression, and overall artistry.  38 graduate students performed semester juries in Fall 05, and all 38 were advanced to the next placement level based on 4 levels of graduate performance standards.  16 graduate students performed solo or chamber recitals in SP 2005, with ratings of “outstanding” given to 15. 

 

2.         Demonstrates advanced analytical knowledge and skill for tonal and/or post-tonal music

 

            Assessment elements: summative assessments in Mus 6450 (advanced tonal analysis) and Mus 6460 (advanced post-tonal analysis)

 

            Data Collected:  Grade distributions for above courses

 

            Data Analysis:  Of 13 enrolled students, 11 received grades of B or higher, with a class mean of 3.3.

 

3.         Demonstrates research skills in music and advanced understanding of the literature and repertoire appropriate for his or her concentration

 

            Assessment elements:  course projects and grades; recital repertoire; comprehensive examinations and exit projects

 

            Data collected:  grade distributions; sample recital programs; comprehensive examination and recital evaluation forms

 

            Data analysis:  Graduate research course (Mus 8000) with 13 enrolled students yielded an overall mean of 3.5 and 10 grades of B or higher.  Recital programs evidence high standards of synthesis and expression of information to support performances.  16 students successfully completed master’s level comprehensive exams, 14 of whom received marks of “pass.”  Recital evaluation forms indicated that 15 of 16 students received assessments of “outstanding” on recital performances.      

 

4.         Demonstrates advanced historical-cultural understanding of music from one or more historical periods and in global context

 

            Assessment elements: summative course grades; recital repertoire and program notes; comprehensive exam

 

            Data collected:  music history course grades; sample recital programs and program notes; comprehensive exam evaluation forms

 

            Data analysis:  Music history (Mus 8830) in SP 2005 for 12 enrolled students yielded a class mean of 3.4, with 11 students receiving grades of B or higher.  Recital programs, performances, and notes indicate a synthesis of historical-cultural and accurate interpretive knowledge at a high level of understanding.  All comprehensive exams include at least one music history/literature question; 14 of 16 students received “pass” assessments on these exams.

 

5.         Demonstrates advanced levels of understanding and skill for teaching music in studio, classroom, and community settings

 

            Assessment elements:  summative course assessments; evaluations of internships and field experiences; exit projects

 

            Data collected:  course grades; exit project grades

 

            Data analysis:  Teaching-related courses that enrolled a total of 15 students in Spring 2005 yielded a grand mean of 3.0 on a 4-point scale.  10 students received Bs or above.  Teaching-related exit projects for eight student yielded a mean of 2.5, and four students received As or Bs.   

6.         Demonstrates accurate applications of technology in theoretical, performance, analytical, research, and pedagogical dimensions of music

 

            Assessment elements:  summative course assessments; observation of applications; comprehensive examinations

 

            Data collected:  no specific data for this assessment period

 

III.  Changes to Procedures or Curriculum Based on Assessment

 

1.         Add fields to departmental student database to ease collection of data for non-course-based assessments, including recital evaluations, qualifying exams for upper-division, and graduate comprehensive examinations

 

2.         Create a faculty Learning Outcomes Assessment Committee (Fall 2005) to designate focus outcomes for given years; relate data more systematically to outcomes; align syllabi, departmental outcomes, and university outcomes; and analyze data

 

3.         Given staff and faculty loads, reduce duplication of assessment effort by ensuring that syllabi specify outcomes and related rubrics tied to grading policies

 

4.         Continue to develop more specific performance standards for performance placement levels and more detailed rubrics for extra-coursework assessments

 

5.         Develop a new course in collaborative piano to emphasize non-verbal and verbal collaboration associated with music learning

 

IV.       Changes in Assessment Goals

 

1.         Based on syllabi review conducted as part of annual faculty evaluation, faculty were encouraged to state clearly assessable learning outcomes and provide grading rubrics for all courses.  Samples will be provided to assist faculty in developing outcomes/rubrics and tying them to departmental and university outcomes.  These will be integrated with grading policies rather than requiring a parallel and duplicative assessment process.

 

2.         Enhance the assessment of written and oral communication skills through more systematic analyses of program notes, teaching processes, and exit projects, allowing for sampling of assessments.

 

3.         Continue goals established in the original assessment plan.