Examples and Analysis of Selected Assessment Plan Models

These samples mostly were selected from School of Humanities as their programs were submitted earliest, have had time for multiple revisions, and have the most discussion with the University committee. Note again, that all of these are working documents that likely will be revised after the experience of the first implementation of the assessment procedures.

Not all of the plans illustrated below are "perfect" nor should they be seen as the "best" or only way to address the issues and formats presented by the University Assessment Committee. They were chosen primarily to show the campus community the range of assessment techniques and strategies developed to meet the individual needs of each program.

I. COURSE-BASED ASSESSMENT USING SENIOR CAPSTONE, including essays and portfolio preparation followed analysis by selected department faculty

BA in English

1. Goal:

Students are expected to be familiar with major writers, periods and genres of English and American Literature and to be able to place important work and genres in their historical context.

2. Objectives:

a. Discuss a major work or author in English and/or American Literature, or compare two or more works and authors, for example, analyze the character of Satan in Milton's Paradise Lost.

b. Analyze a novel, short story, poem, play or a significant piece of prose showing familiarity with the techniques and literary contexts of the particular genre being examined.

c. Show a knowledge of the historical context or literary period of the work or author being examined, for example, a discussion of Cranes Maggie as an example of American Naturalism.

Comment: Clearly stated. Good use of concrete examples. Tells the reader what is expected of student.

3. Outcomes Criteria.

Completion of Senior Project (See description attached) consisting of a portfolio of four papers and a reflective essay demonstrating that the student has met a substantial number of the objectives of the English major.

Comment: Good use of summative assessment and sampling techniques. Assessment built into common core course. Level of achievement and criteria for judging whether the student has reached the desired proficiency and knowledge still is not specified.

4. Assessment methods.

Portfolios reviewed and evaluated by departmental committee.

Comment. Excellent use of special group of faculty to review student products. What the committee will look for, how they will reduce data to useful form, or what scale will be used to judge success is not stated yet. Will students be provided feedback and guidance on the results of faculty analysis?

5. Time Frame:

Students will take the course proposed and will prepare the portfolios before the end of the senior year. Evaluation of the portfolios will be scheduled for each quarter.

6. Who will do the Assessment.

Department Chair and appointed Committee

7. Type of Feedback.

At end of each evaluation, the committee will write a report describing the strengths and weaknesses which the portfolios demonstrate.

Comment. Very useful approach. .

8. How data will be used to improve program or revise curricula?

The department will meet as a whole to discuss findings and will recommend to the chair and curriculum committee methods of improving department procedures and curricula.

Comment: Shares findings with whole department. Provides context for revision of courses and teaching strategies

II. COURSE-BASED "PROFICIENCY EXAMS" as barriers at end of lower division and upper-division work.

BA in Spanish and French

1. Goal.

Students are expected to have achieved at least an Advanced Level in speaking, listening, reading, writing and cultural knowledge. "Advanced is the equivalent to level 3 on the scale developed by the American Council on the Teaching of Foreign Languages. The Scale consists of five levels: Novice, Intermediate, Advanced, Superior and Native.

In oral proficiency, for example, "Advanced" means that the student is able to speak the language with sufficient structural accuracy and vocabulary to participate effectively in most formal and informal conversations on practical, social and professional topics. They must be able to discuss their particular interests an special fields of competence with reasonable ease. Their comprehension is expected to be quite complete for a normal rate of speech. Their vocabulary demonstrates sufficient breadth that they rarely have to grope for a word; their accent may be obviously foreign, although they have good control of grammar, and their error virtually never interfere with understanding and rarely disturb the native speakers.

Comment: Very specific and focused, based on nationally agreed upon standards.

2. Objectives: Speaking

ACTFL Advanced level speaking is characterized by the ability to:

(a) converse in a clearly participatory fashion;

(b) initiate, sustain, and bring to closure a wide variety of communicative tasks, including those that require an

increased ability to convey meaning with diverse language strategies due to a complication or an unforeseen turn of events;

{c) satisfy the requirements of school and work situations;

(d) narrate and describe with paragraph length connected discourse

(e) Students demonstrate use of conversational skills by:

1. Interacting with a fluent speaker on everyday topics

2. Exchanging factual information

3. Providing information on an event

(f) Students describe and provide information using a variety of discourse strategies

(g) Students narrate an event that has already taken place

Comment: The objectives use strong action verbs to operationalize the goals in tangible and observable outcomes that can be evaluated and measured. The criteria for judging whether a student met the objective are concrete and are embedded in the objectives.

3. Outcomes Criteria (How will students demonstrate or complete the objectives?)

Students will demonstrate their listening and speaking abilities by

(a) satisfactory completion of those courses dedicated specifically to developing listening and speaking skills

(b) successful completion of all major courses;

{c) successful completion of the speaking/listening comprehensive portion of the junior and senior assessment examinations.

4. Assessment Methods and Measures to be Used.

Since the acquisition of listening and speaking skills is integral to any foreign language major's outcomes, this skill is assessed at every stage of the educational process. At the beginning levels, students are continually assessed during class time, within each test and at the end of each course. As soon as feasible, students are required to conduct interviews of native speakers or to mimic real life situations. In the remaining intermediate and advanced courses, all lectures and activities conducted in the target language must be understood by the students in order for them to successfully complete the requirements. In general, then, assessment used will include:

(a) Examination(s) for each course;

(b) junior assessment examination using the Video Oral Proficiency Instrument [VOPI], a profession-wide, nationally normed comprehension exercise;

{c) advanced VOPI speaking and listening test.

5.Time Frame (When will the Assessment be done?)

(a) Each quarter, including final examination at the end of the course

(b) Sophomore/Junior year: French 295/Spanish 395 in connection with the respective advanced composition courses (French 310, Spanish 302)

{c) Senior year: French 395 and Spanish 496 are the senior assessment courses which students must take prior to completing the last twelve units before graduation.

Comment: Makes excellent use of "formative" assessment in each core course, where each course have agreed upon content and standards. Builds assessment and diagnosis in to mid-point of major and requires final ("summative") assessment (and remediation where necessary) prior to final term of study.

6. Who will do the assessment, collect and analyze the data?

Individual instructors routinely assess students on a quarterly basis. The instructors of French 310 and Spanish 302 are responsible for administering the speaking/listening section of the junior assessment. The French and Spanish area coordinators administer the speaking/listening section of the senior assessment. The coordinators are in charge of collecting and analyzing the data, while the department chair informs students of both their junior and senior assessment results.

Comment: Work is spread between faculty at various stages of the program. Clear responsibility is assigned.

7. Type of Feedback (Data)

The department chair informs students of their assessment result. While students are considered ultimately responsible for their own educational decisions, each one is assigned a faculty advisor who suggest appropriate remediation and guidance based on individual assessment results.

Comment: Provides direct feedback to individual students who must review and master material/skills required. Allows students to repair their weakness in a timely fashion.

8.How will data be used to improve program or revise curriculum?

Results will be used to determine the strengths and weakness in the area of listening/speaking and to identify needs for changes in or improvements in the curriculum.

Comment: Somewhat vague, but, with regular collection and publication of results of assessments each term and with a small faculty that is intimately involved in the progress of their students, should provide clear choices and constant awareness of where courses and teaching methods need change.

III. MULTIPLE MEASURES OF PROFICIENCY USING PERFORMANCE-BASED STANDARD

BA in Music

1. Goal.

The unified major goal for core curriculum each music major is the acquisition of the understanding and development of musicianship.

Comment: Uses one broad overarching Goal for content of Core of major. Additional goals and objectives are later established for each of the three options (Music Education, Specialized/pre-professional Studies, and General Music Studies). Concept of "musicianship" is well understood by those in the field but may be confusing to others without further definition.

2. Objectives:

All student in the department of Music are required to demonstrate a. an acceptable level of musical and artistic performance skills in vocal and/or instrumental performance, b. an understanding of the nature and expressiveness of music, c. an understanding of various musical genres in the evolution of historical music eras, d. an ability to accurately analyze music scores from various historical music eras, e. an understanding of various music of other cultures

Comment: Rather vague to non-musicians but further refinement and definition occur under specific outcomes criteria.

3. Outcomes Criteria.

All students will demonstrate competencies

a. in performance of composition through quarterly "jury" exam before the entire music faculty,

b. in performance or composition at quarterly public recitals,

c. In an understanding of music historical perspectives and performance practices in western music and other cultures through lecture courses with class projects, exams and public performance,

d. in music history through a Final Assessment

e. For musical synthesis through the successful completion of a Final Project (recital, conducting, original composition.

Comment: level of achievement and standard of competency expected are not specific, relies on use of consensus and involvement of faculty as a whole to developed shared standards for expressive and intangible aesthetic components.

4. Assessment Methods and Measures to be Used

a. for applied studio instruction the designated music instructor will monitor and record student t technical progress on a weekly basis and the entire music faculty will provide written critique for each students at a quarterly jury examination,

b. successful passing of the final written exams in specific courses,

c. successful passing of the final performance exams in specific courses,

d. The Final Assessment will be scored objectively by written exam and subjectively before a faculty committee.

5. Time Frame

a. on a weekly basis in applied individual lesson and at end of each quarter at the jury exam

b. For traditional lecture courses at the end of each quarter

c. the Final Assessment will be evaluated by a faculty committee and a percentage rating of scores will be provided.

Comment: Relies heavily on individual instructors, but involves whole faculty for portion of ongoing and summative evaluation. The "Final Assessment" needs to be further described as to content and format.

IV. COURSE-BASED ASSESSMENT BY INDIVIDUAL FACULTY

(Problematic due to consistency of standards from section to section and year to year. More department involvement in assessment and analysis of results needed.)

BA in Art

1. Goal.

Graduating students will possess the ability to "read" the non-verbal language of art and design, and to express their reading in clear written and verb forms.

Comment: Seems to be a good approach for art history and courses in principles/theory of art.

2. Objective.

Students will be able to demonstrate an understanding of basic design principles, concepts, media, and formats in clear written and verbal form.

Comment: Not stated in measurable, operational terms. Seems to duplicate or expand on the goal but does not clearly state the intended outcomes.

3. Outcomes criteria.

Students will demonstrate their knowledge by 1) studying objects in a museum collection and writing one or more formal analysis paper of passing grade. 2) by writing critiques and exhibit reviews, as well as essay style examinations relating to design principles, in a variety of studio and foundation courses. 3) by clearly verbalizing their understanding of design principles, concepts, media and formats during critiques held in all foundation and studio classes.

Comment: These "criteria" are actually the ways students will reveal their achievement, but the criteria for measuring the expected results are not stated. By "Studying objects", writing papers, and "clearly verbalizing" students will have the chance to demonstrate their competency level, but the level expected is not explicit. When students have done these things, how will the department know or be able to discriminate levels of accomplishment?

4. Assessment Methods and Measures to be used.

1) For papers in Art 210ABC, students receive a detailed formal-analysis assignment and write their essays as homework. Instructor reads essays, grading and commenting on accuracy of perception, clear articulation, utilization of the vocabulary of formal analysis, grammar, spelling, organization, and essay structure. 2) Studio and foundation teachers assess written critiques, exhibit reviews, and essay examinations by grading with criteria above. 3) Studio and foundation faculty evaluates student ability to verbally synthesize material and this ability is reflected in grades for projects and final grades for courses.

Comment: States methods and occasions for evaluation but still sets no real criteria or level of performance expected. How will teachers assess critiques, reviews, and essays?

5. Time frame when assessment will be done.

1) mid-quarter during 210ABC sequence; 2) Ongoing during each quarter of enrollment in foundation or studio courses.

Comment: Relies heavily on course grades and role of individual instructor. Calls for no "summative" assessment at end of degree.

6. Who will do the assessment, collect and analyze data?

Comment: Lacks consistency of criteria and standards from section to section and instructor to instructor. Provides no real mechanism for collecting data on student performance level over time.

7. Type of Feedback (Data).

Feedback to the art department includes formal analysis paper grades, together with records of written faculty comments; grades from written critiques, reviews, as well as from oral critiques, together with faculty comments.

Comment: Needs more detail of how data will be collected, summarized, and interpreted. Seems like a massive record-keeping problem.

8. How will data be used to improve program or revise curriculum?

Data fed back to our department will be used to identify need for changes in the curriculum in order to improve students' ability to articulate understanding of design principles in clear written and verbal forms.

Comment: Seems unlikely that data can be reported and used to improve program when no method of collecting it or analyzing results over time is described in the methodology used. Sampling techniques might be developed. Stated criteria or scaling techniques need to be included if this course-based assessment is implemented

V. COURSE-BASED ASSESSMENT USING "EMBEDDED" QUESTIONS.

BA in Mathematics
(part of plan only)

Goal 2.

To teach advanced mathematical concepts and analytical skills, to stimulate curiosity, encourage persistence, and develop mathematical maturity. We will thereby provide mathematics majors with a strong foundation, suitable for further study and for a career in mathematics or related field.

Comment: Focuses primarily on what department and program will provide rather than what they wish students to achieve or know or accomplish.

Objective 2.

Students can analyze problems using mathematical definitions and criteria.

Outcomes Criteria:

Students must demonstrate knowledge of mathematical definitions associated with such mathematical concepts as groups, rings, homomorphisms, automorphisms, open and closed sets, accumulation points, theories of differentiation, continuity and the theory of differentiation. They must be able to identify relevant definitions, and use them in the logical construction of a valid solution.

Comment: A detailed set to topical expectations is provided but level of achievement is left to the assessment procedures below.

Assessment Methods and Measures:

The assessment procedure for undergraduates will be based primarily on the principle of "embedded questions" in the final examinations in a number of core courses. Designated departmental committees will draw up a list of at least 10 problems for each of these courses; at least one of these questions will be included in the final of the corresponding course. The questions will be chosen so that their solutions reflect the student's achievements in the appropriate objectives. In order to ensure consistency of grading, the committee will predetermine a detailed grading scheme for each of the questions in the list. The Scores from this grading will be kept in departmental database.

Comment: Provides consistency in courses without dictating all of the content, procedures or grades for individual instructor.

Time Frame.

Each student will be assessed in the appropriate courses during the quarter of his or her enrollment in that course.

Comment: Excellent formative assessment. Summative assessment procedures not developed. Use of senior honors projects and colloquia presentations suggested elsewhere in document.

Who will do the assessment?

Class instructors, according to departmentally approved grading schemes.

Type of Feedback:

The raw scores and summary statistics such as mean, median and standard deviations will be collected regularly and reviewed periodically.

Comment: With such reliance on individual faculty including key assessment measures in their courses, more detail on how department will collect and analyze finding would be helpful.