|
| Academic
Affairs
OUTCOMES ASSESSMENT FORUM Fall 1998 Welcome to a new academic year and a new publication. The University Outcomes Assessment Committee is attempting to communicate more directly and effectively with faculty and programs. This first issue will focus on review of current issues and past progress. We welcome your comments, suggestion for topics, submission of articles describing your approach and experiences with assessment, and ideas for discussion. 1998-99 University Outcomes Assessment Committee representatives: Robert
Wilson and Otto Chang , School of Business and Public Administration
Why Aren't Course Grades Enough? The Outcomes Assessment committee still is continually asked why course grades no longer are considered sufficient evaluation of student progress. "Don't grades have any meaning? " Well, of course, classroom grades are useful assessment tools, but the whole thrust of the assessment movement is to provide additional, more comprehensive, and more long-term evidence of student achievement. Why? First, course grades represent evaluation of limited objectives that often are not related to program objectives. At best, grades are a type of "formative" assessment. There is too much inconsistency from section-to-section and term-to-term for grades to be a totally valid overall program assessment. Secondly, there is an inherent conflict of interest when the instructor is the only evaluator of whether the students have met program objectives. Third, we really need to know more about the long-term learning and accomplishments of our graduates. What do they know and what can they do at the end of the program? Finding out about their perceptions, attitudes and skills five years after graduation, on the job, in graduate school, or throughout the student's life is more difficult, but an area we are increasingly asked to provide information about. The university assessment committee encourages departments to use classroom assessment measures and to collect data over time from key courses. We applaud the inclusion of this information on "formative" assessment in their overall assessment of the progress of their majors in meeting the stated goals and objectives of their program. Indeed, some programs have attempted to build their assessment plan around having common outcome expectations for core courses. This is a very intrusive, often difficult, and time-consuming approach, however. The Assessment Committee will not accept assessment plans which rely solely on the course instructor to set the criteria, collect and evaluate the data. Moreover, we need and expect some form of "summative" assessment to be done at the end of the program as well. The best assessment plans have multiple points and methods of assessment of student progress and accomplishment. GUIDELINES, FORMATS AND CRITERIA FOR REVIEW OF ASSESSMENT PLANS The Outcomes Assessment Committee has established very firm expectations for what is to be included in program assessment plans. Each plan needs to have a Mission statement, Goals, and Objectives which are directly related back to the goals. These have been distributed to department chairs several times, but we would like to make sure that all faculty are aware of them. Here are the criteria used by the university committee in reviewing "MGOs": Criteria for Review of Mission, Goals and Objectives Statements 1. Is document clearly written in terms that can be understood by students, faculty outside the disciplines, and public? 2. Do the objectives relate back to the goals? 3. Are the objectives concrete and specific enough to be measured and evaluated? Do they lend themselves to assessment of tangible student learning outcomes? 4. Are there enough objectives to evaluate the entire degree program? Are there too many to measure? Is more focus or detail needed? 5. Are there objectives for each subprograms (options, specializations, tracks, emphases, etc)? 6. Are undergraduate and graduate goals and objectives delineated separately? 7. Do the goals and objectives reflect the uniqueness or special characteristics or strengths of the program? 8. Overall, is the document: __
Excellent, (ready to go on to development of assessment plan, criteria
and methods) We also have a format which must be followed for the plan, criteria and techniques of assessment: Required Components of All Program Assessment Plans: Mission
Statement Goals for program (and each sub-program) While we have attempted to allow programs the flexibility to devise their own MGOs and methods of assessment, the committee has become more and more insistent that the format specified be followed—not only for consistency, but because we have found the format has an inherent logic and pattern that guides the program in developing a clear, organized, and concrete set of assessment measures. Here are the criteria the committee has developed for our discussion in reviewing and approving assessment plans submitted. Criteria for Review of Program Assessment Plans 1.
Are Goals and Objectives clearly labeled and listed? Overall
rating of plan These above guidelines, formats and criteria were announced previously and have been in place since 1996. They are still in effect. Some Broader Aspects of Assessment When we began the assessment process, many were concerned that programs would develop only content oriented goals or employ narrow standardized exams that miss the heart of the curriculum and true meaning of the university experience. Would departments develop simple, easily met objectives and then "teach to the test?" At our February 1998 Assessment Workshop, programs had the chance to disseminate their assessment plans and to share experiences and ideas. Assessment Committee member Dr. Robert London of the School of Education reviewed the assessment plans and documents that programs have submitted and found that many programs were taking the "high road" and had developed very noble and lofty goals and objectives for their students. It was a pleasant surprise to find that so many programs had set themselves such difficult and laudable tasks. Here are the kinds of things that programs are attempting to provide and measure in their students. Broad Types of Goals for Assessment Used at CSUSB Demonstration of : 1.
Appreciation of the diversity of cultures and intellectual points of
view Specific Assessment Techniques Used at CSUSB Some departments have inquired about what possible types of assessment might be used. As you can see from the listing below, there is no one "best" or most appropriate model. The techniques employed to measure student achievement on our campus are broad and varied: Senior
Capstone Seminar Comprehensive Course-Based
Proficiency Exams Public
Presentations Multiple
Measures of Proficiency Using Performance-based Standards Course-based
Assessment by Individual Faculty Locally-developed Comprehensive Examination in Senior Year Course-based
Assessment Using "Embedded" questions Internship
or Field Placement Standardized
Exams Theses,
Written Projects Standardization
of Content and/or Course Methods Exit Interview, Student Program Ratings, Surveys of Employers PRINCIPLES OF GOOD PRACTICE FOR ASSESSING STUDENT LEARNING These principles were developed under the auspices of the AAHE Assessment Forum with support from the Fund for the Improvement of Postsecondary Education with additional support for publication and dissemination from the Exxon Education Foundation. Copies may be made without restriction. The authors are Alexander W. Astin, Trudy W. Banta, K. Patricia Cross, Elaine El-Khawas, Peter T. Ewell, Pat Hutchings, Theodore J. Marchese, Kay M. McClenney, Marcia Mentkowski, Margaret A. Miller, E. Thomas Moran, and Barbara D. Wright. 1. The assessment of student learning begins with educational values. Assessment is not an end in itself but a vehicle for educational improvement. Its effective practice, then, begins with and enacts a vision of the kinds of learning we most value for students and strive to help them achieve. Educational values should drive not only what we choose to assess but also how we do so. Where questions about educational mission and values are skipped over, assessment threatens to be an exercise in measuring what's easy, rather than a process of improving what we really care about. 2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Learning is a complex process. It entails not only what students know but what they can do with what they know; it involves not only knowledge and abilities but values, attitudes, and habits of mind that affect both academic success and performance beyond the classroom. Assessment should reflect these understandings by employing a diverse array of methods, including those that call for actual performance, using them over time so as to reveal change, growth, and increasing degrees of integration. Such an approach aims for a more complete and accurate picture of learning, and therefore firmer bases for improving our students' educational experience. 3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. It entails comparing educational performance with educational purposes and expectations—these derived from the institution's mission, from faculty intentions in program and course design, and from knowledge of students' own goals. Where program purposes lack specificity or agreement, assessment as a process pushes a campus toward clarity about where to aim and what standards to apply; assessment also prompts attention to where and how program goals will be taught and learned. Clear, shared, implementable goals are the cornerstone for assessment that is focused and useful. 4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Information about outcomes is of high importance; where students "end up" matters greatly. But to improve outcomes, we need to know about student experience along the way—about the curricula, teaching, and kind of student effort that lead to particular outcomes. Assessment can help us understand which students learn best under what conditions; with such knowledge comes the capacity to improve the whole of their learning. 5. Assessment works best when it is ongoing, not episodic. Assessment is a process whose power is cumulative. Though isolated, "one-shot" assessment can be better than none, improvement over time is best fostered when assessment entails a linked series of cohorts of students; it may mean collecting the same examples of student performance or using the same instrument semester after semester. The point is to monitor progress toward intended goals in a spirit of continuous improvement. Along the way, the assessment process itself should be evaluated and refined in light of emerging insights. 6. Assessment fosters wider improvement when representatives from across the educational community are involved. Student learning is a campus-wide responsibility, and assessment is a way of enacting that responsibility. Thus, while assessment efforts may start small, the aim over time is to involve people from across the educational community. Faculty play an especially important role, but assessment's questions can't be fully addressed without participation by student-affairs educators, librarians, administrators, and students. Assessment may also involve individuals from beyond the campus (alumni/ae, trustees, employers) whose experience can enrich the sense of appropriate aims and standards for learning. Thus understood, assessment is not a task for small groups of experts but a collaborative activity; its aim is wider, better-informed attention to student learning by all parties with a stake in its improvement. 7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about. Assessment recognizes the value of information in the process of improvement. But to be useful, information must be connected to issues or questions that people really care about. This implies assessment approaches that produce evidence that relevant parties will find credible, suggestive, and applicable to decisions that need to be made. It means thinking in advance about how the information will be used, and by whom. The point of assessment is not to gather data and return "results"; it is a process that starts with the questions of decision-makers, that involves them in the gathering and interpreting of data, and that informs and helps guide continuous improvement. 8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. Assessment alone changes little. Its greatest contribution comes on campuses where the quality of teaching and learning is visibly valued and worked at. On such campuses, the push to improve educational performance is a visible and primary goal of leadership; improving the quality of undergraduate education is central to the institution's planning, budgeting, and personnel decisions. On such campuses, information about learning outcomes is seen as an integral part of decision making, and avidly sought. 9. Through assessment, educators meet responsibilities to students and to the public. There is a compelling public stake in education. As educators, we have a responsibility to the publics that support or depend on us to provide information about the ways in which our students meet goals and expectations. But that responsibility goes beyond the reporting of such information; our deeper obligation—to ourselves, our students, and society—is to improve. Those to whom educators are accountable have a corresponding obligation to support such attempts at improvement. We're not in this alone!! Outcomes Assessment is a national movement. Here are some resources and useful sites to explore: Some great web sites with links to lots of other pertinent assessment sites: AAHE
Assessment Forum: http://www.aahe.org/
Samples homepages from other campuses http://www.ohiou.edu/~insres/assessments/index.html
Also, look for the new CSUSB Assessment Homepage link at the top of the Academic Affairs Web Site: http://academic-affairs.csusb.edu
California State University, San Bernardino
contact name and email/date of update
|
