Presentation:

Professor Owen opened the session with University of North Carolina at Chapel Hill’s history of examining their general education curriculum. The curriculum established in 1980 was reviewed periodically between 1982 and 1994 and then comprehensively examined in 1995 as a part of reaccredidation. Problems emerging from this 1995 examination were methodologically explored by an Intellectual Climate Task Force formed in 1998 to identify ways to enhance the academic experience. In 1999 the Curriculum Revision Steering Committee took advantage of feedback gathered during these prior studies to outline a strategy for developing a new general education curriculum. With the guiding question, “What does an educated person in the 21st century need to know?,” the steering committee conducted focus groups, forums, and surveys with hundreds of faculty, students, and staff.  The results of these assessments revealed the following:  considerable slippage had occurred in alignment of current course objectives with original curricular goals; requirements included little emphasis on globalization and other important issues that had emerged in the last decade; and there was a need to enhance students’ ability to make connections between disciplinary knowledge and in- and out-of-class learning experiences. The curricular revision process was divided into three phases, curriculum development (2001-2004), preparation (2004-2005), and implementation (2006). The revision worked alongside a process that led to development of assessment at the undergraduate program level.

The new general education curriculum, named the “Making Connections,” was based on two primary goal statements:  (1) the University of North Carolina at Chapel Hill should strive to “cultivate the skills, knowledge, values, and habits that will allow graduates to lead personally enriching and socially responsible lives as effective citizens of rapidly changing, richly diverse, and increasingly interconnected local, national, and worldwide communities”; and (2) the undergraduate experience should “foster in Carolina graduates the curiosity, initiative, integrity, and adaptability requisite for success in the complex, demanding environment of the twenty-first century world.” Based on these goals, the “making connections” curriculum seeks to provide students with:

• Foundations: The fundamental skills that facilitate future learning;
• Approaches: Broad experience with methods and results of the most widely employed approaches to knowledge;
• Connections: A sense of how to integrate these approaches to knowledge in a way that crosses traditional disciplinary and spatial boundaries;
• Academic Major: A thorough grounding in one particular subject.

An assessment plan was developed based on the student learning outcomes articulated in the description of the new curriculum.  Various evaluation methods were proposed and are being implemented in phases, including a longitudinal study of the first cohort of students who began their studies in fall 2006.  Consistent with the theme of the curriculum, the assessment plan focuses on attempts to measure students’ ability to “make connections” as a result of their educational experiences under the new curriculum.  For example, results from the National Survey of Student Engagement indicated that the 2006 cohort was more likely than the 2004 cohort to “put together ideas or concepts from different courses when completing assignments or during class discussion.”  Intensive individual interviews also confirmed increased levels of academic engagement occurring as a result of the new curriculum’s emphasis on interdisciplinary study and experiential learning.  At the end of their sophomore year, students progressing under the new curriculum described enthusiasm over being able to pursue their academic interests from the perspectives of a variety of disciplines – many of which they had not originally considered as having a connection to their initial area of interest.  In particular, participation in a first-year seminar was described as a catalyst for further academic exploration in and out of the classroom and created a distinct preference active learning activities over traditional lecture course formats.

Professor Owen presented other examples of how results from recent assessment activities have been used to monitor achievement of the new general education outcomes:

• Changes in the published criteria for courses made as a result of some observed disconnects between actual and intended content;
• Observations of course-taking patterns indicate that students are taking considerably more courses in the sciences, fewer in humanities;
• Increases in international experiences reported by students;
• Original assessment plans have been modified to take advantage of new data collection opportunities and to eliminate assessments (e.g., CLA) that seem to yield results that are not useful.

Discussion:

It was noted that during the 2004-05 academic year, efforts to formalize program-level outcomes assessment campus-wide -- initially driven by the need to address accreditation and accountability requirements --  were also quite challenging to plan, implement, and manage. The presenters were asked how they organized implementation of systematic assessment at the program level.  At the center of their strategy was collaboration among the Office of Institutional Research and Assessment, the Center for Teaching and Learning, well–respected and admired faculty, associate deans, and an assessment advisory committee including representation from the College of Arts and Sciences and the professional schools.  These groups were all involved in training, encouraging, and supporting faculty in understanding the purpose of outcomes assessment; creating instruction manuals, examples, and templates; and clarifying expectations. The assessment advisory committee also advocated for policy statements concerning the value of assessment and additional resources for faculty support and administrative oversight of the process.

The presenters shared that part of their strategy for introducing faculty to outcomes assessment was to bring nationally-known assessment consultants (e.g., Peter Ewell and Trudy Banta) and faculty from neighboring institutions to campus for presentations and informal peer-focused advice.  These outside experts helped analyze the existing curriculum and offered feedback on assessment methods that might be most effective, and in general, helped initiate dialogue about assessment at the University of North Carolina at Chapel Hill. More recently, this conversation has been reinvigorated in response to an opportunity to participate with other institutions in a project funded by the Spencer and Teagle Foundations that focuses on the systematic improvement of undergraduate education.

One conference participant saw a contradiction in creating learning outcomes having to do with “curiosity” and “adaptability” when general education at the campus is structured in a way that “makes” (forces) students to take certain courses, bringing into question the amount of freedom students actually have in completing their general education. Another participant asked about course enrollment patterns within the curriculum and the extent to which students are choosing course on their own.  The presenters indicated that analyses were underway to examine actual course-taking behavior in relation to the stated goals of the curriculum.

Principal Findings:

Professor Owen observed, “It takes a long time to deliberate and reinvent a curriculum. It takes a long time to get buy-in from a university community, from faculty, staff, and students.” At the University of North Carolina at Chapel Hill, the process of revising the general education curriculum began in 1999, reached full implementation in 2006, and involved multiple phases that engaged all university stakeholders. Throughout this period, support for the process and for assessment was built by drawing faculty into the process, by solidifying support among associate deans and other administrators, by creating an Assessment Advisory Committee, and by senior administrators voicing their support.

A phased approach to introducing assessment at the program level enabled progress and decreased resistance. Program-level assessment was introduced after development of the “Making Connection” curriculum that had included extensive campus-wide discussion on the purpose of general education. The academic units were given several months to attend training and to develop an assessment plan, and were encouraged to begin implementing assessment in simple, incremental manner initially.  Faculty were provided with feedback from peers and the Office of Institutional Research and Assessment and advice on experimenting with a wide variety of direct measures of assessment (e.g., creating scoring rubrics to evaluate student work products, etc.), an annual assessment reporting schedule was introduced, and plans to periodically evaluate the effectiveness of the assessment process were developed.

A number of robust methods to assess the new general education curriculum was put in place. The longitudinal study of the 2006 student cohort, for example, was designed to track the academic performance and out-of-class activities of 700 students from their first day to graduation.  Annual analyses of course enrollment patterns, surveys, activity inventories, and intensive interviews are part of this study. Assessment of the “foundations” and “approaches” elements of the new curriculum include: sampling courses mapped to specific requirements and assessing papers and other embedded course assignments using rubrics; adding questions to end-of-course evaluations to determine the extent to which students perceive that they achieved the intended outcomes of the course; and conducting focus groups of advisors and instructors concerning their observations of the obstacles and successes of the curriculum implementation process.

The presenters emphasized that curriculum review and assessment of general education and in the major is a continuous “work-in-progress.”  They report that many great ideas – in terms of the curriculum components as well as assessment strategies – simply do not work as well in actual practice.  While the vast majority of the curriculum implementation and assessment activities have been quite successful, a number of modifications have been made to accommodate unforeseen problems, improve processes, and to capitalize on new opportunities.  Examples include recognizing the financial and logistical burdens of expanding linked or “cluster” courses, rethinking how we might encourage students to participate in data collection efforts as response rates to surveys drop, and finding effective ways of analyzing how students link in-class and out-of-class experiences. 

Recommendations:

• Recognize that the general education curriculum revision process is lengthy and demands involvement from all university stakeholders.
• A comprehensive approach to assessment that includes developing assessment processes at the program level can be effectively implemented using a phased approach which gives faculty a chance to realize the value of the feedback without being overwhelmed by accountability-related work demands from administrators.
• Employ a variety of robust quantitative and qualitative evaluation methods to assess the impact and effectiveness of a new general education curriculum.

Resources/References:

POWERPOINT PRESENTATION
Reforming and Assessing General Education. The Reinvention Center Conference Proceedings 2008. Washington, DC. Bobbi Owen, Senior Associate Dean for Undergraduate Education,
College of Arts and Sciences, and Professor of Dramatic Art; Lynn Williford, Assistant Provost for Institutional Research & Assessment , University of North Carolina at Chapel Hill.