| |
Every few months The Center spotlights a topic of
significance to research university faculty and administrators. Its approach
is through Thoughts and Models. The Thought consists of a short essay on the
particular topic being highlighted. The Models represent different campus
approaches to the topic.
In recent years, empirical assessment methods
increasingly have been applied to higher education. Under the broad rubric
of assessment, faculty and administrators at research universities are
thinking about ways to design studies and gather evidence to measure the
effectiveness of their programs and the extent to which their curriculum and
pedagogy are meeting their goals and achieving desired learning outcomes.
Further, they are experimenting with a variety of approaches and tools
ranging from on-line surveys of targeted populations, to small-scale,
informal assessments of individual class sessions, to multi-year
e-portfolios to measure students’ progress, to standardized testing of all
students. The goal is to build a culture of assessment that informs their
undergraduate education.
This
Spotlight consists of a Thought and four Models of assessment presented at
Reinvention Center meetings this past year. The Thought, written by Robert
Thompson, Jr., Professor of Psychology and Dean of Trinity College
at Duke University, calls upon research universities to become proactive in
shaping the national conversation on assessment; instead of allowing
external forces to control the conversation and demand “accountability”
based on criteria that may be inappropriate or inapplicable to higher
education, universities should reposition the discussion of assessment in
terms of their own academic values. The Models that follow illustrate the
range of assessment activities being implemented and the ways in which the
findings can be used to improve teaching and learning. The University of
Nebraska-Lincoln has undertaken a centralized, systematic assessment of
student learning across schools and colleges. Its centralized approach has
enabled the University to gather data that are being used to inform major
curricular changes. In contrast, North Carolina State University has
decentralized its assessment activities and given responsibility for
creating an assessment plan to departments which devise their plans based on
faculty expertise and disciplinary requirements. In an altogether different
vein, at the University of Wyoming, university faculty and high school
teachers are engaged in a joint effort to gather data that will help the
University to understand the educational experiences of Wyoming high school
students and design appropriate transitions from high school to university
education. The final Model represents a collaboration among the National
Council of Teachers of English, the Conference on College Composition and
Communication and the Council of Writing Program Administrators. These
organizations are working together to create models for assessing student
writing, the teaching of writing and writing programs.
From Assessment to Accountability: Reframing the Conversation
Robert Thompson, Jr.,
Dean of Trinity College, Duke University
“And if assessment becomes synonymous with standardized
testing, what will happen to assessment undertaken for the
purpose of guiding improvement in instruction, curricula and
student services?” (Banta, 12)
Many in higher education are uneasy about the relationship
between assessment and accountability. We worry about the
impact of increasing expectations for accountability, about
whether these expectations will alter the educational
process and about how assessment data will be used. Some
have tried to differentiate assessment for accountability
from assessment for improvement. Carol Schneider, for
example, captures this difference in her article for the
Peer Review when she states: “Assessment can and
should be designed to deepen and strengthen student
learning, not just to document it” (3). If we accept this
difference in terms as a signal of the different purposes of
assessment, it would seem that assessment and accountability
are not mutually exclusive and could, in fact, be
complimentary. But a part of our uneasiness stems from the
more fundamental problem of how accountability is being
framed, and it is this framing that bothers and threatens
us. An alternative framing of accountability would serve to
enhance the quality of undergraduate teaching and learning
and help us to identify where change in our practices is
warranted. Let me first describe the way accountability is
currently framed, and then demonstrate how a reframing would
allow a more productive conversation.
Currently,
conversations about assessment and accountability are framed
by the following recommendations of the Commission on Higher
Education (Eaton, 2007):
· Provide
more evidence of student achievement and institutional
performance and make this evidence primary when judging
academic quality
-
Make information easily understandable and readily
accessible to the public
-
Develop various means to compare institutions regarding
their success in student achievement and institutional
performance
-
Establish threshold standards for collegiate learning
The first two
recommendations evoke little disagreement; we have no
trouble agreeing that providing evidence of student
achievement and institutional performance to the public is a
good idea and that using such evidence to judge academic
quality is appropriate. Similarly, we have no trouble with
the idea of making information easily understandable and
readily accessible. But the last two recommendations evoke
concern and, it is agued, constitute a misplaced emphasis on
an external locus of control and a market place
perspective. When accountability is considered from the
market place perspective and coupled with the threat of an
external locus of control that would define standards and
judge academic quality:
·
Education is seen as a product that is
provided
·
The focus is on student achievement and
institutional performance outcomes, and
·
The approach is comparative and competitive
among institutions
In contrast,
when we take an academic view of accountability, with
an internal locus of control with regard to defining
standards and judging academic quality:
-
Education is seen as a process of
enabling growth
-
The focus is on instituting a culture
of experimentation and evidence with regard to
teaching and learning, and
-
The approach is evaluative within
institutions and collaborative among institutions in
pursuit of best practices.
Once the marketplace perspective of accountability is
adopted, even the idea that colleges and universities should
make information more readily available becomes distorted to
that market place perspective. For example, Kevin Carey
demonstrates such a shift when he states that “students
choosing colleges currently have little or no information
about which institutions actually provide the best
education” (Change, 26). The difficulty is with the
expectation that there is “the best education” as opposed to
the perspective that educational experiences vary along a
number of dimensions and that what should be sought is the
optimal match between the education an institution can
provide and a particular student’s interests and talents.
By
shifting to the academic perspective of accountability,
the values of scholarship rather than those of the
marketplace will be at work. In reframing accountability in
this way, institutions of higher education will hold
themselves accountable to their multiple constituencies
by continuously engaging in systematic, iterative processes
to improve the quality of teaching and learning. But to
attain an internal locus of control with regard to
accountability rather than succumb to the threat of external
controls, we must be willing to work through institutional
and faculty resistance and establish ”a willingness to
reexamine familiar practices and search for new methods that
could serve the purpose better” (Bok, 2006, 313). A research
university must, in effect, become a learning
organization that engages in a continuous and systematic
process of experimentation with innovative teaching and
learning approaches, evaluation of attainments, and revision
of approaches. This can be accomplished by committing to the
self-evaluative and self-correcting process with regard to
teaching and learning that characterize universities’
approach to scholarship.
References
Banta, T. (2007). Can Assessment for
Accountability Complement Assessment for Improvement?
Peer Review. 9 (2), 9-12.
Bok, D. (2006) Our Underachieving Colleges.
Princeton: Princeton University Press.
Carey, K. (2007) Truth without action: The myth of
higher-education accountability. Change, 39 (5),
24-29.
Eaton, J. (2007). Institutions, accreditation, and the
Federal government: Redefining their appropriate
relationship. Change, 39 (5), 16-23.
Schneider, C.G. (2007) From the President. Peer Review,
9 (2), 3 |
|
| |
The following four models provide examples of innovative programs with a broad range of
assessment experiences. Each model is highly flexible, applicable to range of disciplines, and reproducible on many scales. For additional examples and information, please visit the Undergraduate Research Opportunities section of our Resources page.
University of
Nebraska-Lincoln
Rita Kean, Dean of Undergraduate
Studies |
|
Student Learning Outcomes:
From Assessment to Accountability
In
today’s milieu of educational accountability, the research
university must clearly demonstrate to both its internal and
external constituencies that the discovery, application and
efficacy of knowledge are the foundation for the undergraduate
experience. If we hold true to the belief that students should
be active participants with faculty in the discovery and
application of knowledge, then we should be able to assess that
participation, discovery and application in a way that makes
clear to our constituencies the value of undergraduate
experience at a research university. Institutions of higher
education are responsible for insuring the merits of the
undergraduate experience. The University of Nebraska–Lincoln (UNL)
instituted a systematic assessment of student learning outcomes
at the program and institutional levels in 1996.
This attention to assessment at
both the program and institutional levels has been essential to
creating a culture of assessment as a process for improvement as
well as for accountability. As faculty at UNL have become
actively involved in formulating and evaluating their program
assessment plans, assessment has become embedded throughout the
undergraduate curriculum in all eight of the University’s
undergraduate colleges. The numbers of units participating in
assessment of student learning outcomes at the program level and
the use of assessment as a means of implementing program
improvements have increased steadily over the past twelve years.
Results of assessment measures have highlighted a broad array of
insights into student learning, including the continuous
improvement of learning outcomes, coherence of the curriculum,
advising, co-curricular experiences, student-faculty interaction
and recruitment/ retention. For more information on these
aspects of assessment at UNL, please see
http://www.unl.edu/ous/faculty_resources/assessment.shtml
The
key objectives underlying the UNL assessment process are
threefold:
-
Determine the desired student learning outcome(s):
What do we want students to know?
2.
Identify best measures for determining whether
these outcomes have been realized: What
are the intellectual frameworks/underpinnings of the knowledge
base? How we can best deliver them to meet the needs of the
learners (pedagogy)?
-
Use
results to either confirm or improve instructional and
curricular practice: What did we
discover about the learner’s understanding of the content and
delivery (reflection, critical analysis, evaluation) and in
the same context, what did we learn that changed our own
understanding of the content and/or our delivery?
Constructing a
culture of effective assessment is a developmental process. If
the goal of assessment is continuous improvement in the learning
environment, then faculty members need to understand the
benefits of using assessment to improve student learning.
Disciplines approach assessment differently; therefore, respect
needs to be shown for each discipline’s scholarship and
traditions. The School of Biological Sciences, for example,
employs a standard measurement for assessment of their
graduating seniors. The Major Field Test for Biology consists of
150 multiple-choice questions, a number of which are grouped
into sets based on the description of laboratory and field
situations, diagrams or experimental results. Questions within
each of the major areas are designed to test students’
analytical skills. The Department of Art uses senior exhibits
for assessment purposes. Peer review for assessment techniques
and student learning outcomes is beneficial. For many UNL
faculty members, the peer review process has been invigorating
and confirming.
To facilitate
the peer review process, UNL has invested in tools to assist
faculty with their assessment efforts. In 2004, UNL’s College of
Education and Human Sciences and the College of Agricultural
Sciences and Natural Resources established a partnership with
Colorado State University to pilot an on-line assessment
management system designed for program improvement. As of
January 2008, four of the eight UNL undergraduate colleges and
the Division of Student Affairs were employing this system,
referred to as PEARL
http://www.unl.edu/ous/pearl/indepth.shtml.
PEARL allows faculty and staff to take the lead in the
assessment process through participation in a continuous on-line
dialogue between a trained peer reviewer and the faculty/ staff
member responsible for the program’s assessment, increasing
interaction and engagement among faculty/ staff. Information
obtained from the PEARL system has already been used in
academic program reviews as well as in reporting to professional
groups/agencies. For example, the Department of Food
Science and Technology, in their self study for their academic
program review in fall 2007, described their use of the PEARL
system to assist with their assessment of student learning
outcomes, as well as the mechanisms used to assess those
outcomes. The Department’s learning outcomes, opportunities for
student learning, questions of interest, assessment methods,
assessment results and use of the results for improvement were
included in the academic program review.
UNL expects
systems such as PEARL to be key elements of the
assessment of its new general education program,
Achievement Centered Excellence,
which is based upon
student learning outcomes
http://ace.unl.edu.
After attending the 2005 AAC&U Summer Institute on General
Education, UNL’s General Education Planning Team (GEPT)
concluded that a student’s general education experience
should not be viewed as a separate function of his/her
educational program, but, rather should be integrated throughout
the entire educational experience.
The GEPT and
the larger General Education Advisory Committee (GEAC),
approached reform efforts by defining four overall institutional
objectives, each accompanied by a set of measurable student
learning outcomes. Table 1 presents each of these objectives,
followed by an example of a student learning outcome for that
objective. A major strength of a general education program based
on student learning outcomes is that assessment of student work
is accomplished through direct measures. One measure, for
example, might be a sample of student products from ‘certified’
general education courses, along with reflections from the unit
as to how the student work met the stated student learning
outcomes for the course or program.
Table
1 ACE Institutional Objectives with Examples of Student Learning
Outcomes
|
Objective |
Example of Student Learning Outcome |
|
Develop intellectual and practical skills,
including proficiency in written, oral and visual
communication; inquiry techniques, critical and creative
thinking; quantitative applications; information assessment;
teamwork; and problem-solving. |
Write texts,
in various forms, with an identified purpose, that respond
to specific audience needs, incorporate research or existing
knowledge, and use applicable documentation and appropriate
conventions of format and structure.
|
|
Build
knowledge
of
diverse peoples and cultures and of the natural and physical
world through the study of mathematics, sciences and
technologies, histories, humanities, arts, social sciences
and human diversity.
|
Use
scientific knowledge of the natural and physical world to
address problems through inquiry, interpretation, analysis,
and the making of inferences from data, to determine whether
conclusions or solutions are reasonable.
|
|
Exercise
individual and social responsibilities through the study
of ethical principles and reasoning, application of
civic knowledge, interaction with diverse cultures,
and engagement with global issues.
|
Explain
ethical principles, civics, and stewardship, and their
importance to society.
|
|
Integrate
these abilities and capacities, adapting them to new
settings, questions, and responsibilities.
|
Generate a
creative or scholarly product that requires broad knowledge,
appropriate technical proficiency, information collection,
synthesis, interpretation, presentation, and reflection.
|
UNL has been
involved in several efforts to assess the undergraduate
experience at the institutional level. A key tool has been the
National Survey of Student Engagement (NSSSE), administered at
UNL on a triennial basis. UNL participated in the 2005-06
Council of American Education pilot of the Collegiate Learning
Assessment (CLA).
During AY
2006-07, UNL administered the CAAP measurement to a sample of
students through their participation in Parsing the First
Year of College Study, led by Patrick Terezini and Robert
Reason of the Pennsylvania State University Center for Higher
Education Research. Terezini and Reason offer a comprehensive
model and measure of the students’ educational experience as
well as providing data for institutional advancement and
improvement. This study considers the relationships between
inputs (entering student characteristics and experiences),
processes (organizational context and educational environments)
and outputs (learning, development and persistence), thus
offering a more holistic view of our students’ undergraduate
experience.
Presently, the
Office of Institutional Research and Planning is coordinating
UNL’s contribution to the Voluntary System of Accountability (VSA).
UNL believes the assessment of the student learning outcomes
associated with their general education program will provide
them with institutional data as to the efficacy of the program
and its effect on student learning; however, they also realize
the choice of instrument may be determined by the larger
University system rather than the individual institution.
|
North Carolina State University
Alan Dupont, Director of
Assessment |
|
Assessment of Undergraduate
Student Learning at NC State: A Decentralized Approach
Like most large, public,
research-extensive universities, North Carolina State is
decentralized. It has nine colleges offering undergraduate
degrees (a tenth college, the College of Veterinary Medicine,
does not offer undergraduate degrees); they range in size from
the College of Engineering with approximately 5,400
undergraduates (18% of total undergraduate enrollment) to the
College of Design with roughly 470 undergraduates (1.5% of total
undergraduate enrollment). Each college has its own culture that
shapes how things are organized and how things get done. The
roughly 85 departments into which these nine colleges are
organized, in turn, have their own distinctive cultures and
differing degrees of autonomy. Alan Wolfe, in a 1996 article in
The Wilson Quarterly, wrote that “so organized, the
university is not an entity with a common purpose or at least
organized around a self-defining core. It is a set of linked
fiefdoms that find temporary advantage in belonging to a larger
organization.”
It has been noted that this
decentralization and concomitant faculty autonomy have been
integral to the research success of the modern university (see,
for example, the Kellogg Commission report Returning to Our
Roots: Toward a Coherent Campus Culture, 2000). However,
this structure also means that processes or activities mandated
“from above” are often resisted strenuously by the faculty. This
resistance does not apply to basic business functions, of
course, which are generally carried out by non-faculty
employees. A good example of a “mandate from above” is
assessment of student learning outcomes, which is (or should be)
carried out by the faculty. Until 2006, NC State utilized a
centralized approach led by the Committee on Undergraduate
Program Review (CUPR). Programs (typically but not always
synonymous with departments) were given lengthy assessment
guidelines that went into some detail about the format of the
required reports. The communications went from CUPR to the
departments, bypassing the college administration in most cases.
This approach caused many faculty members to see assessment as
an added burden that was disconnected from their day-to-day
work.
In 2004, the most common
question faculty asked about assessment was “What does CUPR
want?” The question was indicative of a “compliance mentality”
that sought to complete the reports with the least amount of
time and effort expended. While some programs and faculty were
authentically engaged in assessment, most program faculty saw
assessment as a fad, an unjustified additional burden, and/or
not worth doing even if they had the time and energy to spare.
To put it bluntly, the centralized approach provided some
extrinsic motivation (though not enough in most cases to
motivate the faculty to authentically engage in the process),
but it did not provide intrinsic motivation. Given that the
purpose of assessment is to improve student learning, it is
critical that any assessment process be primarily developed and
implemented by the faculty as it is the faculty who make the
changes to courses and curricula that result from assessment.
Recognizing this, and
spurred on by increasing calls by the faculty and the department
heads to re-think its top-down approach, NC State switched to a
decentralized process in which every college is responsible for
developing an assessment plan and carrying it out. This approach
gives the colleges considerably more latitude in determining how
to organize and implement assessment than they had under the
centralized system. Specifically, the Associate Dean of the
college who oversees its undergraduate programs now has, as part
of the job, responsibility for ensuring that every program
within the college submits an annual program assessment report.
This decentralized approach has been a success, at least in part
because the faculty members in the colleges know their Associate
Dean and have in many cases known them for years. Perhaps just
as important, the assessment process can be tailored to fit the
unique culture and needs of the individual college. For
instance, the College of Engineering has a specific approach to
student learning outcomes assessment that is driven by its
national accrediting body, ABET. Under the decentralized
approach, the reports that the College prepares for ABET
purposes can be used with only minimal modification to meet the
University’s assessment requirements. .Similarly, the College of
Management can take advantage of its AACSB accreditation process
and the College of Education can align its assessment with its
NCATE accreditation process, as can other colleges and
departments that have special assessment situations. It should
be noted that assessment support is itself decentralized; in
addition to the Office of Assessment in the Division of
Undergraduate Academic Programs, the College of Engineering and
the College of Education have their own assessment professionals
who focus on the specific assessment and accreditation needs of
their respective colleges. The Graduate School, the Division of
Student Affairs, and Distance Education also have assessment
professionals on their staffs. Finally, University Planning and
Analysis carries out some university wide assessment projects
under the direction of an assessment professional on the UPA
staff.
While the decentralized
approach increases the level of control by the faculty over the
process, it does not in and of itself provide any intrinsic
motivation. In an effort to engage faculty, and in conjunction
with the decentralization, the staff of the Office of Assessment
in the Division of Undergraduate Academic Programs has
successfully switched re-cast its role so that instead of being
perceived as “assessment enforcer” its members have become
“assessment consultants.” This change makes talking to program
faculty about the purpose of assessment much easier and staff
members are very pleased with the new approach. There appears to
be an attitude among many more faculty members that assessment
is a useful tool for improving undergraduate student achievement
of specified learning outcomes. The annual program assessment
reports reflect the improved attitude and increased faculty
engagement as they now focus more on questions that program
faculty find compelling than they did in the past, and they
include complex analysis and judgment. At the same time, there
appears to be a corresponding decrease in annual reports that
reflect a “compliance mode.”
There remains a level of
University oversight within this decentralized structure in that
the program reports and the Associate Deans’ summaries of those
reports are made available to the Office of Assessment, whose
staff members review them and provide feedback to the Associate
Deans and to the departments’ curriculum or assessment
committees. The staff members also look for themes or issues
that cross college boundaries. These are brought to the
Associate Deans for their consideration. Finally, the Associate
Deans themselves share the assessment results from their
departments at an annual Council of Associate Deans meeting that
is devoted to assessment. The Office of Assessment staff also
prepares an annual report for the Provost and other interested
parties (primarily the liaison to the University’s regional
accreditation agency) that includes pertinent information needed
by those individuals and offices.
The decentralized approach
to assessment at NC State meshes well with the decentralized
institutional culture, allowing colleges and departments to
fashion assessment plans that work for the program faculty. The
program faculty can address the questions that are of interest
to them and gather information that is meaningful to them. This
approach puts the staff of the Office of Assessment in a
position to be consultants rather than enforcers, which leads to
more productive interactions.
|
University of
Wyoming
Audrey Kleinsasser, Professor and
Director, Wyoming School-University Partnership
Rollin Abernathy, Professor and Associate VP for
Academic Affairs |
|
The University of Wyoming's
Curricular Alignment Initiatives
A land grant institution, the University of Wyoming was
charged by the Wyoming Constitution to provide instruction “as
nearly free as possible” (Wyoming Constitution, Section 16,
1889). The Wyoming Legislature made the ideal of nearly free
postsecondary education a reality in 2006 by enacting
provisions for a $400 million Hathaway Scholarship Endowment
Program. Interest from the endowment is providing merit and
need-based scholarships to graduates of Wyoming high schools
who enroll at the state’s community colleges or the four-year
university, utilizing ACT scores and high school grade point
averages as determinants of the level of scholarship support.
As with many scholarship programs in other states, Wyoming’s
Legislature stipulated that recipients of the Hathaway
Scholarships complete a Success Curriculum beginning in ninth
grade: four years of English, mathematics, and science; three
years of social studies; and two years of a sequenced foreign
language. The Success Curriculum will be fully phased in by
2011. Concurrent with the implementation of the Wyoming
Hathaway Scholarship Program, the University partnered with
the Community College Commission, the Wyoming Department of
Education, and business to craft the Wyoming P-16 Education
Council. The goals of the Council include effective
implementation of the Success Curriculum and data systems that
allow statewide assessment of effectiveness for improvement.
Along with this extraordinary opportunity come enormous
challenges. During the first year of full implementation,
2006-2007, approximately 25% of the Hathaway recipients failed
to maintain a high enough GPA to keep their scholarships. The
25% failure rate indicates significant alignment issues
between secondary and postsecondary schooling in Wyoming,
especially Grades 11-14.
To address curricular alignment across grades 11-14, reduce
remedial education, and close gaps apparent in Wyoming’s P-16
education system, UW is leading an innovative professional
development model piloted by the Wyoming School-University
Partnership. The Partnership model, which builds on a
successful UW 2003-2006 FIPSE-funded initiative, brings
together a multi-level cohort of high school, community
college, and university instructors who teach students in
Grades 11-14. This four-year grade span represents a series of
necessary academic, social, and emotional transitions that are
critical to limiting remedial education, completing a
postsecondary degree, and thriving as a lifelong learner. The
model is applicable to virtually all secondary through
postsecondary alignment initiatives in the nation
Wyoming Life Sciences Summits
In February 2006, over fifty life sciences teachers
representing the University of Wyoming and statewide high
schools and community colleges convened in a day-long science
summit organized by the Wyoming School-University Partnership.
Summit planners asked each participant to bring multiple
copies of student work representing high, medium, and low
quality. During the meeting, this work was examined in
multi-level, small groups using an assessment protocol
designed and guided by a skilled facilitator. For all of the
participants, including the planners, the event was unique.
None had ever met in a multi-level cohort to examine student
work and the learning it revealed. Summit evaluations
confirmed that participants valued the opportunity for candid
peer discussion about assignments and student learning
outcomes. They returned to their classrooms with immediate
plans for changing their practices. Secondary teachers
reported holding their students to higher expectations.
University teachers created course stewardship committees to
align curriculum and evaluate assessment practices. In effect,
this first summit made the barrage of national reports
concerning the secondary-to-postsecondary transitions real,
local, and high priority. It revealed that such multi-level
discussions are foundational to major changes that must occur
in Grades 11-14.
By February 2008, six follow-up life sciences summits had
occurred in Wyoming, involving over 200 participants who came
to some deeper insights of student learning across levels.
They identified several valuable features they wanted to
replicate. First, student work revealed student understandings
and misconceptions as well as a range of performances. Second,
by focusing on genuine student work—tests, quizzes, lab
reports, homework assignments—the teachers came to better
understand particular grade level challenges. The vocabulary
of science, for example, is especially challenging for eighth
and ninth grade students, with more new words introduced in a
year than students would learn in a modern language course
over a similar time period. Third, and most consequential,
teachers came to see that even across levels there were more
similarities than differences in teachers’ concerns about
student achievement. Teachers at all levels became alert to
students’ failure to critically read assignments of increasing
length and complexity, especially as students in high school
bridged college-level work. Uniformly, teachers saw problems
with writing and the necessity of multiple drafts to produce
final texts. For the life sciences faculty, a significant
issue became student difficulty at all levels with
interpreting and analyzing quantitative data represented in
graphs and charts.
An early result of the first few summits was the introduction
of a mathematics achievement gate for UW’s Life Sciences
Program. The program director and faculty have documented that
a mathematics ACT score of 21 predicts success in the
University of Wyoming’s General Biology course as measured by
a grade of C or better. The course is pivotal since students
in many majors across all of the university’s six
undergraduate colleges must successfully complete it. By
establishing several ways for students to meet the equivalent
of an ACT score of 21 before taking the course, the program
director has significantly decreased the number of students
who must retake the General Biology
course.
The requirement became the source of much discussion at four
2007-2008 life sciences summits, in particular the conclusion
that the increasingly synthetic nature of the life sciences
makes the subject harder for students, especially in its
quantitative reasoning demands. The teachers also discussed
the reality that most students took their first and often only
high school life sciences course in eighth or ninth grade.
Then, four or five years later, students entered a
college-level course with heavy reading and lab requirements,
a faster pace, and work that required students to make
significant connections between math and science. For the
postsecondary teachers, that four- to five-year gap explained
many of the difficulties first-year students exhibit.
Writing Colloquium
Building on the
success and momentum of the life sciences summits, Wyoming
School-University Partnership planners adapted the model for a
statewide Teaching Writing in Wyoming Colloquium in April
2008. Key elements of the two-day meeting for more than fifty
participants included multi-level work groups involving junior
high, senior high, community college, and university
instructors, the examination of student writing samples and
scoring rubrics, a discussion of the first year
university-level writing course, and a critical reading
workshop.
Similar to teachers attending the life sciences summits,
writing colloquium participants found the meeting unique and
valued the multi-level cohort experience, especially peer
discussion. They identified significant gaps in expectations
for homework, and they began to understand that
outside-of-class reading and writing assignments are key
features of successful postsecondary student achievement.
Participants also drafted a document that compared eleven
instructional features of secondary and postsecondary writing
courses. The document is remarkable in that it is created and
co-owned by a broad-based, multi-level group of teachers.
The Power of this Model
The two statewide projects described above, one in life
sciences and one in writing, illustrate the power of a
multi-level cohort professional development model in a core
content area. The model breaks down the barrier of dedicated
secondary and research university teachers working together to
strengthen the quality of secondary instruction and help
students successfully negotiate transitions to the next level
of study. At the same time, the model is non-threatening,
featuring low risk strategies that are also frequent and
interactive. The model honors the hard and necessary
intellectual work of effective teaching—deliberations around
student learning and academic achievement that are
longitudinal.
In the Wyoming work to date, the participants have been
genuinely curious and respectful about each other’s successes
and challenges. As a result, the meetings have not devolved
into mutual blaming or debilitating cynicism. Trust grew, in
fact, with participants eager to meet again in a similar
configuration to examine student work. Most important, we have
growing evidence in the UW Life Sciences Program that the
model supports a more sophisticated way of solving secondary
to postsecondary transition problems and reflects deeper
student learning as documented by higher achievement and
reduced remedial education.
For more information
Audrey Kleinsasser, Director, Wyoming School-University
Partnership,
dakota@uwyo.edu
Mark Lyford, Director, University of Wyoming Life Sciences
Program,
mahler@uwyo.edu
April Heaney, Director, University of Wyoming LeaRN Program,
aprilh@uwyo.edu |
National Council of Teachers of English, Conference on College
Composition and Communication and Council of Writing Program
Administrators
Linda Adler-Kassner and Howard
Tinberg for the NCTE/WPA/CCC Task Force |
|
The White Paper on Writing
Assessment
Valid and reliable assessment is consistent at the level of
principle and conceptualization: it is discipline-based, locally
determined, and used to inform teaching and learning at the
local level.
Assessment.
The word often strikes fear into the heart of college and
university faculty, who sometimes view it as an imposition into
their academic freedom and teaching practices. This perception
was particularly acute after the publication of A Test of
Leadership, the final report from the Spellings Commission
on the Future of Higher Education. But assessment is really
about investigating the questions at the heart of our
professional lives: What do we teach, how are we teaching, and
why are we teaching as we are? What is the effect of our
teaching on students’ learning – that is, how do we know
students are learning what we say they are? The responses to
these questions also form the cores of our disciplines. After
all, if faculty are not in a position to define what students
should learn, how and why they should learn it, and how that
learning should be assessed, then we run the risk of losing
control over what happens in our classrooms. At the same time,
assessment also provides opportunities for us to engage in
dialogue with interested others outside of our disciplines – and
perhaps even our departments or institutions – who are also
interested in what students learn, how, and why. In our own
discipline of composition and rhetoric, for instance, we are
regularly reminded that the questions at the center of our work
– how to help students become flexible writers, readers, and
thinkers – are often of interest to other faculty,
administrators, and employers.
The challenge that
those of us working inside the academy now face is how to can
embrace the opportunities provided by increased interest in
assessment to investigate our teaching practices and build
alliances with interested others with an eye toward improving
teaching and learning within our disciplines and across our
campuses. This challenge includes, for instance, reframing
assessment – away from terms like “accountability” and
“transparency” toward ones like “responsibility” and
“visibility.”
As a part of this
work (and stemming, in part, from regional hearings on the
Spellings Commission Report), a joint National Council of
Teachers of English (NCTE) and Council of Writing Program
Administrators (WPA) Task Force was convened to create a
White Paper that summarizes existing position statements on
assessment in writing drafted by National Council of Teachers of
English (NCTE), the Conference on College Composition and
Communication (CCCC), and the Council of Writing Program
Administrators (WPA). The White Paper articulates the
common thread running through these research and best-practice
informed documents: Valid and reliable assessment is consistent
at the level of principle and conceptualization and is
discipline-based, locally determined, and used to inform
teaching and learning at the local level.
Among the principles
articulated in the White Paper:
·
Writing assessment should place priority on improvement of
teaching and learning.
·
Writing assessment should provide the foundation for
data-driven, or evidence-based, decision making.
·
Writing assessment should use multiple measures and engage
multiple perspectives to make decisions that improve teaching
and learning.
·
Writing assessment should be based on continuous conversations
with as many stakeholders as possible and engage multiple
perspectives to make decisions that improve teaching and
learning.
To illustrate these principles, readers of the White Paper
will be directed to descriptions of specific assessments
undertaken by nine institutions of varying types across the U.S.
While the assessments address questions developed at the local
level and use methods that are locally determined and
appropriate for improving teaching and learning at the
local level, these assessments also demonstrate how these varied
institutions have drawn on common principles and
conceptualizations of writing and writing assessment to develop
assessment practices that are appropriate for them, and used to
improve teaching and learning at the local level. Assessment
models run the gamut, from sophomore writing portfolios
(Carleton College) to external assessments of a community
writing center (Salt Lake Community College) to evaluating
students’ “expert insider prose” in a writing in the disciplines
program (Seattle University) to assessment of a first year
program (University of Kentucky).
Readers also will be directed to a set of communication
strategies that addresses the question: how can we communicate
with external stakeholders about writing assessment? These
strategies provide a snapshot of practical, hands-on activities
that writing instructors and program administrators can use to
communicate with others to help them understand that valid and
reliable assessment is consistent at the level of principle and
conceptualization: it is discipline-based, locally determined,
and used to inform teaching and learning at the local level.
Among the strategies discussed:
Act locally
Develop alliances
Identify shared values with all stakeholders.
Together, these
documents will provide concepts, models, and strategies for
faculty to engage with assessment practices that can improve
teaching and learning. Such assessment need not be considered
extrinsic to the work of the classroom. Rather these documents
demonstrate that assessment remains at the core of our
professional work. The resources will be available July 1
through the NCTE and WPA web sites (www.ncte.org
and
www.wpacouncil.org ).
|
If you have a creative program in instructional technology you would like listed on the Resources
page, please send
us a brief description (250 words maximum). Be sure to include the
name of the program as well as a link to a Web site or the name and email
address of a contact person.
AN
INVITATION
We invite you to take the lead in framing
future Thoughts and Models. If you're interested and have a "Thought" in mind, please
send us an e-mail: reinventioncenter.
We will identify "models" that relate to it.
THOUGHT: The
Thought will consist of a short essay focusing on an issue central to
undergraduate education at research universities. The specific topic to
be addressed may vary. It may for example relate to an institutional challenge,
an aspect of student learning, a societal need, or a recent research finding
that may influence the way undergraduate education generally or in a specific
discipline is conceived and delivered at research universities.
MODELS: Each
Thought will be accompanied by reports on programs and experiences that
exemplify or expand upon the Thought. The models will be drawn from different
research universities, utilize different strategies, and, to the extent
possible, focus on different disciplines. Collectively, they will become
part of a database that will yield insights into what works or does not
work and why.
Together, the Thoughts
and Models will be incorporated into reports to be distributed through
this web site, professional society newsletters and our own mailings.
We welcome your comments and look forward to hearing from you. |