Presentation:

Session leader Dr. David M. Hanson brings tremendous expertise in the subject matter of conducting meaningful assessments within large science courses.  In addition to receiving several awards for his teaching, he developed the Process Workshop model for teaching introductory courses in chemistry that led to the NSF-supported POGIL National Dissemination Project (Process-Oriented Guided-Inquiry Learning).

Dr. Hanson began by asking each of the session participants to introduce themselves and share their objectives for attending the session.  Responses included interest in improving the assessment of understanding, problem solving, and team activities, as well as how to better transfer assessment skills to the faculty.  By presenting the Stony Brook University approach to assessment in large science classes, Dr. Hanson successfully addressed many overarching issues such as: How can students be motivated?  How can students be engaged to think and develop understanding?  How can there be student-faculty interaction?  How can the instructor learn if anyone is learning?

The first subject that Dr. Hanson addressed was the meaning of assessment.  It is important to distinguish assessment from evaluation.  Evaluation is the process of measuring performance quality to make a judgment or determine whether a standard has been met.  In addition, evaluation is judgmental and summative.  Assessment, on the other hand, is the process of measuring and analyzing performance to provide frequent feedback with clear and meaningful directives and insights to improve future performance.  Unlike evaluation, assessment is nonjudgmental, formative, and most important, it benefits the student. 

Dr. Hanson then highlighted some of the differences between large (more than fifty students) and small (less than fifty students) classes.  In a small class, motivation, engagement, interaction, and real-time assessments are relatively easy to accomplish. At Stony Brook University, the benefits of having smaller recitation classes associated with courses with large enrollments are highly valued.  In this setting, process-oriented guided-inquiry learning can be practiced.

In addition to the learning skills that most people would accept as important for students, such as information processing, critical and analytical thinking, and problem solving, Dr. Hanson submits adding teamwork, oral and written communication, and metacognition.  In particular, metacognition is very important because it facilitates a reflection on learning and the assessment of performance.

However, promoting these learning skills in a class of 350 to 500 students can be very challenging.  There are many issues associated with this context that faculty must address in order to engage students, monitor and support their work, and assess their performance in a meaningful way.  While generally nothing can be done about the design of existing lecture halls, Dr. Hanson suggests that if faculty or staff are able to participate in the design of new classroom spaces, they promote large lecture halls with swivel seats that enable students to collaborate with those behind them, create an aisle every fourth row that allows professors to walk around and interact with students, and provide power and internet access for the expected increase of laptop use in the classroom. 

When distributing materials, Dr. Hanson encourages faculty to provide students access to class materials in advance by uploading them on the web and/or creating an activity book that students can bring to class.  Dr. Hanson’s publisher prints the activity book (Guided Explorations) and packages it with the textbook at no extra cost to the student.

Another challenge that large classes present is getting students to work together and help each other.  One strategy is to organize students into teams using a seating chart for the lecture session so students in the same recitation class sit near each other, and their recitation instructor also is that section.  Such structure may not be necessary if a concerted effort is made from the beginning to promote such collaborative learning as an intrinsic part of the course.

In a successful team learning environment, students actively help each other and support from instructors or teaching assistants may not be necessary.  Also, the activities used during the large lecture class should not be as challenging as those assigned in the smaller recitation sessions in order to avoid frustration.

Accountability, according to Dr. Hanson, is the key to success. Tools that give students something to do during class and assesses their performance in real-time include electronic tablets and tablet computers in organic chemistry for drawing structures and clickers in general chemistry.  With these tools, students receive instant feedback because the technology allows the professor to collect and display the class results immediately.  Students see how well they did have done and are encouraged to continue their participation.  The bottom line is to “give students something to do; tell them to work together; let them work, think, and communicate with each other; and ask for their results.” 

The centerpiece of the presentation was Dr. Hanson’s class session design.  When students walk into class, they see a problem or question from the last session that encourages them to review before class and to be on time.  To introduce the topic of the day, he asks students for their thoughts and/or predictions on a particular example in order to connect to their prior knowledge and stimulate their interest and curiosity for the topic ahead.  The lecture, which generally is not longer than fifteen minutes, presents a model that represents in some way what students are to learn.  The model comes from the activity book or textbook, and critical thinking questions are used to guide exploration of the model and elicit discussion.  The last part of the class is devoted to problem solving.  A problem based on the model presented is given, and students are encouraged to work with others to arrive at a solution.  If they encounter difficulty, Dr. Hanson then offers to show how to solve the problem and follows up by offering another similar problem for practice.  Students are continually reminded of an approach to problem solving that involves first visualizing, summarizing, and analyzing the problem statement followed by working the problem and validating the solution.

Dr. Hanson has found that with this student-centered interactive pedagogy combined with real-time assessment, attendance and participation are high, most students seem to really like the courses, most students do pretty well on challenging examinations, and everyone has more fun.

Discussion:

On the subject of facilitating problem solving, Professor Chuck Weiss from the University of Utah, asked Dr. Hanson how to strike a balance between challenging students while preventing frustration.  The use of various, but similar, problem-solving methodologies were suggested along with asking students questions that guide their thinking without telling them what to do.

Others contributed their positive experiences with cooperative and collaborative learning.  Ms. Gili Marbach-Ad from the University of Maryland brought up the experiences of her faculty with study groups and incorporating writing skills into science classes. They found that students who studied in groups scored better on exams than those who stated they did not work in groups.  Professor Gerald Feldman from George Washington University shared an experience where he and a colleague taught two sections of the same course with one section being a taught using an in-class collaborative mode with 25 students and the other taught in a traditional lecturing mode with 50 students.  The collaborative class scored about 9 points higher on the exam, had a higher Force Concept Inventory score gain, and produced more positive anecdotal comments.

Ms. Marbach-Ad expressed her concern for incorporating essential skills such as written and oral communication into science classes. One way in which her University of Maryland faculty group has been able to incorporate a writing component to classes is through online discussion groups.  Dr. Hanson commented on this subject, explaining how recitation sections can be designed to reinforce communication skills. At the beginning of the semester, the professor should address the criteria for a good presentation of problems. Professors can ask each group to think about what a good presentation would entail and then expect them to abide by their own criteria.  In the problem-solving recitation section, each team has a recorder or spokesperson that presents the answer to their respective problem on the board in front of the rest of the teams.  Several solutions can be presented simultaneously and additional time spent only on those that require it.  This “parallel processing” gives students the opportunity to present their work and practice their communication skills when called upon.

A couple of participants had specific questions about the structure of the recitation sessions at Stony Brook.  In the course Dr. Hanson teaches, there are two recitation sessions and three large class meetings each week.  In one recitation class, there are eight or nine students and the focus is on problem solving.  Dr. Hanson added that part of a research project on problem-solving, observers are sent to these sessions three times each semester to characterize how the students approach solving a problem and how the instructor helps them.  The observers also provide constructive feedback to the students and instructors (teaching assistants) on how to improve the problem solving process.  The other recitation is larger and involves nine team activities of four if they are doing text-based activities and twelve teams of three if they are working on the computer.  Laboratory sessions that include hands-on experiments are offered as separate courses Stony Brook.

Another issue that was raised was the need to incorporate problem-solving into the evaluation instruments.  Dr. Hanson explained that although exams are multiple-choice, at least one problem-solving question is included where students have to document their problem solving process for points.  This question is graded by the teaching assistants following a rubric.  Usually five teaching assistants score 350 problems in about two hours.  Another participant emphasized that it is important to explain to students that memorizing and pattern matching are not the right techniques to use in solving problems, but rather they need to analyze the problem, identify the tools that are needed, and use them in the right order to solve the problem.

Keith Garbutt from the University of West Virginia, added that faculty should consider making the first exam count less than the others; especially for freshmen, who may not do so well in their first exams due to other issues related with transitioning into college.

Mr. Garbutt also commented that his students did not take the clickers seriously because of how little their use counted towards the course grade.  Other problems with clickers involve students forgetting them or malfunctioning problems.  Dr. Hanson responded that he and others simply circumvent these problems by dropping the 25% of the clicker scores.  Professor Feldman, mentioned that many science faculty choose not to grade the clicker questions at all to encourage more participation while warning students against just guessing the answers as a waste of everyone's time.

Another assessment tool discussed was homework and whether it should be assigned and graded. Dr. Hanson commented that he uses class time to help students develop understanding with guided-inquiry activities, and then uses homework to reinforce and apply this understanding. Professor Weiss brought up the use of pre-class quizzes that students could take as many times as needed, so they serve as learning tools and not evaluative tools.  The provision of online quizzes was also discussed as a “self-correcting mechanism” that provide real-time assessment and feedback.  In response to a concern that online testing tools are too answer-driven. Professor Weiss mentioned that there are instruments with a technology that makes problems in students’ online quizzes similar but different.  With this feature, Professor Weiss can be sure that even if students help each other, they will likely be teaching the concept to each other and not just sharing the answers.  Dr. Hanson added that he also does a few surprise quizzes throughout the semester that test problem-solving processes specifically.

The last issue that was tackled in the group discussion was how to transfer assessment skills to other faculty members.  Many of the participants mentioned that there is a gap between the material professors cover and the material students actually understand and retain.  Assessment skills can be very important to help teachers learn that students are not fully understanding concepts or retaining important information.  Some participants agreed that some faculty would need to be enticed with national comparisons of student achievement using alternative assessment methods.  Others mentioned offering incentives to increase attendance at summer training sessions where assessment skills would be discussed.  Dr. Hanson shared his positive experience with encouraging his colleagues to use in-class assessment to check what students are actually learning in a lecture.

Recommendations:

For Individual Campuses

• In addressing the challenges of teaching large classes, individual campuses and departments should consider the new paradigm and associated techniques of process-oriented guided-inquiry learning, which engages students and recognizes that the development of learning process skills is at least as important as mastery of discipline content.

For The Reinvention Center

• The Reinvention Center should sponsor a discussion session among faculty and administrators to identify how a culture of assessment and continual improvement can be created on campuses.  In preparation for such a session, faculty should examine their courses and come prepared to share strategies for implementing learning assessments leading to improvements in the teaching – learning process.  Administrators should come with ideas for facilitating such assessments, rewarding successful efforts, and preventing assessment from becoming evaluation.

Resources/References:

Websites

1. The POGIL National Dissemination Project: http://new.pogil.org/info/introduction.php.
2. Stony Brook University Chemistry Department http://www.chem.stonybrook.edu/.

Publications

1. Instructor's Guide to Process-Oriented Guided-Inquiry Learning, David Hanson (Pacific Crest, Lisle, IL, www.pcrest.com, 2006).
2. A Cognitive Model for Learning Chemistry and Solving Problems, David Hanson, in Process-Oriented Guided-Inquiry Learning, edited by R.S. Moog and J.N. Spencer (American Chemical Society, Washington, DC, 2008) pp. 14 – 25.
3. POGIL Implementation in Large Classes: Strategies for Planning, Teaching, and Management, E.J. Yezierski, C.F. Bauer, S.S. Hunnicutt, D.M. Hanson, K.E. Amaral, and J.P. Schneider in Process-Oriented Guided-Inquiry Learning, edited by R.S. Moog and J.N. Spencer (American Chemical Society, Washington, DC, 2008) pp. 60 – 71.
4. Faculty Guidebook: A Comprehensive Tool for Improving Faculty Performance, edited by S.W. Beyerlein, C. Holmes, and D.K. Apple (Pacific Crest, Lisle, IL, 4th Ed, 2007).

POWERPOINT PRESENTATION