Presentation:

Dr. Laura Damuth is currently the Director of Undergraduate Research and Fellowship Advising at the University of Nebraska-Lincoln. She has been in this position since the inception of the program in 2000. Dr. Damuth received her bachelor’s degree from Vassar College and her Master’s and Doctorate from Columbia University in Music History. Her education provides a unique perspective to research.

Dr. Damuth asked how many schools in her audience have centralized undergraduate research programs.  Most in the audience said they do; a few have new programs; and some others have decentralized programs.  Dr. Damuth stressed the importance of the program’s director leading the assessment initiative. Only someone intimately involved with the program would really know the right questions to ask in order to shape the program. 

The first section of the presentation focused on explaining the undergraduate research program at the University of Nebraska-Lincoln (UNL) known as Undergraduate Creative Activities and Research Experiences (UCARE). The program, begun in 2000, includes over seventy disciplines in all nine of the University’s colleges. It is a campus-wide program sustained through funding from a Pepsi-Cola endowment and Program of Excellence funds provided by the senior administration at UNL. With this top-down initiative, the goal was to bring together the three missions of the land-grant institution: excellence in teaching, research, and service, with strong emphasis on the first two missions.

The mission of the program is two-fold: to enrich the undergraduate experience while also providing benefit to the UNL faculty sponsors. The goals are to engage students during their sophomore and/or junior year and provide a mutually beneficial experience for both faculty and students. Because the mission and goals focus on both faculty and students, it is incredibly important that both be assessed.

UCARE is designed as a two-year program, and over sixty percent of the students return for the second year. During the first year, students work with the faculty members on the faculty sponsors’ research project in order to learn the language of the discipline and also to provide a strong foundation for the students’ own research project for the second year. The first year of UCARE is a training year. The second year project can be related or as seemingly unrelated as the student and faculty member decide how best to develop the undergraduate student as a primary researcher on a project. At the end of the second year (or first year if they are not returning for a second year), the student is required to present his or her research to the university at the annual undergraduate research conference.

Because the UCARE program was new, Dr. Damuth needed to pay attention to both program assessment and the assessment of student learning outcomes. Initially, UCARE consisted of a folder with 100 approved applications.  To begin the process of creating the program, she spoke with some key players with established undergraduate research programs, including, but not limited to: Janice De Cosmo at University of Washington, Sandy Gregerman at University of Michigan, and Carolyn Ash Merkel at CalTech.

Dr. Damuth then discussed the different phases of the assessment. Phase I focused on determining if the program was working. This was a formative assessment, which consisted of a survey at the end of the first semester to determine whether students and faculty were working together on research. The first survey assisted in determining what faculty and students needed from the program and resulted in a more developed website, application forms, student and faculty handbooks, and agreement forms among the faculty, student, and UCARE program.

Phase II of the process was to meet with key faculty and students in the arts, humanities, and social sciences through a focus group project. Fifty-six percent of students in the UCARE program are from the arts, humanities, and social sciences. Dr. Damuth hoped to see how the UCARE program could continue to serve these fields, as they are not the ‘typical’ fields that research programs serve. The groups took on a life of their own and helped shape future surveys by both asking and answering questions about what students were gaining from the experience. Dr. Damuth mentioned that she was able to recruit students from the arts, humanities, and social sciences because she spoke with these disciplines individually.  Because she is outside the hard sciences herself, she was able to talk on their level about how the program could assist them in their research.

The focus group showed that both faculty and students appreciated the individual attention to learning through research. This also created another opportunity for students to be advised and encouraged in their goals while gaining confidence in their own work and approaching other faculty members. The research helped students to see different possibilities for careers, and faculty began to recognize students as junior colleagues. While Dr. Damuth pointed out that focus groups provide a certain kind of information which is somewhat anecdotal, it was necessary to achieve a broad-brush picture of the program that could share the foundation of a more-detailed future evaluation questionnaire.

Phase III coincided with a watershed year in terms of funding for the UCARE program. There was a significant increase in funding, which resulted in a greater need for financial accountability. UCARE students are paid student workers in the faculty sponsor’s home department. The UCARE office oversees the financial outlay, but the individual departments handle the payroll process. As a result of more funding (and therefore more students), Dr. Damuth began working with the financial contacts in each department to make sure that each person understood how UCARE students were to be paid. From these meetings, pre-trip approval and pre-purchase approval forms were created. In addition, the website was further updated to include instructions for financial contacts within each department.

At the same time, the UCARE first survey went out to students. The purpose of the survey was to understand what students were gaining from their experience, how students found out about UCARE, student perception of their field of research, and providing a means for self-assessment. The survey also asked about the faculty interactions, and focused on outcomes rather than processes. The students were also asked about what they were planning on doing with their research and whether they had plans to attend graduate or professional school.

After discussing the evolution of assessment within undergraduate research, Dr. Damuth talked about the program’s next steps, which look to assess growth in each student via pre- and post-test measures. In addition, the program is beginning to assess faculty satisfaction and incentive for being involved in the program.

Discussion:

Throughout the presentation, Dr. Damuth posed different questions to the audience and was often asked questions or given feedback along the way. This is a summary of the discussions and feedback.
 
It was obvious that many of the schools present during the presentation were going through similar evolutions of assessment. One individual mentioned that her program needed to look at participation by demographic group, by discipline, and so forth, and that it was oversubscribed in certain areas. There is a necessity for the program to determine whom it serves.  The participant’s program has consequently done significant outreach to the admissions office to target different students even before they reach campus.

In particular, assessment was being used during a time of budget crises on one campus in order to sustain the program. When assessment showed that students in the program were being accepted at top tier institutions, the program could make a strong case for itself in difficult financial times. Dr. Damuth suggested that assessment is vital to justify a program’s existence. Also serving as the Fellowship advisor, she noted that most students awarded nationally competitive scholarships and fellowships have been previous UCARE students.

One recurring theme was whether students needed to be paid to do research. Georgetown has been asking students recently if they would do the same research project on a volunteer basis, and most agreed that money is not a primary reason for doing research. One audience member mentioned that her institution used an open-ended question that asks what other types of research students have engaged in to see if the funding is important. To this, Dr. Damuth responded that most students in the UCARE program do not use their entire allocated stipend. She suggested that students seem enticed by the payment option but once they get involved, it became less important.

The topic of the budget for UCARE was raised. The University of Nebraska-Lincoln financially supports over 400 students in undergraduate research each year.

Participants recommended using outside sources to conduct the focus groups in order to minimize bias. In addition, Dr. Damuth asked for feedback on the questionnaire. The group offered several suggestions, including wording and question etiquette. They also discussed different ways to assess both faculty and students to see whether students really did achieve the learning outcomes that were desired.

Alternative means of assessment mentioned included a town-hall forum for students to see that the program considers their opinions. Others suggested that involvement in assessment as an expectation built into the agreement form.

Recommendations:

• Institutions should engage one another in finding better ways to assess. One of these means is through PEARL (Program Excellence through Assessment, Research, and Learning), an online assessment software designed for program improvement, which the UCARE program is beginning to use as it continues to develop its assessment measures. Another option would be to create a publication focused on undergraduate research assessment.

• Focus on individual stories throughout the assessment process. One student succeeding is important and remembering their story to share with others can help others see the value in a program.

• Recognize that assessment strengthens the program. Assessment should shape the program, but should also develop as the program develops.

Resources/References:

Websites

1. The UCARE Program.  www.unl.edu/ucare
2. PEARL Project. http://www.unl.edu/ous/pearl/pearl.shtml

POWERPOINT PRESENTATION
Assessing Undergraduate Student Participation in Research. The Reinvention Center Conference Proceedings 2008. Washington, DC. Laura Damuth, Ph.D. Director of Undergraduate Research, University of Nebraska-Lincoln.