Undergrad Research Makes a Difference

The educators who advocate undergraduate participation in research are convinced that such programs enhance learning and retain students. Because assessing the effects of undergraduate research is difficult, these educators have until recently relied on their gut instincts.

"There has been a dearth of assessment studies," according to Thomas J. Wenzel, a chemistry professor at Bates College, Lewiston, Maine. He spoke last month at a symposium sponsored by the Division of Chemical Education at the American Chemical Society national meeting in San Diego. Many professors don't see the need for these assessments, perhaps even fearing that the outcomes of the studies will result in the loss of autonomy, he said. Such studies are difficult to do, and chemistry professors don't really know how to do them.

Nevertheless, such studies are a necessity, Wenzel noted. They are likely to improve the experience for all involved and can identify effective practices. In addition, Wenzel hopes that they can make the case for undergraduate research with "doubters, funding agencies, and policymakers. We need all the ammunition we can get."

These studies are now becoming more common. Organizations hope to gain valuable insights into the effects of undergraduate research by surveying students who participated in such programs. For example, the National Science Foundation recently sponsored several studies to assess the impact of its Research Experiences for Undergraduates, or REU, program.

For one study, Joseph J. Grabowski, associate professor of chemistry at the University of Pittsburgh and director of Pitt's REU program, evaluated the nationwide applicant pool for chemistry REUs to determine the "market demand." The question was this: Is the program big enough or is there a pool of potential participants who aren't being served? Across the U.S., there are 66 chemistry REU sites, serving approximately 650 students. "Nobody had any sense of how many students were looking for an REU experience," Grabowski said.

To answer those questions, Grabowski conducted surveys of the 2003 and 2004 applicant pools. He plans a similar survey for this summer. Grabowski described the survey results at the symposium. In 2004, 5,729 applications were received from 3,024 students. Despite an average of almost two applications per student, two-thirds of the applicants applied to only one site, and about 80% applied to only one or two sites. Because 85% of the students whose applications were accepted received only one offer, the sites are "not chasing the same students," Grabowski said.

THE SURVEY revealed that the chemistry REU program is able to accommodate only one out of every 4.4 students who apply. "In terms of growing a pool of domestic graduate students, it's a great sign that a large number of people are out there," Grabowski told C&EN. He worries, however, about the students who aren't accepted. "What impact does the fact that 77% of the students get a 'Dear John' letter have on them? If they're young and in their formative stage, does this really knock them back so that it points them in a different direction?"

In addition to the survey of the chemistry REU program, NSF has commissioned a large-scale study of all its undergraduate research programs. The study, conducted by SRI International, Menlo Park, Calif., includes three phases, two of which have been completed. The first phase surveyed students following their participation in undergraduate research programs during the summer of 2002 or the 2002–03 academic year. The second phase included a nationwide sample of people who received undergraduate degrees in science, engineering, or math between 1988 and 2003. The third phase will contact the same participants from the first phase to find out where they are two years later. At the ACS meeting, Susan H. Russell of SRI described the results obtained so far.

"Our study supported what people have felt intuitively all along, which is that undergraduate research opportunities are very successful in encouraging students to pursue an advanced degree and giving them motivation to do that," Russell said.

The surveys have looked at students' needs in terms of a research environment and the effect of the experience on their educational and career decisions. Important components for students include being involved in a variety of research-related activities, having good communication with one's faculty mentor, and being able to gain increasing independence. Gains were largely independent of the mentor's race or sex.

According to Russell, the few differences in the students' needs by gender and race have been among the most surprising findings. "It's perfectly clear that there are lots of individual differences, but we couldn't identify very many group differences," she told C&EN.

"The kids who go into targeted programs are the ones for whom that is important," Russell said. "We can't do away with those programs, but there's no evidence that every woman needs a female mentor or every minority student needs a minority mentor."

One important finding is that most of the students became interested in science or math as children. Efforts to encourage students to enter scientific fields therefore need to start in the elementary grades, Russell said. "If this is where kids are becoming interested, it's also probably where they're becoming turned off," she told C&EN.

NSF is now recognizing that assessment is important from the very beginning of a program. For its new Undergraduate Research Centers program (see page 39), proposals must include an assessment plan. "I think that the evaluation programs will be an important factor in whether or not a site is funded," Richard D. Foust Jr., a program officer in NSF's Chemistry Division, told C&EN. "There are a lot of people doing good chemistry out there, but not all chemists can do good evaluations."

In addition to broad studies such as the ones NSF has sponsored, academic institutions are assessing their own programs.

The University of Michigan has one of the oldest programs emphasizing undergraduate research as a means of retaining students. When the Undergraduate Research Opportunity Program (UROP) started in 1988, it was open only to underrepresented minorities. Today, the program is open to all undergraduate students, though it continues to emphasize underrepresented minorities and women in science.

Michigan's program takes a "nonremedial approach to retention," said Sandra R. Gregerman, director of UROP. "It tells students that they belong" at the university.

The students participate in faculty-led research six to 12 hours per week and in biweekly research peer group meetings, where they discuss topics such as research ethics and careers. Students can receive either work-study funding or academic credit.

Gregerman described a number of studies that Michigan has performed to assess whether UROP is having its intended effect. One study found that retention effects were strongest for African American students and for sophomores, rather than freshmen.

To find out if UROP had effects beyond graduation, alumni were surveyed to see how the postgraduation paths of UROP and non-UROP students differed. The students who participated in undergraduate research (UROP or other research) were more likely to pursue postgraduate education and to be engaged in postgraduate research activity. There were no differences by race or ethnicity.

The University of Delaware, Newark, has also done an assessment of its undergraduate research program. Andrew L. Zydney, formerly at Delaware and now a chemical engineering professor at Pennsylvania State University, State College, described the results of the survey.

Delaware's program is open to students in all majors. Each year, 600–700 students participate, and the participants are involved for anywhere from one to three years. More than 60% of the faculty provide undergraduate research opportunities, a number that climbs to more than 90% when only science and engineering faculty are counted.

Delaware has done two assessments, Zydney said: a content analysis and an alumni survey. The content analysis involved students writing free-form letters about their research experience ranging from one to four pages at the end of their sophomore summer.

The alumni survey included 986 students (59% of whom were science and engineering majors) who graduated between 1982 and 1997. The survey included 418 students who had participated in the centralized undergraduate research program, 213 who had done other undergraduate research, and 355 who had done no undergraduate research.

The alumni identified internships, undergraduate research, the senior thesis, intercollegiate sports, and study abroad as the activities with the greatest perceived benefit for grad school and their careers. The perceived benefit correlated with the number of semesters the student had been involved. Research students were twice as likely to have gone on to earn Ph.D. degrees, whereas one-third more alumni who had done no research were employed in careers unrelated to their major.

IN ADDITION to statistical studies, qualitative studies are also being conducted. Anne-Barrie Hunter, a member of the ethnography and evaluation research group at the University of Colorado, Boulder, described the second set of results from a study that aims to establish the benefits of undergraduate research experiences. In this second study, 76 undergrads, along with faculty members and university administrators, from four small liberal arts colleges with long-standing undergraduate research programs were interviewed about the effects of undergraduate research.

Analysis of the data sets showed that students and faculty overwhelmingly view undergraduate research as a beneficial experience: For both groups, just over 90% of all comments were positive. Faculty and students reported the same types of gains, although they had different emphases. Students emphasized gains in confidence, their ability to do science, and discovering whether "research is for me," whereas faculty noted the development of attitudes they view as necessary if the students continue in science.

These studies are a start, but there's still plenty that educators don't know about the effects of undergraduate research, including why and how it has the effects that it does. Perhaps future assessments can help educators answer those questions.