Forcing Change In Forensic Science

Five years ago, the National Academy of Sciences put out a report condemning the state of forensic science. It concluded that many common forensic techniques—the analysis of fingerprints, bite marks, blood splatter, and ballistics, for example—lack sufficient scientific underpinnings. Thousands of convictions were thrown into question.

But in the years since, little has been done to shore up the discipline’s scientific base or to make sure that its methods don’t result in wrongful convictions. Quality standards for forensic laboratories remain inconsistent. And funding to implement improvements is scarce.

While politicians and government workers debate changes that could help, fraudsters like forensic chemist Annie Dookhan keep operating in the system. No reform could stop a criminal intent on doing wrong, but a better system might have shown warning signs sooner. And it likely would have prevented some of the larger, systemic problems at the Massachusetts forensics lab where Dookhan worked.

A glimmer of progress is starting to emerge, though, in the form of initiatives at the Department of Justice and the National Institute of Standards & Technology. These agencies are creating two oversight organizations that will attempt to make reform ideas a reality, both in Washington, D.C., and in forensic labs nationwide.

New Federal Infrastructure Attempts To Reform Forensic Science

The National Academy of Sciences’ scathing 2009 report called for massive changes to forensic science oversight and further research to shore up the discipline’s methods. But five years later “not much has happened,” says Jay A. Siegel, a forensic scientist who was on the committee that wrote the report.

Big changes may finally be in the offing for forensic science, though, as the federal government, Congress, and the larger scientific community attempt to address forensics’ fundamental flaws.

The Department of Justice (DOJ) and the National Institute of Standards & Technology (NIST) have teamed up to create a National Commission on Forensic Science, which will attempt to take the National Academy’s broad recommendations and turn them into action. And NIST is starting a new organization to create uniform standards across forensics disciplines, including several chemistry-related fields.

Chemists are playing an important role in those changes. A half-dozen chemists are on the new national commission, and more will undoubtedly join NIST’s effort to create new standards. The American Chemical Society, which publishes C&EN, recently adopted a policy statement that calls for increased scientific rigor in forensic science (http://bit.ly/StMhT2). In addition, ACS and other scientific organizations have supported pending legislation in Congress aimed at reforming the discipline.

“Seeing the full power of the scientific community come into this is thrilling to watch,” says Madeline deLone, executive director of the Innocence Project, which works to exonerate prisoners who have been wrongly convicted. “It is not a simple process to change the way forensic science has been done. These changes are the biggest step forward in years.”

But some observers are not so sure that the current changes will be able to restore faith in forensic evidence. And they have a right to be skeptical. The White House reacted quickly after the 2009 report was released: Within months it charged the interagency National Science & Technology Council with making reform recommendations. Almost five years later, however, nothing from that effort has been publicly released. Congress hasn’t had much luck either. Bills introduced last year haven’t yet gone anywhere.

In fact, national-level reform seemed stalled until 2013, when DOJ and NIST announced their plans to create oversight bodies intended to jump-start change. Those organizations—the National Commission on Forensic Science and the Organization of Scientific Area Committees—are just now starting their work.

The National Commission on Forensic Science held its inaugural meeting in February and meets again this week. Its 30 members, chosen from around 300 applicants, represent the forensic science, legal, and law enforcement communities. The commission also includes high-profile scientists outside of forensics, such as Nobel Prize-winning chemist Thomas R. Cech and University of Maryland physicist S. James Gates Jr.

The commission will map out what must be done to ensure that forensic scientists produce reliable evidence using scientifically rigorous methods. The U.S. attorney general will then have to decide whether to make federal labs or those who get federal money to follow the commission’s recommendations.

But the attorney general doesn’t have the power to force the thousands of forensic labs overseen by states to do the same. “The federal government doesn’t have enforcement power over the states,” explains commissioner John Fudenberg, assistant coroner with Nevada’s Clark County Office of the Coroner/Medical Examiner.

Instead, the commission will have to rely, in large part, on the power of persuasion to bring change nationwide. “Our job is to be the bully pulpit,” says commissioner Suzanne Bell, a forensic chemist at West Virginia University.

Unlike the National Science & Technology Council’s effort at reform, the national commission’s meetings will be held in public, and its recommendations will be available whether the attorney general endorses them or not. That means they will be available to defense attorneys to use when cross-examining forensic scientists and for states to consider when looking at whether their state or local forensic labs are doing their job.

“Make no mistake. People in the forensic sciences want to do a good job. They want standards, and they want guidelines,” Fudenberg says. “This is a unique opportunity for the forensic science disciplines to have a voice at the federal level. We can make a big impact on the entire nation.”

Some of the major issues that the commission will need to tackle became clear during its first meeting. Just a few examples include ensuring accreditation of forensic labs and certification of forensic scientists, explaining scientific uncertainty in the courtroom, training current and future forensic scientists, and creating an enforceable code of ethics.

Underlying it all, the commission must lobby for research into what aspects of forensic science are actually science. Fingerprint analysis, a staple of many forensic cases, is the most commonly cited example: No study has ever shown that every individual has unique fingerprints or defined what is needed to ensure that two fingerprint samples match.

“If we can make forensic science more science-based from the bottom up, I think that would make a big difference,” says Bell, who is chairing the commission’s Scientific Inquiry & Research Subcommittee.

But the commission won’t go so far as setting standards for how individual forensic scientists should perform specific experiments, whether it’s testing drugs or comparing ballistics.

That job will fall to NIST’s new standards-setting organization, the Organization of Scientific Area Committees (OSAC). Approximately 650 forensic scientists and other experts will be part of this multitier organization, which will set standards and guidelines to improve the quality and consistency of forensic science. Recruiting ended last week, and NIST hopes to have OSAC up and running by the fall. “It is a logical path to more enforceability,” says Mark D. Stolorow, who is leading the effort at NIST.

OSAC will replace the scientific working groups, a somewhat ad hoc system that laid out guidelines for different forensic science disciplines but didn’t have a uniform system of organization or oversight. Some of these groups, for example, those focused on forensic toxicology and drug testing, put out thoughtful guidelines for the community and secured federal support for their work. But others were underfunded, rarely met, allowed unqualified members to join, and didn’t rely on research, observers note.

The new OSAC structure will bring similar disciplines together under one umbrella to share ideas, plus provide stable funding for meetings. “It seems like a logical fit for us,” says Robert A. Middleberg, a director of NMS Labs, which provides forensic services, and chair of the Scientific Working Group for Forensic Toxicology.

Eventually NIST hopes OSAC will create a registry of forensics standards that could be used to accredit forensic labs. At the moment, there is no federal requirement for such accreditation, nor any oversight of the organizations that currently offer it. The two largest accrediting organizations now require little in the way of discipline- or technique-specific standards. They say they are open to the idea of ensuring that labs meet more detailed standards but want to make sure OSAC is working before they commit to following its lead.

“The challenge is how mobile or agile is it going to be? Is it going to be bogged down in bureaucracy?” asks Keith Greenaway, vice president at ANSI-ASQ National Accreditation Board, which accredits many industries, including forensics.

Even if OSAC rises to that challenge, there are still problems in forensic science that can be fixed only through legislation. The biggest one might be money. NIST and DOJ are funding their current efforts out of their existing budgets. But that funding isn’t sufficient to support the research required to shore up the scientific underpinnings of forensic science. Other agencies, including the Department of Defense and the National Science Foundation, have expressed interest in funding such research. But none of those agencies have dedicated money to such an effort.

Congress could address the money problem by appropriating funds for forensic science research. There is no indication, however, that Congress will add such funds to the relevant agencies’ budgets.

The forensic bills currently in the Senate and House of Representatives instead add to existing calls for reform. Parallel versions of a bill recently introduced in the House and Senate would create a more robust forensic science research program (H.R. 6106 and S. 2022); another bill recently introduced in the Senate would create a forensics oversight structure within DOJ (S. 2177). If passed, the legislation could work seamlessly with what NIST and DOJ are currently creating.

These bills would codify into law what the Administration has been doing, says the Innocence Project’s deLone. That’s important because a future Administration could choose to address forensics differently.

Clearly, much work still needs to be done, but members of the original National Academy of Sciences panel are excited just to see things start to happen.

“Momentum is now building,” Siegel says. “This is the best and maybe last chance to implement some of the recommendations for forensic science.”

Pinpointing Red Flags In Massachusetts Drug Lab Scandal

Disgraced forensic chemist Annie Dookhan was sentenced to prison more than five months ago. Yet her saga continues to roil both her home state of Massachusetts and the forensics community, providing a very public example of the need for reform in forensic science.

Dookhan fabricated drug test data and lied under oath. Her misdeeds affected cases involving more than 40,000 individuals, cases Massachusetts has set aside $30 million to review.

Meanwhile, a March report from Massachusetts Inspector General Glenn A. Cunha revealed a rash of management failures at the William A. Hinton State Laboratory Institute, the now-shuttered drug lab where Dookhan worked. Managers failed to provide proper chemistry training, and they ignored the concerns of Dookhan’s colleagues.

The findings flabbergasted many in the forensics community. The quality bar for forensic chemistry labs has risen dramatically in recent decades, notes West Virginia University’s Bell. But the Hinton lab, she says, “would’ve been considered off the cliff back in the 1980s—the report is that appalling.”

The report reveals more than bad management, however. It also highlights Dookhan’s forgery of gas chromatograph/mass spectrometer (GC/MS) quality-control records—actions that have a small but not insignificant chance of affecting other chemists’ results. In addition, it shows that multiple chemists—not just Dookhan—unknowingly performed a drug identification technique improperly.

The Hinton lab’s history contributed to its dysfunctional culture. From 1910 until 2012, when a long-planned reorganization brought it into the Massachusetts State Police department, the lab was situated within the state’s Department of Public Health. While the rest of the department concerned itself with disease outbreaks or wellness promotion, Dookhan and her colleagues struggled with a backlog of criminal cases that didn’t mesh with her administration’s mission statement. “No one in that management structure knew the pressures in a forensic lab,” or in the justice system in general, says forensic chemist Siegel.

Because most managers’ backgrounds were in scientific disciplines other than chemistry, they also were ill-prepared to catch Dookhan’s fabrications.

In particular, the inspector general’s report shows that on at least four occasions, Dookhan falsified quality-control (QC) records for the lab’s GC/MS. Raw data showed that the instrument failed to find cocaine or codeine in a standard mix. But Dookhan signed off as if the instrument had detected cocaine and codeine. “And then management signed off because she signed off, instead of looking at the underlying data,” says José R. Almirall, a forensic analytical chemist at Florida International University.

Every chemist at the lab relied on gas chromatography/mass spectrometry, the gold standard for confirming the chemical identity of seized drug samples. But Dookhan’s QC forgery doesn’t automatically invalidate every chemist’s results for the day, says Jack Mario, a forensic chemist and consultant hired by Cunha’s office to assist with its investigation. Labs use several procedures to make sure a GC/MS gives chemists reproducible spectra that can be compared with standards, he says. The instruments in drug labs, he adds, are workhorse machines that tend to be very robust.

With respect to QC, “Dookhan committed an egregious lapse in protocol,” Mario says. “Do I think it really affected others’ results? I can’t imagine it would. But I don’t know that that reasoning could overturn the concerns of a defense attorney.”

Other chemists think Dookhan’s actions call for close inspection of all GC/MS data from the days in question. “It raises a red flag as soon as there is fabricated GC/MS data,” says Nelson Santos, a chemist with the Drug Enforcement Administration and an expert in the analysis of seized drugs.

“I would want to review every case that was done on those days,” agrees Bell. Certain cases, such as those involving designer cannabinoids, demand that an instrument be in top working order.

Dookhan was the only chemist at the Hinton lab who deliberately forged records, the investigation concluded. Unintentional errors, though, were widespread. According to the report, many chemists in the lab were incorrectly using sampling techniques in drug trafficking cases.

In seized-drug cases, sample weight can be critical in determining charges, such as possession or trafficking, or the severity of a penalty. Samples often arrive at a drug lab in multi-item batches, so weighing is not trivial. Chemists often weigh and chemically characterize each item, or do so until they reach a threshold weight for a charge. But this can be impractical.

So international forensics working groups have agreed upon a small number of techniques that permit an analyst to test a portion of a multi-item population and then make statistical inferences about the identity of a larger portion of that population. One of these methods, the hypergeometric approach, was in use at the Hinton lab.

But most of the lab’s chemists lacked sufficient grounding in statistics to use this approach properly. They selected samples incorrectly, made improper conclusions about their data, and failed to report the inherent uncertainty associated with their measurements.

At times, these errors resulted in chemists overstating the weight of a population of drug samples, “a critical error in a case near a statutory trafficking weight threshold,” according to the report. Or, as George Washington University forensic spectroscopist Walter F. Rowe puts it, “they knew just enough science in that lab to get themselves in trouble.”

The report stopped short of recommending that the 55 cases affected by this issue be retested. In each of those cases, chemists did not statistically identify enough of a drug sample to exceed the relevant trafficking threshold. The report notes that the inspector general’s office will notify the agencies that prosecuted these cases.

Retesting samples of seized drugs stored in evidence for months or years might be problematic in certain cases, Siegel says. Cocaine picks up moisture with time, marijuana leaves can dry out, and other samples can decompose.

Rowe has often retested seized-drug samples and says that those weights are typically a few percentage points lower than originals. New tests may indeed raise more questions than they answer, he says, “but I think it’s worth looking again, especially for cases near a threshold.”

The report does recommend mandatory accreditation for all forensic labs in Massachusetts. Currently, accreditation is voluntary in that state. The Hinton lab was not accredited, but the state police lab that now conducts most Massachusetts drug testing is. According to the Bureau of Justice Statistics, 83% of publicly funded crime labs nationwide were accredited in 2009.

Accreditation “would’ve been an enormous benefit for this lab,” Mario says. But even those safeguards aren’t a guarantee against human nature, he cautions.

“Annie was an employee that I think many supervisors would welcome,” he says. She worked overtime, often without pay, and was eager to please in all aspects of lab life. Annie’s bosses failed to recognize her crimes, Mario says. “But plenty of other managers would be subject to the same kind of human frailty.”