If you have an ACS member number, please enter it here so we can link this account to your membership. (optional)

ACS values your privacy. By submitting your information, you are gaining access to C&EN and subscribing to our weekly newsletter. We use the information you provide to make your reading experience better, and we will never sell your data to third party members.


Analytical Chemistry

How The Internet Enabled Information Sharing Through Massive Databases

Online stores of current data allow scientists instantaneous access to the information they need

by Jyllian Kemsley
August 16, 2015 | A version of this story appeared in Volume 93, Issue 32

Illustration of a 'database'.
Credit: Shutterstock/C&EN

Credit for the rise of databases—collections of data organized for rapid search and retrieval—rightly goes to the advent of computers. But access to those databases was limited and slow in the early years, requiring researchers to wait for punch cards or magnetic tape to arrive in the mail. What the Internet allowed was for thousands of users to instantaneously access up-to-the-minute information.

Over the past 50 years, “I can’t think of a single change that’s been as dramatic as the Internet in enabling the scientific community to share its data,” says Paul Davie, general manager of the U.S. operations of the Cambridge Crystallographic Data Centre, which manages the Cambridge Structural Database (CSD).

In the past 10 years, CAS has added 75 million entries to its registry—triple the number added during the first 40 years. SOURCE: CAS
Increase in substances in CAS database over the years.
In the past 10 years, CAS has added 75 million entries to its registry—triple the number added during the first 40 years. SOURCE: CAS

One example is the evolution of how chemists have used Chemical Abstracts Service (CAS), founded in 1907. For many decades, CAS analysts would read print journal articles and create abstracts and index information for each paper. The information would include author and subject, as well as formulas and ring information for compounds described in the paper. The abstracts were compiled into monthly mailings. Indexes came out at six-month intervals, then were gathered together into 5- or 10-year compilations.

For a literature search, a researcher would start with a keyword or formula, checking recent six-month indexes and then the larger volumes for leads to particular abstracts. Next, the researcher would go searching in the library stacks for the referenced journal articles. The researcher would repeat the search for different keywords or structure permutations.

“It was really a slow, tedious process,” says Dana Roth, chemistry librarian at California Institute of Technology.

It didn’t help matters when several—or several hundred—people needed access to resources at the same time. At Stanford University, a sophomore organic chemistry laboratory class required students to use the literature to help identify unknown compounds they characterized in the lab. In the days of print, “some students would have the indexes and some the abstracts, so someone could search an index and then have to walk all over the library to find the abstract book,” says chemistry librarian Grace Baysinger.

Over time, CAS started feeding information into a computer database and shipping microform and tape products as well as print. The Internet enabled direct access to the database in the 1980s, albeit through a command-line interface plus numeric codes to describe structural features. In 1995, CAS introduced SciFinder and a graphical interface.

The wonderful web of chemistry data, by Antony Williams

While working at Eastman Kodak in the mid-1990s, long before phones were our cameras of choice, I was involved in building a Web-based system for organizing the company’s analytical data. Simply put, my team’s motto was, “The Web Is the Way.” Although we didn’t know exactly where Web technology would take us, it was clear online tools were going to be important.

I left Eastman in 1997 to join a small cheminformatics start-up company that was simultaneously developing ways of delivering cheminformatics programs via desktop software and via the Web.

In 2006, a friend and I decided that access to the explosion of chemistry data appearing online should be facilitated by a central hub with a chemical search service. So we developed ChemSpider, a free Web-based, crowdsourced database connecting various sources of information about chemicals. ChemSpider went live with more than 10 million chemicals early the next year. Following an acquisition by the Royal Society of Chemistry, the site now holds data for more than 30 million chemicals drawn from almost 500 sources.

Today, there are many free chemistry databases available, including NIH’s PubChem and EBI’s ChEMBL. The Web is still the way, but with one difference: We all take it for granted now. We are generally one free search away from finding data or information to help us on our journey of investigation and discovery.

Antony Williams is known for establishing ChemSpider, the online chemical database. Now at the U.S. Environmental Protection Agency, he blogs about chemistry at

This year CAS registered its 100 millionth substance, a silicon-based compound created by drug discovery company Coferon to treat acute myeloid leukemia. The compound’s chemical name is (4S)-6-(4-chlorophenyl)-N-ethyl-8-[2-[[4-[(hydroxydimethylsilyl)-methyl]benzoyl]amino]ethoxy]-1-methyl-4H-[1,2,4]triazolo[4,3-α][1,4]benzodiazepine-4-acetamide.

A print search in 1960 for the compound, starting from triazolo benzodiazepine-4-acetamide, would have taken an experienced searcher at least five hours, estimates Matthew J. Toussant, who started at CAS in 1980 as an analyst and is now senior vice president for product and content operations. A command-line search in 1980 would have taken at least an hour, again for someone familiar with the necessary structure codes. Today, the search takes about as long as necessary to type in the name or draw the structure in SciFinder or ChemDraw—and thousands of users can successfully search for it simultaneously.

In addition to promoting easy access to database users on the front end, the Internet now plays a key role in funneling information into databases on the back end. Like CAS, other database resources started with people manually collecting information.

The U.S. Protein Data Bank (PDB) decided to start having researchers deposit structures electronically via the Web in the 1990s. “At the time there was nothing like it,” says Joel Sussman, who led PDB from 1994 to 1999 and is now director of the Israel Structural Proteomics Center at the Weizmann Institute of Science. The software PDB created incorporated validation algorithms to ensure researchers correctly filled in required fields. “We checked it and checked it and checked it,” Sussman adds. “I was afraid that if we released it prematurely and it didn’t work, no one would look at it again.”

Similar electronic entry tools applied at CAS, CSD, and elsewhere have sped up data entry, aided quality control, and lowered costs even as the amount of scientific information has exploded.

Meanwhile, organizations are forging ahead to link information repositories in ways that couldn’t be imagined without the Internet. CAS, for example, includes hyperlinks to journal articles and patents. PDB and CSD managers are now collaborating to match small molecules included in protein structures with corresponding CSD entries.

And new fields of cheminformatics and bioinformatics have developed to exploit the collective information contained in these and other scientific databases. “There are whole groups of computational biologists who rely on being able to have PDB data available to design new kinds of proteins synthetically,” adds Helen Berman, a chemistry and chemical biology professor at Rutgers University and associate director of the Research Collaboratory for Structural Bioinformatics, which now manages PDB.

Chemical safety information is also much more widely accessible, at places such as the National Library of Medicine’s Hazardous Substances Data Bank and Toxline, both of which link to literature references. Those involved in the regulatory side of the chemistry enterprise can readily access Occupational Safety & Health Administration and Environmental Protection Agency regulations and guidance online. “Through the 1980s, I subscribed to and was delivered the Federal Register every day” to stay informed about new rules and public notices, says Neal Langerman, a chemist who held faculty positions at Tufts University and Utah State University before starting a consulting business, Advanced Chemical Safety. “Now, I get the Electronic Code of Federal Regulations to deliver to me exactly what I want.”

The ease of accessing so much information does concern some. If an Internet search fails, “How often do our young scientists ask, ‘Well, maybe the data are there, but they haven’t been digitized?’ ” questions Langerman. For a young researcher used to searching through Google, turning to a library to find information that predates the digital era may be a high hurdle.

Also, the work that goes into maintaining access to computerized data is not inconsiderable. Think about what it means to upgrade personal computer software and retain access to 10-year-old documents or data files. Then consider what it must take to ensure data integrity and instantaneous worldwide access to 100 million chemical substances or 103,000 protein structures for 50 or more years. Additionally, for open access databases, funding can be a challenge. “There seems to be a lot of appetite within funding agencies to start resources, but later there seems to be a reluctance to continue funding them,” says Stephen Burley, director of the Research Collaboratory for Structural Bioinformatics and the Center for Integrative Proteomics Research at Rutgers.

All totaled, databases accessed through the Internet have clearly eased scientists’ work. Says CAS’s Toussant, “What we do today is quicken the pace of all of the work that people do.”  


This article has been sent to the following recipient:

Chemistry matters. Join us to get the news you need.