The Deal With Big Data

It was interesting to read about the growing attention paid to “Big Data” (C&EN, Oct. 21, 2013, page 19). The “overall chaos of ‘too much’ ” is something that could still easily overwhelm the industry. Chemists need information from a multitude of different sources, each with its own origins. But there’s a huge gap between volume and relevance that needs to be bridged.

For example, many chemists will use freely available search engines at the beginning of a project to look for connections or inspiration. However, a single search can return a huge amount of information, relatively little of which will be relevant or useful. But this is not always the case, particularly in highly complex areas of science.

Big data tools must be able not only to crunch the numbers and information into carefully filtered and analyzed results, they must also present the user with the exact information required. Even in that first handful of pages of results there may be only specific nuggets that are truly valuable. As a result, any tool must be able to contextualize and classify information to ensure that researchers are working with the most relevant results. Big data must be focused on breaking huge blocks of information down to the smallest particles.

Only when we can ensure that our tools enable confident decision making at every stage of chemical research will we realize big data’s value rather than feel as if we are drowning in the chaos of too much.

Meeuwis van Arkel
Frankfurt, Germany