In my introductory blog post, I mentioned that this series would approach scientific data management from both bottom-up and top-down perspectives. The last two posts introducing the small lab and research center described data management issues specific to those sites. In this post and the next, I’ll introduce important general concepts about scientific data management. This will give us a common language with which to analyze the case studies in greater detail.

We need a general conceptual model and language that help us describe what happens to research data once it is acquired. There are many lifecycle models to choose from. In this series, I’ll use a preliminary model we have been working with here at Prometheus that is general enough to encompass both of the case studies, but specific enough to provide context for meaningful analysis of data management practices. As I describe each stage of the lifecycle, you may find it helpful to think about what happens to your research data at that stage. For the sake of clarity, I’ve omitted certain steps, such as producing a data management plan and archiving data, that are important but are not formally part of the cycle I will describe. They will be the topics of later posts.

The Data Management Lifecycle

Data undergoes four primary stages: acquisition, curation, use, and repurposing. These activities are connected via data management tasks, quality assurance (QA), exploration, transformation, and expansion (see figure below).

Data first enters the lifecycle by being acquired. Researchers typically acquire data from multiple sources, which must be centralized and stored to enable QA testing. QA testing helps you ensure that (1) you’ve collected the right data, (2) it isn’t corrupted, and (3) the structure is what you expected it to be.

Once QA is complete, the data is ready to be curated — that is, organized, cleaned, and monitored for its intended use. Curation is one of the most important steps in the data lifecycle because it improves data quality and makes the data easier to analyze and reuse.

When the data is clean, it’s usually first explored.  Exploration can involve many activities, such as getting familiar with the columns and rows in a data set and seeking to understand the relationships among variables. At some point, exploration crosses over into actual data use, such as conducting statistical analyses, creating reports, and sharing the data with others. Data may spend a long time in this stage.

Data sometimes has a life beyond the project that generated it. It might be used again for the same purpose (data reuse), or it might be transformed for an entirely new purpose (data repurposing). Google is a prime example of data repurposing. When you conduct a Google search, Google doesn’t only use your search string and general location to provide search results, it also uses that data to deliver customized advertising and even to model flu trends.

Not all data gets repurposed, of course. Sometimes, as data is explored or analyzed, the analyst realizes that more data is needed to answer the original question; or, the results prompt a new question. In either case, one must have access to the data to answer the respecified question. The cycle starts again when this data is acquired or accessed.

Now that we have covered the basic data management lifecycle, we are almost ready to analyze different data management scenarios. In the next post, I’ll introduce Prometheus Research’s Integrated Data Management model. By the end of that post, we’ll have covered the analytical tools needed to critique the data management challenges that the stakeholders in each of our case studies are facing.

1) Decentralization of electronic data capture (EDC). Researchers are moving away from costly in-house study visits for EDC. Instead, for some types of research, they are equipping participants with convenient and budget-friendly ways to complete study requirements through mobile devices like phones and tablets. However, this trend is not without implementation challenges. Our OS RexDB team will need to carefully evaluate the security and compatibility issues that accompany remote EDC.

2) Multidimensional data integration. The days of siloed, single-lab or single-data-type research are long gone. Research data are now collected across multiple sites, studies, and data types (e.g., biospecimens, phenotype, genotype, images), as well as across different time points and subsets of people. Given this variable complexity, creating one-off Access databases or merging hundreds of Excel spreadsheets to create a few data sets is not going to cut it. Luckily, Prometheus is ahead of the curve in this area since we have been solving data integration problems for over a decade.

3) Reproducible research and open science. This trend encourages transparency in research methods and analyses to facilitate the replication of research findings. The Open Science Framework, a web application developed by the Center for Open Science, is a useful tool for making your study open-science ready and reproducible. It supports collaborative, scientific workflows and can help researchers and their collaborators better document their study design, materials, and data plans. Even better, users can control what information they want to make public or keep private.

RunMyCode is another useful way to document study analyses for a published research article. Researchers can create a companion site for a published paper by adding their data and calculations at runmycode.org. Other users can access the information and run the code through the website, making it easy to inspect and verify the findings published in the original article.

OS RexDB technology fits nicely with the Open Science trend because the system will have an audit trail, will make it easy to document meta-data, and will enable quick data exports.

4) Data interoperability and standards.  With the wide implementation of electronic medical records (EMR) systems, researchers need to be able to “plug and play” easily across data systems (e.g. data exchanges between EMR and a clinical research database). One of the main challenges with data interoperability is the lack of a standard information model. One organization that has spearheaded the standardization effort for protocol-driven clinical research isthe Clinical Data Standards Interchange Consortium (CDISC). CDISC provides a common framework for defining the structure, format, and exchange of clinical research data throughout the data lifecycle. The OS RexDB team will need to evaluate the challenges of enabling native support for standards before they facilitate data interoperability with these standards and formats.

Prometheus aims to be deliberate and thoughtful with regards to the features we support in OS RexDB.  Staying abreast of the latest research trends is just one way in which we can identify real-time data management challenges and debate creative ways in which to solve them.

The HyperText Rapid Application Framework (HTRAF) is an efficient way to embed data into HTML pages. HTRAF is unique because it works with HTSQL to fetch data from a database and display it directly on a page. Check out the introduction video below to learn more about how HTRAF enables speedy screen development.

HTRAF demo 2013-02-05 from Prometheus Research, LLC on Vimeo.

Tweets by @rexdb

Prometheus announced today that it has been awarded $700,000 by the National Institutes of Health (NIH) Small Business Innovation Research (SBIR) program to extend its Open Source Research Exchange Database (RexDB) for the management of autism spectrum disorders research. The overarching aim of the project is to empower autism investigators to make more effective use of data locally and more efficiently exchange data with the scientific community.  “Existing technologies do not support the need to maintain integrated data management systems in dynamic research environments,” explains Dr. Leon Rozenblit, CEO of Prometheus Research and Principal Investigator on the project.  “Our existing platform, RexDB, is close, but we know from experience that there are a number of essential features still to be developed to close this gap.”

The Research Exchange Database was initially developed to meet needs of the Simons Foundation Autism Research Initiative (SFARI) for use on their pioneering multi-site Autism study, the Simons Simplex Collection (SSC).  In that study more than 12 universities collected and analyzed data from over 2,700 families with at least one child affected by Autism.  The resulting genetic and phenotypic repository, known as SFARI Base, is underpinned by RexDB technology.  Since the SSC, RexDB has been adopted by leading academic research centers across the US to overcome the unique challenges of mental and behavioral health research.

 
Partners who have agreed to collaborate with Prometheus on the grant include the Yale University Child Study Center, the Marcus Autism Center, Weill Cornell Medical College, and the University of Missouri Thompson Center for Autism and Neurodevelopmental Disorders.  The NIH SBIR grant will support two years worth of research and development until the end of 2014.

 
“This software will allow researchers to make better use of scarce research dollars, and will help accelerate progress in understanding autism and other mental disorders,” Dr. Rozenblit added.  “We’re thrilled that the NIH has chosen to support the next implementation of RexDB.”

This research is supported by the National Institute Of Mental Health of the National Institutes of Health under Award Number R43MH099826. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

In this video, Charles gives a brief primer on how to use HTSQL directly to access your data.  The talk is intended for business analysts or researchers, not DBAs or software developers.  The true strength of HTSQL is that in ten minutes you could be querying your data with ease!

HTSQL Primer for Business Analysts from Prometheus Research, LLC on Vimeo.

In my last post, we learned about our first case study: a small fictional lab that, like many, has formidable data management challenges. In this post, we’ll get acquainted with the challenges confronting a fictional research center, examining its research, personnel, data management goals, and tools. But since we’re dealing with such a large operation, let’s first take a look at how it is structured, financed, and governed.

Organization and Governance
The Eating Disorders Research Center (EDRC) was founded 5 years ago with funds from a private, nonprofit eating disorders foundation. Its mission is to reduce the prevalence of eating disorders in the U.S. by advancing translational research and specialty care. The Center occupies an entire floor of a building in a teaching hospital and is affiliated with both the hospital and the psychiatry department of a large medical school. Its operating costs are paid for by a mixture of private donors, the hospital, and indirect costs from the federal research grants its PIs are awarded. Federal research grants and private donations fund its research.

The Board of Trustees, composed of prominent leaders from nonprofits, business, research, and the public, helps set the strategic direction of the center (see figure). The Center’s chief administrator is the Executive Director, who reports to the Board. Three other directors report to the Executive Director. Their titles reflect the primary administrative divisions of the EDRC: Director of Clinical Services, who oversees all clinics and medical staff; Director of Research, who oversees all research activities, PIs, and labs; and an Administrative Director who coordinates all in-house administrative departments and resources. The Center’s organizational ties to the hospital and medical school mean that they can utilize these institutions’ Information Technology (IT), electronic medical records (EMR), medical labs, and neuroimaging core, among other areas.

Research
According to the EDRC’s charter, its research portfolio spans the translational science continuum of basic, translational, and clinical research. In practice, research groups from these areas collaborate extensively with each other and with groups outside the Center to pursue complex, interdisciplinary research, such as large-scale observational or clinical trials. As such, the research involves many different data types, tools, and data-management practices. The Center has been around long enough now that they have completed several of these large-scale studies and are in the middle of several more. At any one time, the PIs and their colleagues are writing several papers based on these completed studies while coordinating ongoing research. The need to centralize and streamline study recruitment is a pressing issue that is acknowledged at all levels of the organization. Such processes are currently siloed within research divisions.

Personnel
The Research Division of the Center houses 10 principal investigators (PIs) who study different aspects of eating disorders. Each research division (basic, translational, and clinical) is led by one PI and has one data manager associated with it, who supports requests from the 2 to 4 research groups conducting research in that domain. In addition to the research arm, there is a Clinical Services Division composed of clinical providers, including physicians, psychologists, nurses, and health technicians. About half of the PIs in the Research Division provide care through the clinic.

Tools
Lab personnel use many technologies to manage data throughout its lifecycle. They collect and  manage data via a patchwork of paper forms, spreadsheets, cloud-based survey tools, and a locally based web and file server. Clinical providers use Epic, the hospital’s EMR system; some of the translational and clinical research staff use it as well, but it is not integrated with their data managements systems. Many of the research groups collect “big data” such as genomics, neuroimaging, and psychophysiological data using a diverse array of sensor hardware tied to equally diverse data processing pipelines.

Data Management Goals
The Center shares the same broad goals as DARP, the small research lab: accuracy, efficiency, security, and shareability. Unlike DARP, however, the Center has a formal center-wide data management plan that specifies its general data policies and governance. Additional goals are to facilitate shared access to data, deal with data requests, integrate clinical and research data, and create patient registries.

Discussion
As you might imagine, the EDRC faces formidable data management challenges. They must acquire, store, organize, clean, reuse, share, and archive their data while integrating across multiple data types, sites, and studies. Research-facing staff in the Center are acutely aware that they could be making much better progress toward this goal. Some of the research needs to securely interface with the EMR system. Because each lab might be running several research projects at the same time, they’re always advertising their studies and screening participants for eligibility both by phone and then in person. The PIs are often busy writing papers and grants and traveling to conferences to present their latest research, which means they’re routinely requesting new data sets from the center’s data manager or research assistants. The lab that primarily conducts clinical trials has a host of federal regulation and guidelines it must follow with respect to data management and reporting.

Clearly, an organization like this has a lot of issues to out, but it will be worth the effort: Because the center has such a high throughput of patients and heterogeneous data, any improvements they can make to their data management processes offer significant gains in research efficiency, data quality, staff morale, and return on investment. In other words, such improvements offer something for everyone in the organization.

What’s Next?
Now that we’ve been introduced to both the small lab and research center, we’re ready to discuss the data management lifecycle. That will prepare us for subsequent posts in which I will critically examine how each organization handles lifecycle phases and transitions — and how they could be improved.

In my inaugural post on RexDB.org, I shared my plan to ground this series on data management for scientific research in common real-world scenarios. In this post, I’ll get this started by introducing a small lab, the first of two fictional research organizations that will serve as data management case studies to illustrate the pros and cons of different approaches to data management. Let’s take a brief tour of the small lab’s research, personnel, data management goals, and tools.

Research
As its name implies, the Depression and Anxiety Research Program (DARP) lab conducts  basic and interventional research (clinical trials) on mood and anxiety disorders. The lab has completed enrollment and data collection on its first major research project, a 5-year R01 study funded by the National Institutes of Health (NIH), and has just received notice that it has been awarded an R01 to continue the research of the first grant. Both studies were conducted jointly by DARP and a similar lab at another major research university. Although both sites collect data for the two studies, DARP is responsible for managing it. The first study was a clinical trial that evaluated the relative efficacy of two types of psychotherapy among people with co-occurring anxiety and depressive disorders. The new study will follow up on the people who participated in the clinical trial to see which therapy was associated with the best long-term outcomes. Both studies involve many types of data, including self-report, clinical interview, observational, psychophysiological, and genetic data. In addition to the old and new R01s, there is always a steady stream of smaller research studies run by graduate students and postdocs in the lab.

Personnel
DARP’s organizational chart is shown below. Gretchen, the Principal Investigator and Lab Director, is an Associate Professor of Psychology at a major research university. She started the lab 7 years ago and is coming up for tenure review at the end of the year. (Odds are good that she’ll get it.) Her postdoc, June, oversees the day-to-day management of data and works closely with the two RAs. When Gretchen or June needs a data set to address a particular set of research questions, June is usually the one who puts it together within SPSS. Shawn, the undergraduate RA, manages data related to the administration of the study, such as phone screen, recruitment, enrollment, medication, and randomization logs. He does the bulk of the data entry and makes sure everyone else gives him complete data in a timely manner. Elaine is a second-year graduate student who is paid to work half-time as a graduate RA. Her primary responsibility is to make sure the data collected on the old R01 are clean. Both Elaine and Shawn, as well as the two remaining graduate students, Sam and Nancy, are involved in the massive data-cleaning effort and in doing phone screens to recruit subjects.

Tools
DARP is transitioning from the set of tools they used to manage data for the old R01 and those they plan to use for the new R01. For the old study, they collected most research data on paper, which then Shawn entered directly into SPSS files stored on a local file server. He has also been managing all of the study management data through a constellation of Excel spreadsheets whose idiosyncrasies are known only to him. On the new study, they plan to use electronic data capture (EDC) methods wherever possible. They are currently evaluating several web-based solutions that we’ll discuss at a later point.

Data Management Goals
Gretchen’s primary concern is with the accuracy, efficiency, security, and shareability of her data: accuracy, because she values scientific validity and knows that errors can lead to retractions; efficiency, because she doesn’t want data issues to get in the way of doing research or writing papers; security, because her lab collects sensitive psychiatric data that must remain private; and shareability, because she believes that data should be shared with other colleagues or research repositories in the interest of advancing science. The lab has done a good job keeping the data secure, but they have struggled with accuracy, efficiency, and shareability.

Discussion
DARP is a small lab involved in large-scale, complex research. Although their data management goals are clear and well-intentioned, they have much room for improvement in their approach to achieving them (and they know it). Fortunately, they are in a good position to reevaluate their approach as they get the new R01 started and prepare to write their first papers based on the old R01 data. The lab could improve their data management situation considerably by (1) centralizing the data from their recent clinical trial and forthcoming longitudinal study in an open-source relational database; (2) implementing data-cleaning processes that can be applied automatically to incoming data; and (3) writing and sharing a lab-wide data management plan. I’ll elaborate on these and other recommendations in future posts, as we help members of DARP find better ways to meet their data-management goals.

In the next post, I’ll introduce you to a large research center struggling to manage data across several labs, studies, and sites. From there, we’ll move on to an examination of different facets of data management at the lab and center.

RexDB will use a custom framework to automatically generate screens for data display and editing. We put together this diagram showing how it works.
Our auto-generated HTML and Javascript screens will depend on DBGUI (Database graphical user interface), which contains a library of standard UI elements. DBGUI is the intermediary between the screen and the HTSQL catalog, which is generated from the database schema and allows the screens to rapidly execute queries.
DBGUI will also rely on a set of standard screen templates. Each screen’s layout and functionality will be determined by a template configuration file, which can be easily updated by a designer or analyst at Prometheus. These configuration files may also contain custom HTML and Javascript screen elements.

My name is Frank Farach, and I recently joined Prometheus Research as a Staff Scientist. I’m thrilled to be blogging here about scientific data management. In my 12-year career as a clinical psychology researcher and statistical consultant at several top research institutions, I’ve learned that data management practices can profoundly affect the quality, efficiency, and utility of research. In fact, this realization played an important role in my joining Prometheus Research, an organization of talented folks who share my passion for improving research through better data management practices and tools, such as open-source RexDB.

Data management may be one of the least liked and most overlooked aspects of research, but it is integral to good science. And just like good science, it isn’t easy to do. Part of the problem is the big gap between the recommendations of best-practice guidelines for data management and the actual management of data. The “how” does not always accompany the “what.” On top of this, guidelines often focus on each stage of data management in isolation, which can miss systemic problems that emerge across multiple phases of data management. Also missing is a discussion of human and organizational factors in data management. (Hey, I’m a psychologist, what do you expect?)

In future posts, I plan to directly address some of these gaps between data management theory and practice. Rather than adopt a strictly top-down approach to data management guidance, I’ll first consider the needs of investigators and their stakeholders under common data management scenarios, paying particular attention to how data is actually being managed, warts and all. Once we understand the unique goals and constraints operating in each scenario, we’ll evaluate the pros and cons of different data management practices and discuss how open-source RexDB can be used to adopt the best approaches.

In the next post, we’ll dive right in by learning about the data-management goals of a typical research group struggling to manage data across several research studies.