The e-Solution Space Part 5: The Benefits of Automation

[Reader note: This article was originally published by Clinical Researcher, Volume 4, Number 4, April 2004]

In the first four parts of “The e-Solution Space”, several key problem areas that arise when employing clinical trial automation technology at the investigator site were discussed. Now that the critical – although certainly not all – problems have been touched upon, it’s time to begin an investigation of the benefits that automation technology can offer. Vendors and visionaries alike make grandiose claims about how efficiency, quality, and throughput can all be dramatically improved by employing automation. In this article, we start to consider the benefits of employing automation in clinical trials.

It’s all about the data

Viewed from the sponsor’s perspective, a clinical trial is a large, geographically dispersed, data collection and management project. Of course, there is interesting science behind this relatively mundane activity. However, once the decision to execute a trial has been made, the substantial majority of man-hours are spent in the “protocol factory”, which takes subjects in the front door and delivers completed case report forms (CRFs) out the back door. Like any factory, there are two core activities:

• people or machines do work to produce the product
• other people do work to manage the production process
• The discussion that follows is limited to the first activity.

Today’s state of affairs

Before arguing how automation can make things better, let’s briefly recap the data management process used in most trials today. The clinical research monitor represents the only project team member that investigators interact with on a regular basis. However, behind the monitor, there is a whole project team working in concert towards one end: a clean, closed, and locked database. A typical clinical trial project team, not including investigator personnel, consists of the following:

• a medical director
• a project manager, perhaps with an assistant
• a lead monitor
• one additional monitor for every 5–10 sites¹
• a lead data manager
• one or more additional data managers
• a lead statistician
• one or more statistical programmers
• a safety monitor

For a typical 30-site Phase III trial, that would be a total of about 13 team members. The duties of two of them – the medical director and the safety monitor – have a significant clinical component. For everybody else, it’s all about the data. Why is this important to the investigator? For this simple fact: each one of these team members spends a significant part of their working day looking at everything that you – the investigator – do, through the prism of the data that you feed them.

The monitor is closest to the investigator. During on-site visits, the monitor checks the data for various kinds of problems, including:

• legibility
• completeness (eg, if “race, other” is checked, then a race is filled in)
• consistency (eg, no pregnant males)
• protocol violations (eg, accepting a patient that shows an exclusion criterion)
• consistency with the source documentation

Often, monitors accomplish these activities in difficult working environments (eg, nonexistent or crowded desk space, and distracting or foreign surroundings) and under exhausting conditions. Site-visits, by definition, do not occur in the comfort of the monitor’s office. It is not uncommon for 80% of a monitor’s time to be spent on the road, and we all know how relaxing and restful travel is these days. Under such conditions, it’s a tribute to monitors, throughout the world, that studies are accomplished with such professionalism.

The output produced by the monitor (ie, CRF pages) is used as input by data managers. First, each CRF page is keypunched into a computer: usually twice, by two different people, with yet a third person “refereeing” discrepancies. Legibility issues that were either understood or missed by the monitor now become a major problem.

Once the data are entered into the computer database, the data managers go to work. Data managers inspect the data for all of the problems that concern the monitor, except legibility, but from a very different perspective. First, monitors are more “clinically” oriented. They tend to view the data from the viewpoint of medical expertise. On the other hand, data managers are more “logically” oriented. They tend to view the data from the perspective of predicate logic. Naturally, there are some very “logically oriented” monitors just as there are some very “clinically oriented” data managers, but the point is that these viewpoints are different and consequently find different problems.

Second, data managers have a tool available that monitors do not – a computer. Since the data manager is inspecting data in a database, aggregate measurements of the data can be taken. Such measurements are not available to the monitor. An aggregate measurement calculates some metric on a set of data rather than a single data point. For example, observing that a systolic blood pressure is 80 by observing just that one measurement is a non-aggregate metric. Observing that subject “AAA:101” has a systolic blood pressure that is among the 10 lowest systolic blood pressures of all of the subjects in the study is an aggregate metric. In this case, because the data manager knows that the blood pressure is unusual, he/she is prepared to look for a problem. Therefore, when the data manager looks at “80/120”, he/she immediately sees the issue—that the reading has been recorded wrongly and it should be 120/80. Contrast this to the monitor, who is reviewing the blood pressure having already reviewed 200 pages of data after 5 hours of sleep due to a canceled flight. What blood pressure do you think the monitor is likely to read?

Let us now turn to biostatistics. Two additional team members take yet another look at the data – the statistical programmers and the biostatistician. Again, each of these people has a slightly different view of the data. Invariably, the programmer finds further problems with the data while debugging statistical programs. Similarly, the statistician finds problems when evaluating various statistical measurements.

What’s wrong with this picture

There are several critical deficiencies in the process outlined above. The process indicates a “chain of custody” of data, which goes from: investigator to monitor, to keypuncher, to data manager, to statistical programmer, to statistician. Now, consider each link in the chain (see Table 1. Each link of the chain:

• works with a different data source
• is further and further removed geographically from the data source
• is further and further removed chronologically from the data source

Add the punch line from the investigator’s point of view: every problem with the data results in a query sent back to the investigator. Wow, what a nightmare! Clearly, the current process indicates major deficiencies. Proper application of automation can dramatically improve this state of affairs.

Imagine

Imagine a clinical trial where the complete trial team is present at each subject visit. The subject walks into the examination room, followed by the investigator and the site coordinator – so far, so good. Oops, here comes the monitor, the data manager, the statistical programmer, and the statistician. Let’s even throw in the project manager and medical director for good measure. The exam room is crowded. The investigator is irritated. The subject is horrified!

Protocol-directed procedures begin. Every measurement taken by the investigator is checked and double-checked by each, and every, team member assembled in the examination room. Protocol questions can be answered immediately. Data errors are identified and rectified. When the subject-visit is complete, the data are spotlessly clean. Weeks, months, and years later, the investigator will never, ever be asked to recall this patient-visit. The event is perfectly recorded for all eternity: a recording never to be questioned. In fact, under such a scenario the investigator’s work during a clinical trial has been reduced to one task: to practice medicine. The patient is still horrified, but a sense of peacefulness returns to the investigator.

Turn fantasy into reality

Careful employment of automation can produce a virtual world that is nearly identical to the fantasy described above. What if the data were entered into a computer at the examination site after the investigator completed a procedure, the same day the procedure occurred? (We can do better than this, but let’s start here.) In this scenario, paper CRFs do not exist; the normal patient chart represents the only paper that ever exists. Each team member is given two things:

• access to the clinical data.
• a method to communicate to each and every other team member

Ponder such a reality. How can this be accomplished? What are the implications?

To be continued.

¹The monitor to site ratio varies substantially between trials. In reality, monitors are (or ought to be) distributed on the basis of a ratio of monitors to CRF pages.

Table 1. The current “chain of custody” model of data management.