The e-Solution Space Part 2: Automated Systems and the Investigator

[Reader note: This article was originally published by Clinical Researcher, Volume 3, Number 11, December 2003]

Sponsors have made efforts to install computer-based data collection and management systems at investigative sites for more than 20 years. As time marches on, such efforts are becoming more and more common. The reasons for this evolution are clear enough: efficiency, quality, and throughput can all be dramatically improved by employing automation. In spite of the varied track record of clinical trial automation systems to date, one of these systems is coming to an investigative site near you, probably in the near future. In this, the second part of a new series of articles in Clinical Researcher discussing the challenges of using information technology (IT) in clinical trials, particular areas of concern for investigators when automated systems are used are highlighted.

As I discussed in the first part of the series, when computer based data collection and management systems are well designed and properly configured, installed, and maintained, they can serve investigators and sponsors alike by providing a powerful tool for managing the complexities of the clinical trials process. Unfortunately, the opposite is often true. Many systems are so broken – due to flaws in their design, configuration, installation, and/or maintenance – as to make them unusable. In fact, such systems result in significantly increased investigative-site costs.

Don’t forget the investigator

Before identifying areas of risk for investigators, I’d like to take a brief detour into industry politics. In the late 1990s, I was involved in a complex Phase III oncology trial involving more than 100 investigators in more than 20 countries. The case report form was also complex. In fact, it took more than 6 months to get the electronic case report form (eCRF) approved by the sponsor. Here’s what happened:

The sponsor’s lead monitor ran the first round of form design meetings. After 3 weeks of progressive refinements, she was happy with the eCRF except for a very few minor changes that needed to be made, and a sign-off meeting was set for the next week. Unfortunately, the lead monitor could not attend that meeting. Not to worry: the medical director was going to attend the meeting – for the first time. He took one look at the forms and uttered words to the effect of: “These won’t do – all my disease data are missing.” Never mind that the protocol mentioned nothing of these data, they were data of scientific interest to him. The medical director became intimately involved in eCRF design until it reached the stage where he was happy with the eCRF except for a very few minor changes that needed to be made, and a sign-off meeting was set for the next week.

Of course, he didn’t make that meeting. But he did send out a memo to everyone who was involved in the project to come take a look at the final eCRF. The lead data manager – and only the lead data manager – showed up. He uttered words to the effect of: “This won’t do – our standard data management routines can’t handle these data.” And, yet another cycle began.

This sequence was eventually stopped with some very high-level involvement at both the sponsor and the vendor. I’ll never forget the words of the medical director at the final approval meeting: “It’s the best case report form I’ve ever seen, but this electronic data capture technology sure does cost a lot!”

As gross, exhaustive, and time-consuming as the process was, do you see what was utterly absent? Not once was the clinical researcher consulted. At least, not until the investigator meeting – at which point several valid criticisms about the eCRF were given. It is ironic that sponsors view investigators as a precious resource, but they rarely consider the impact of clinical trial automation technology on this resource.

Make no mistake, clinical trial automation technology does affect investigators. There are five broad topics of interest:

1. Installation and maintenance of system hardware/software
2. System performance
3. System usability
4. Training and support
5. System security

Each of these topics deserves a detailed discussion and this will be covered in greater depth later on in the series. But for now, I want to give you the hot-button issues to consider when you become involved in a trial that employs automation.

System hardware/software

Will dedicated hardware be installed? If so, then can you use your current workstation (ie, keyboard, mouse, and monitor) with it? If an entire workstation is installed, then you may have to find dedicated desk space for this, or you may be provided with a laptop. While laptops are great for traveling, they have significant drawbacks relating to ergonomics and performance. Also, laptops are a serious security risk: they are easy and desirable prey for thieves.

If the sponsor intends on using your existing hardware, then who is responsible for maintaining it? Who is responsible for the interactions between your software and the clinical trial software? Who will maintain the system and fix it when it breaks? What is the frequency and difficulty of maintenance? Surely, research coordinators ought not to be expected to act as local IT managers.

System performance

Simply put, how long does it take for the system to display an eCRF? I am aware of several systems that take tens of seconds. This kind of performance may be acceptable for browsing a web site for 10 minutes. It is utterly unacceptable for entering 5,000 case report forms and 15,000 data clarification forms. Failing to recognize this will result in a significant increase in research coordinator hours.

System usability

Closely related to performance is usability. Unfortunately, this topic is highly subjective and complex. Here are a few points to consider:

• Is the eCRF logical and appropriate? How many warnings are issued during data entry?
• Is navigation intuitive? Is the context of the current eCRF always clear?
• Can the display be scaled to different screen resolutions?
• Are user interface objects intuitive (good) or techie (bad)?
• Are there a few, simple paradigms to learn (good), or many, complex paradigms to learn (bad)?
• Is it easy to describe what’s on the display in words? (This will become important during support calls.)
• Can the system be operated without a mouse? (This allows proficient users to dramatically increase input efficiency.)

Training and support

Experienced users will require little training in order to use good systems. Bad systems and/or inexperienced users require better training. Failing to accept this corollary will result in poor quality data, increased coordinator, monitor, and data manager man-hours, late data, and perhaps even protocol violations. In addition to a well-planned syllabus, good training must occur just before the trainee begins to actually use the system and it should include the following:

• one-on-one training time
• practice time
• measured proof of proficiency
• IT support must be available. Answering incoming calls within three rings, 24–7 is irrelevant if the caller is deposited into a 20-minute queue. Instead, either immediate or callback response should be offered.

Security

In order to be useful, clinical trial automation systems must communicate over a network. Any system connected to any network is vulnerable to attack and compromise. And if that network is the Internet, then that vulnerability is all the more marked. While all automation vendors assure that their servers are highly protected, many pay no attention whatsoever to the Web browser used. This is incredible, for compromising an unprotected browser is a simple matter. Once compromised, data can be snooped in on and/or corrupted. This results in the violation of various regulations, the system is opened to industrial espionage, and patient confidentiality is compromised. Perhaps, most importantly, this could result in invalid research with catastrophic consequences. In fact, security is so critical – and, today, so overlooked – that one paragraph simply won’t do. To be continued.