In the last decade, the number of clinical trials including multicenter trials has increased worldwide and consequently, the related clinical data have increased and grown more complex in many types of sources. Furthermore, medical imaging involvement in the recent clinical trials is another main reason for increasing the intricacies of clinical data [1-3]. These increased clinical data help develop information technology (IT) systems such as electronic data capture (EDC) systems or clinical trials management systems in order to access and collect data efficiently [4]. As a result, a separate dedicated IT system for imaging data is required to manage data and control any risk from the clinical trials because of increased medical imaging usage for various imaging endpoints in multicenter trials.

Since image biomarkers have been used in a variety of clinical trials [5], the US Food and Drug Administration (FDA) has continuously emphasized on the importance of tumor response on medical images for regular approval of oncology drugs [6,7]. To provide instructions for the standardization process of image biomarker use in clinical trials, the FDA issued “Clinical trials Imaging Endpoint Process Standards Guidance for Industry” in 2018 (hereafter referred to as 2018 FDA imaging guidance) [8].

According to the FDA imaging guidance, the standardization of clinical trials imaging endpoints describes processes to manage imaging acquisition, quality check, anonymization, transfer, archive, quantitative image analysis, and independent blinded image review by multiple readers [9]. These processes must also comply with regulations and guidelines of clinical trials, including the Good Clinical Practice (GCP) guidelines [10] and Health Insurance Portability and Accountability Act (HIPAA) [8]. Both imaging processes and data should follow standard global data formats outlined by the Clinical Data Interchange Standards Consortium (CDISC), the Health Level 7, and the Digital Imaging and Communications in Medicine (DICOM) [9]. Furthermore, a diversity of parties such as pharmaceutical companies, contract research organizations (CROs), sites or hospitals, and central imaging readers should access the updated data in real time, especially for multicenter trials, to track the study and review the imaging data. These factors have necessitated image data integration and management, resulting in the development of an IT system for clinical trial imaging [11].

Recently, several different IT platforms have been developed specifically for clinical trials that enable integrated imaging data management and efficient imaging workflows [1,9]. Such platforms are referred to as Clinical Trial Imaging Management Systems (CTIMS). A typical CTIMS contains a Web-based EDC system with either a standalone or Web-based DICOM image viewer. CTIMS are expected to maintain clinical trial regulatory compliance, use globally standardized data formats, and be validated with GxP-based protocols, where G stands for “Good”; P stands for “Practice”; and x stands for the regulatory fields such as good clinical, laboratory, or manufacturing practices [2,12].

The GxP is a collection of quality guidelines and regulations to ensure medical product safety; their intended use; and quality processes during clinical development, manufacturing, and distribution [12-14]. Although many guidelines and regulations for clinical trials are issued, the most important guideline for the CTIMS and EDC system is GCP, which is provided by the International Conference on Harmonization (ICH) [10]. However, the ICH-GCP does not cover the detailed requirements for computerized systems. Instead, regulatory agencies [15,16] and international societies [17,18] provide standards to build and use computerized systems in clinical trials. Hence, the term “GxP compliance” includes various guidelines, including GCP, Good Clinical Laboratory Practice, and Good Manufacturing Practice.

Many studies have focused on CTIMS [9,11,19], but no study has thus far comprehensively discussed the regulations and guidelines related to CTIMS, particularly with regard to GxP compliance. Therefore, we developed a CTIMS, named the AiCRO system, designed to comprehensively meet the current regulatory guidelines and perform GxP-based computerized system validation. In this article, we aim to describe the methods, considerations, and recommendations for the development and validation of CTIMS from a range of different perspectives.

Based on the 2018 FDA imaging guidance, which is the most important regulation covering a wide range of imaging processes required in clinical trials, we extracted key functions of CTIMS to perform imaging processes and enhance efficient workflow in the clinical trials.

To systematically extract key GxP requirements, which are not described in the 2018 FDA imaging guidance, we thoroughly reviewed the related regulations/guidelines from regulatory agencies and international societies (Figure 1) as follows:

Based on extracted regulatory requirements, we designed the AiCRO system architecture and functional modules. Java and HTML5 programming languages were used to develop a platform-independent system, which can be executed on a standalone system or Web-based system regardless of the operating system. The image viewer in AiCRO system was developed using JDK [computer software] (Version 1.8.1. Redwood City, CA: Oracle Redwood Shores).

A multidisciplinary software development team was organized, including radiologists, imaging technicians, clinical trial specialists, regulatory medicine professionals, and IT engineers, who worked toward the same goal. We also recruited potential AiCRO system users, including medical researchers, and pharmaceutical company staff.

The AiCRO system was validated both internally and externally. We validated the AiCRO system internally using the V-model in accordance of the US FDA CFR 21 Part 820 and GAMP 5 guidelines, which are currently regarded as the global standards for computerized system validation for clinical trials (Figure 2) [17]. According to the GAMP 5 guidelines, the validation process of the AiCRO system included the installation qualification, operational qualification, performance qualification, and design qualification. Installation qualification refers to the verification of installation and configuration of software and hardware based on preapproved specification.

Operational qualification and performance qualification verify that a system operates and performs all activities under normal and challenging conditions. Design qualification verifies individual system designs for different projects to ensure that they proceed efficiently through the work process. Internal validations are to be performed once a year using at least 10 personal computers in Korea, the United States, China, Australia, and France. For test purposes, we built a sample project that requires typical work processes used in clinical trials for cancer. The AiCRO system was externally validated for global clinical trial use by a global quality assurance and auditing company. Use of a third party enabled different perspectives on our system, including those related to adherence to regulatory requirements.

The key regulatory requirements extracted through our systematic review of the FDA guidelines and regulations are summarized in Table 1. The functional requirements for imaging processes in clinical trials are discussed in detail in the 2018 FDA imaging guidance [8]. The ultimate goal of the guidance is to increase assurance of imaging data for analyzing drug efficacy. The guidance comprehensively streamlines imaging processes such as study startup, standardization of image acquisition/interpretation, site monitoring, reader management, document management, image data management, image analysis/review, and data report and export. Of these, several processes require the CTIMS functions, particularly for image data management such as anonymization, transfer, archiving and storage; image analysis/review such as image viewer, electronic case report form (eCRF), and independent review workflow; and the data report and export (Table 1).

The FDA has continuously emphasized and recommended compliance with regulations for computerized system use in the clinical trial, including the FDA 21 CFR part 11 and many other regulations, as illustrated in Figure 1. In addition, the CTIMS must comply with the clinical trial regulation such as privacy protection law. In order to statistically analyze the data with the standardized data structure at the end of the trial, CTIMS should adapt the CDISC format, as described in the Modules section below (Electronic Case Report Form Data Management).

The predominant factor in the system design was compliance with a diverse set of regulations. We developed the AiCRO system for imaging processes in clinical trials in strict accordance with the relevant guidelines, with a focus on FDA 21 CFR Part 11 and the FDA imaging guidance. AiCRO system is developed based on Server and Client system architecture, as illustrated in Figure 3. The server system was organized with the database server, a Web-Picture Archiving and Communication System (PACS) based on dcm4chee, the AP server for Web application of eCRF, and Network-Attached Storage (NAS) for data backup. The database server was defined to handle large DICOM image files. It archives only DICOM images and provides functional modules to upload and download images to rest application programming interface (API). On the other hand, the AP server archives thumbnails or image measurement values provided by reviewers and eCRF text data along with API for these processes.

The client system was designed with various modules to manage clinical information data. The system transfers the eCRF clinical data to an AP server via API and transfers images to the server through database API after the image data are deidentified. Reviewers can also check the uploaded DICOM image with the image viewer and analyze images to, for example, measure the tumor size or volume. These measurement values are delivered to the AP server by the client system. If the client system is installed on the user’s laptop, clinical data can be easily managed and images can be viewed and analyzed from any location. The screen snapshots of modules are presented in Figure 4.

For internal system validation, we organized the overall validation plan according to the V-model (Figure 2), which includes the purpose, process validation, test setting and methods, and schedule. All validation processes and results were documented.

For installation qualification, the AiCRO’s client system was successfully installed in all test personal computers, regardless of the operating system such as Windows and Mac as well as Web-browsers including Chrome, Internet Explorer versions 10 and 11, and Firefox. The AiCRO’s server system was installed appropriately on our institutional server and Amazon Web Services Clouds in Seoul and China (Beijing).

For operational qualification, the system developer created test scripts to prove the proper functioning of all functional specifications and system operation. Subsequently, an independent tester executed test scripts in a predefined testing environment and recorded the test log and all results including test date, observations made, completeness of each function, problems/deviations encountered, and other information relevant to the operational qualification. These assessments found that all AiCRO functions could be operated without difficulty.

For performance qualification, we prepared a sample project with real data and different user accounts. An independent tester then evaluated module function using the sample project with different amounts of imaging data, work load sizes, and computing and network environments. All functions of the AiCRO worked appropriately under normal operating conditions of minimum required personal computer specifications, and the upload and download of several computed tomography/magnetic resonance imaging scans was performed at usual internet speeds of over 10 megabytes per second. However, during testing under challenging conditions, limiting factors that might cause an interruption or significant delay in activity were identified, which were machines with less than 4 GB RAM, uploading and downloading large image data for at least 3000 DICOM files at one time, internet speeds below 10 MB per second, and viewing large image data for at least 3000 DICOM files at once.

For design qualification, AiCRO validation is essential because each clinical trial project will have a different study design. We checked whether AiCRO was able to create a customizable data entry form and working algorithm per user specifications. Using a test project that simulated a sophisticated clinical trial for stroke, our imaging clinical research associate wrote the user specification, and our system developer created the customizable project in the AiCRO system. Thereafter, a test team, including a project manager, image analyst, and radiologist, performed user acceptance tests. The customized test project was successfully created, and its functional modules garnered acceptance from different kinds of users.

Finally, the AiCRO system was validated by Zigzag (Berkshire, United Kingdom), an external global quality assurance company. A professional quality manager from Singapore visited the core lab in Seoul for 3 days and reviewed all validation documents. This manager also required a demonstration of the process, including creation of a new customized project and associated functions. The quality manager verified that installation qualification, operational qualification, performance qualification, and design qualification were performed appropriately in accordance with the relevant regulations and guidelines.

During the 2000s, the FDA [4] was concerned about the objectivity and validity of imaging endpoints, since there were several critical issues such as reliability of imaging biomarker and results, and appropriateness of imaging data management with regard to GCP, FDA guidelines, and HIPAA. The FDA addressed this issue by developing the Clinical Trial Imaging Endpoint Process Standards Guidance for Industry. The first draft guidance was issued in 2011, with a revision in 2015 and finalization in 2018 [8]. The guidelines emphasize adherence to regulations for imaging data collection, data transfer, and imaging analysis. At present, all stakeholders including pharmaceutical companies, imaging scientists and clinical trial professionals, and hospitals are expected to follow the 2018 FDA guidance to conduct clinical trials successfully.

CTIMS thus needed to comply with FDA guidelines, which comprehensively cover the functional and regulatory aspects of imaging, thereby underscoring the importance of developing a CTIMS that was GxP-compliant. The appropriate use of medical imaging in clinical trials necessitates specific considerations including standardization of imaging acquisition, archiving, and QC; centralized independent blinded image review; and systems that can handle complex workflows while maintaining regulatory compliance.

Asan Image Metrics (Seoul, Korea), an academic imaging core lab, developed a CTIMS—the AiCRO system—which adheres to all regulatory requirements for medical imaging in clinical trials and enables stakeholders to access easily medical images obtained from different sites. It contains modules including deidentification, data transfer, data archive, eCRF, image viewing/analysis, data management according to CDISC, audit trail, query/alarm, complex workflow algorithm, and security. The platform thus ensures quality of results and minimizes data risks during and after clinical trial periods. In the last 2 years, we have implemented the AiCRO system on more than 20 domestic and international pharmaceutical multicenter clinical trials, and several audits and inspections have been conducted as well.

The main advantage of AiCRO is that it is an all-in-one system that meets virtually all regulatory and functional requirements for clinical trial imaging while maintaining considerable user flexibility. For example, if a user needs a new function for the specific trial, our development team can easily build or customize the necessary function according to user specifications, after which our quality assurance and data management teams offer continuous user and regulatory support for the new function. For example, in the cases of trials on rare diseases requiring sophisticated magnetic resonance imaging analyses, a dedicated imaging postprocessing module and disease-specific terminology can be added in the CDISC data format.

Although our system has several advantages, one of the limitations is that each site or hospital has its own security system containing firewalls to protect their patients’ information. Nowadays, many institutions strictly regulate the external transfer of medical data, and hospital network system firewalls are becoming stronger. Ideally, a CTIMS should strike a balance between security and function within the boundaries of regulations and institutional policies. Another limitation is that AiCRO is an independent system separated from other EDC system. Thus, in clinical trials, we need to link our system to other EDC systems. Otherwise, users have to use two different IT systems in a clinical trial.

We introduced a CTIMS, called AiCRO system, to conduct clinical trials more efficiently in accordance with regulatory requirements. In this paper, we discussed the development of the AiCRO system to meet a diverse array of regulatory requirements and the design of multiple modules for users to customize the system to the needs of their clinical trials. We also discussed the internal and external validation of AiCRO according to the GxP guidelines and FDA 21 CFR. The AiCRO system is an all-in-one platform enabling high-quality clinical trial imaging data, but further study is required to describe the results of implementation obtained for different types of trials and any necessary systemic improvements.