Original Paper
Abstract
Background: With the rapid increase in utilization of imaging endpoints in multicenter clinical trials, the amount of data and workflow complexity have also increased. A Clinical Trial Imaging Management System (CTIMS) is required to comprehensively support imaging processes in clinical trials. The US Food and Drug Administration (FDA) issued a guidance protocol in 2018 for appropriate use of medical imaging in accordance with many regulations including the Good Clinical Practice (GCP) guidelines. Existing research on CTIMS, however, has mainly focused on functions and structures of systems rather than regulation and compliance.
Objective: We aimed to develop a comprehensive CTIMS to meet the current regulatory guidelines and various required functions. We also aimed to perform computerized system validation focusing on the regulatory compliance of our CTIMS.
Methods: Key regulatory requirements of CTIMS were extracted thorough review of many related regulations and guidelines including International Conference on Harmonization-GCP E6, FDA 21 Code of Federal Regulations parts 11 and 820, Good Automated Manufacturing Practice, and Clinical Data Interchange Standards Consortium. The system architecture was designed in accordance with these regulations by a multidisciplinary team including radiologists, engineers, clinical trial specialists, and regulatory medicine professionals. Computerized system validation of the developed CTIMS was performed internally and externally.
Results: Our CTIMS (AiCRO) was developed based on a two-layer design composed of the server system and the client system, which is efficient at meeting the regulatory and functional requirements. The server system manages system security, data archive, backup, and audit trail. The client system provides various functions including deidentification, image transfer, image viewer, image quality control, and electronic record. Computerized system validation was performed internally using a V-model and externally by a global quality assurance company to demonstrate that AiCRO meets all regulatory and functional requirements.
Conclusions: We developed a Good Practice–compliant CTIMS—AiCRO system—to manage large amounts of image data and complexity of imaging management processes in clinical trials. Our CTIMS adopts and adheres to all regulatory and functional requirements and has been thoroughly validated.
doi:10.2196/14310
Keywords
Introduction
Background
In the last decade, the number of clinical trials including multicenter trials has increased worldwide and consequently, the related clinical data have increased and grown more complex in many types of sources. Furthermore, medical imaging involvement in the recent clinical trials is another main reason for increasing the intricacies of clinical data [
- ]. These increased clinical data help develop information technology (IT) systems such as electronic data capture (EDC) systems or clinical trials management systems in order to access and collect data efficiently [ ]. As a result, a separate dedicated IT system for imaging data is required to manage data and control any risk from the clinical trials because of increased medical imaging usage for various imaging endpoints in multicenter trials.Since image biomarkers have been used in a variety of clinical trials [
], the US Food and Drug Administration (FDA) has continuously emphasized on the importance of tumor response on medical images for regular approval of oncology drugs [ , ]. To provide instructions for the standardization process of image biomarker use in clinical trials, the FDA issued “Clinical trials Imaging Endpoint Process Standards Guidance for Industry” in 2018 (hereafter referred to as 2018 FDA imaging guidance) [ ].According to the FDA imaging guidance, the standardization of clinical trials imaging endpoints describes processes to manage imaging acquisition, quality check, anonymization, transfer, archive, quantitative image analysis, and independent blinded image review by multiple readers [
]. These processes must also comply with regulations and guidelines of clinical trials, including the Good Clinical Practice (GCP) guidelines [ ] and Health Insurance Portability and Accountability Act (HIPAA) [ ]. Both imaging processes and data should follow standard global data formats outlined by the Clinical Data Interchange Standards Consortium (CDISC), the Health Level 7, and the Digital Imaging and Communications in Medicine (DICOM) [ ]. Furthermore, a diversity of parties such as pharmaceutical companies, contract research organizations (CROs), sites or hospitals, and central imaging readers should access the updated data in real time, especially for multicenter trials, to track the study and review the imaging data. These factors have necessitated image data integration and management, resulting in the development of an IT system for clinical trial imaging [ ].Recently, several different IT platforms have been developed specifically for clinical trials that enable integrated imaging data management and efficient imaging workflows [
, ]. Such platforms are referred to as Clinical Trial Imaging Management Systems (CTIMS). A typical CTIMS contains a Web-based EDC system with either a standalone or Web-based DICOM image viewer. CTIMS are expected to maintain clinical trial regulatory compliance, use globally standardized data formats, and be validated with GxP-based protocols, where G stands for “Good”; P stands for “Practice”; and x stands for the regulatory fields such as good clinical, laboratory, or manufacturing practices [ , ].The GxP is a collection of quality guidelines and regulations to ensure medical product safety; their intended use; and quality processes during clinical development, manufacturing, and distribution [
- ]. Although many guidelines and regulations for clinical trials are issued, the most important guideline for the CTIMS and EDC system is GCP, which is provided by the International Conference on Harmonization (ICH) [ ]. However, the ICH-GCP does not cover the detailed requirements for computerized systems. Instead, regulatory agencies [ , ] and international societies [ , ] provide standards to build and use computerized systems in clinical trials. Hence, the term “GxP compliance” includes various guidelines, including GCP, Good Clinical Laboratory Practice, and Good Manufacturing Practice.Objectives
Many studies have focused on CTIMS [
, , ], but no study has thus far comprehensively discussed the regulations and guidelines related to CTIMS, particularly with regard to GxP compliance. Therefore, we developed a CTIMS, named the AiCRO system, designed to comprehensively meet the current regulatory guidelines and perform GxP-based computerized system validation. In this article, we aim to describe the methods, considerations, and recommendations for the development and validation of CTIMS from a range of different perspectives.Methods
Extraction of Regulatory Requirements
Based on the 2018 FDA imaging guidance, which is the most important regulation covering a wide range of imaging processes required in clinical trials, we extracted key functions of CTIMS to perform imaging processes and enhance efficient workflow in the clinical trials.
To systematically extract key GxP requirements, which are not described in the 2018 FDA imaging guidance, we thoroughly reviewed the related regulations/guidelines from regulatory agencies and international societies (
) as follows:- Regulatory agency: ICH-GCP E6 (R2) [ ], HIPAA, FDA Guidance for Computerized System Used in Clinical Trials [ ], or 21 Code of Federal Regulations (CFR) Part 11 [ ], etc.
- International society: Good Automated Manufacturing Practice (GAMP) 5 Guide for Compliant GxP Computerized Systems [ ], European Clinical Research Infrastructures Network standard [ ], good practice for computerized systems in regulated GxP environments, Pharmaceutical Inspection Co-operation Scheme Inspectors Guide [ ], etc.
Development of AiCRO System
Based on extracted regulatory requirements, we designed the AiCRO system architecture and functional modules. Java and HTML5 programming languages were used to develop a platform-independent system, which can be executed on a standalone system or Web-based system regardless of the operating system. The image viewer in AiCRO system was developed using JDK [computer software] (Version 1.8.1. Redwood City, CA: Oracle Redwood Shores).
A multidisciplinary software development team was organized, including radiologists, imaging technicians, clinical trial specialists, regulatory medicine professionals, and IT engineers, who worked toward the same goal. We also recruited potential AiCRO system users, including medical researchers, and pharmaceutical company staff.
Computerized System Validation
The AiCRO system was validated both internally and externally. We validated the AiCRO system internally using the V-model in accordance of the US FDA CFR 21 Part 820 and GAMP 5 guidelines, which are currently regarded as the global standards for computerized system validation for clinical trials (
) [ ]. According to the GAMP 5 guidelines, the validation process of the AiCRO system included the installation qualification, operational qualification, performance qualification, and design qualification. Installation qualification refers to the verification of installation and configuration of software and hardware based on preapproved specification.Operational qualification and performance qualification verify that a system operates and performs all activities under normal and challenging conditions. Design qualification verifies individual system designs for different projects to ensure that they proceed efficiently through the work process. Internal validations are to be performed once a year using at least 10 personal computers in Korea, the United States, China, Australia, and France. For test purposes, we built a sample project that requires typical work processes used in clinical trials for cancer. The AiCRO system was externally validated for global clinical trial use by a global quality assurance and auditing company. Use of a third party enabled different perspectives on our system, including those related to adherence to regulatory requirements.
Results
Key Regulatory Requirements
The key regulatory requirements extracted through our systematic review of the FDA guidelines and regulations are summarized in
. The functional requirements for imaging processes in clinical trials are discussed in detail in the 2018 FDA imaging guidance [ ]. The ultimate goal of the guidance is to increase assurance of imaging data for analyzing drug efficacy. The guidance comprehensively streamlines imaging processes such as study startup, standardization of image acquisition/interpretation, site monitoring, reader management, document management, image data management, image analysis/review, and data report and export. Of these, several processes require the CTIMS functions, particularly for image data management such as anonymization, transfer, archiving and storage; image analysis/review such as image viewer, electronic case report form (eCRF), and independent review workflow; and the data report and export ( ).The FDA has continuously emphasized and recommended compliance with regulations for computerized system use in the clinical trial, including the FDA 21 CFR part 11 and many other regulations, as illustrated in
. In addition, the CTIMS must comply with the clinical trial regulation such as privacy protection law. In order to statistically analyze the data with the standardized data structure at the end of the trial, CTIMS should adapt the CDISC format, as described in the Modules section below (Electronic Case Report Form Data Management).Imaging process | Regulatory requirements | Functional requirement | |||
Study startup | |||||
Imaging protocol set up and imaging charter development | Validation:
| Provision of templates:
| |||
CTIMS set up | Validation:
| User requirement specification:
| |||
Image data management | |||||
Deidentification | Deidentification:
| DICOMb file deidentification:
| |||
Transfer/archiving/storage archiving/storage | Back up:
| Secure file transfer function:
| |||
Image QCc | Standards for image acquisition:
| Image QC:
| |||
Image analysis/review | |||||
Image viewer | Copies of records and record retention:
Digital signature:
| DICOM image viewer:
| |||
Electronic CRFd | Copies of records and record retention:
Digital signature:
| Image CRF:
| |||
Centralized imaging review workflow | Security system (authorized access):
| Customizable workflow:
| |||
Report and export | |||||
Tracking report | Audit trail:
| Master tracking report:
| |||
Data export | CDISCe:
| Review data export:
|
aCTIMS: Clinical Trial Imaging Management System.
bDICOM: digital imaging and communications in medicine.
cQC: quality control.
dCRF: case report form.
eCDISC: Clinical Data Interchange Standards Consortium.
fDM: Data Management.
Overall AiCRO System Architecture
The predominant factor in the system design was compliance with a diverse set of regulations. We developed the AiCRO system for imaging processes in clinical trials in strict accordance with the relevant guidelines, with a focus on FDA 21 CFR Part 11 and the FDA imaging guidance. AiCRO system is developed based on Server and Client system architecture, as illustrated in
. The server system was organized with the database server, a Web-Picture Archiving and Communication System (PACS) based on dcm4chee, the AP server for Web application of eCRF, and Network-Attached Storage (NAS) for data backup. The database server was defined to handle large DICOM image files. It archives only DICOM images and provides functional modules to upload and download images to rest application programming interface (API). On the other hand, the AP server archives thumbnails or image measurement values provided by reviewers and eCRF text data along with API for these processes.The client system was designed with various modules to manage clinical information data. The system transfers the eCRF clinical data to an AP server via API and transfers images to the server through database API after the image data are deidentified. Reviewers can also check the uploaded DICOM image with the image viewer and analyze images to, for example, measure the tumor size or volume. These measurement values are delivered to the AP server by the client system. If the client system is installed on the user’s laptop, clinical data can be easily managed and images can be viewed and analyzed from any location. The screen snapshots of modules are presented in
.Modules
Image Upload With Deidentification
Every medical image acquired from the participating sites should be uploaded to the AiCRO system in DICOM format by each site’s staff. Uploaded image data are anonymized, transferred, and archived in the central server of CTIMS [
]. Deidentification of medical images is one of the most important components for GxP compliance with regard to GCP and the HIPAA privacy guideline [ , ].Personal identification information is recorded in the DICOM file metadata or presented in the medical images as graphical information. The metadata contains both patient identification information such as patient name; date of birth; hospital name; and hospital medical record number; and imaging modality/protocol information such as manufacturer and software of the modality, sequence, field of view, or slice thickness. According to the DICOM standard part 15 [
], the AiCRO system extracts DICOM elements containing personal identification information, automatically removes the extracted information, and asks uploaders to enter alternative information such as the subjects’ screening number or clinical participant code ( and ). However, some imaging machines also use private DICOM elements to store complementary information such as patient or physician initials, telephone number, or national identification number. We created a function that would detect any potential patient identification information from DICOM metadata by using keywords and number formats.Besides metadata in the DICOM file, patients’ identification information is presented on medical images as graphical texts or image pixel data, especially in the ultrasonography, 3D rendering computed tomography images, radiation dose reports, screen shot images, and endoscopy images. We implemented two functional modules to remove this type of information by incorporating the fully automatic text-removal function and semiautomatic imaging-processing blackout function.
Specifically, graphical patient identification information was detected through the incorporation of optical character recognition, which is based on the convolutional recurrent neural network algorithm to detect text burned in the images [
]. Subsequently, we used a rule-base text recognition module to detect patients’ name, date of birth, medical record number, and institution name. However, it might not be accurate and therefore requires the uploaders to check the deidentified images. If further action is required, a manual imaging-processing function can be applied to hide such information.Action code | Intended action |
D | Metadata of name and IDa are replaced with word “DE-IDENTI.” |
R | Metadata of scan/birth date are replaced with format “0000-0000.” |
E | The original metadata are removed. |
C | The original UIDb metadata are replaced with new UID. |
N | A specific word is replaced with a new dedicated word. |
aID: identification.
bUID: user identification.
Attribute name | Tag | Action code | Example |
Patient name | 0010, 0010 | N(Initial) | Jhon Doe → patient01 |
Media storage SOPa instance UIDb | 0002, 0003 | C | 1.3.12.2.1107... → 1.2.410.200... |
SOP instance UID | 0008, 0018 | C | 1.3.12.2.1107... → 1.2.410.200... |
Accession number | 0008, 0050 | E | 00009 → Remove |
Institution name | 0008, 0080 | D | AMCc → “DE-IDENTI” |
Institution address | 0008, 0081 | E | 88, Olympic-ro 43-gil...→ Remove |
Referring physician name, | 0008, 0090 | E | Jane Doe → Remove |
Performing physician name | 0008, 1050 | E | Jane Roe → Remove |
Patient IDd | 0010, 0020 | N(Subject NO) | 11112222 → Subject01 |
Patient’s birth date | 0010, 0030 | R | 19890215 → 19890101 |
Other patient names | 0010, 1001 | E | Jhon Roe → Remove |
Other patient IDs | 0010, 1002 | E | 22223333 → Remove |
Patient address | 0010, 1040 | E | 17, Misagangbyeon… → Remove |
Study ID | 0020, 0010 | N(Project ID) | 33334444 → Project01 |
Study instance UID | 0020, 000D | C | 1.3.12.2.1107... → 1.2.410.200... |
Series instance UID | 0020, 000E | C | 1.3.12.2.1107... → 1.2.410.200... |
aSOP: standard operating procedure.
bUID: user identification.
cAMC: Asan Medical Center.
dID: identification.
The semiautomatic and manual blackout deidentification functions enable uploaders to select regions of interest that need to be removed (so-called blackout regions) either in a single image or multiple series of images (
). If the blackout function is successfully applied, the processed images are stored in new anonymized DICOM files and then uploaded into the AiCRO system.Centralized Imaging Data Storage and Security
Data storage and security are other significant points emphasized by FDA guideline 21 CFR part 11. Hence, all DICOM files are centralized and stored on the AiCRO server system (
). Web-PACS of AiCRO system was developed using the dcm4che library [ ], and Oracle 9i was developed for storing DICOM files. The eCRF data are stored in the AP server, which was built with HTML 5.0 and the MySQL 4.1 database. Two separate databases were used because, although Oracle enables large-scale databases to store large DICOM files, MySQL is better for managing smaller amounts of data such as eCRF text data.Most hospitals and institutions use DICOM metadata tagging in accordance with DICOM standards, but specific information therein may vary across sites depending upon the institutional policy. If the metadata does not follow standard DICOM protocols, image upload and view may be disabled by the Web-PACS. We resolved this issue by developing a custom DICOM library for the standardization of DICOM metadata. The custom DICOM library enables creation of new metadata by modifying the pre-existing DICOM metadata. These changes are stored as per the custom DICOM library, which must be regularly updated to apply to different or emergent circumstances.
Regarding a system security to prevent different types of system attack, we established an IT system security plan. For example, separation of the AP server and database server can localize any damage in the server. To protect the local computer, we adopted a zero footprint design and symmetric encryption algorithms based on advanced encryption standards. Images can be only loaded into our custom-built DICOM viewer due to the encryption key, and there are no plugins or data in the local personal computer.
To protect against any unforeseen disaster (fire, inundation, or hack), backup data are stored regularly with a dedicated program using NAS hardware physically located at different sites. When any issue arises, the server administrator receives an automatic email alert with the relevant information for troubleshooting. Backup data include the database, audit trails, DICOM, and configuration files. These overall securities allow us to guarantee data integrity.
Image Quality Control
All uploaded medical images should pass through quality control (QC) before image analysis in accordance with the FDA imaging guidance [
]. The image QC module in the AiCRO system can be customized according to the predefined imaging acquisition protocols and QC criteria for each clinical trial [ ].Our image QC module is composed of an automatic quantitative QC module and a manual qualitative QC module. The automatic quantitative QC module extracts several DICOM elements containing imaging modality/protocol information such as machines, sequences, matrix size, spatial resolution, field of view, slice thickness, and interslice gap. The module automatically decides whether the extracted values meet the predefined image quality requirements. For instance, if a trial requires axial and coronal computed tomography images with a matrix size of 512 × 512 and slice thickness of ≤5 mm with 0 interslice gap, the automatic quantitative QC module checks whether the uploaded images meet those specific requirements.
Image QC staff is involved in the manual qualitative QC module. The AiCRO system supports image QC staff to perform qualitative image quality checks for presence of image artifacts, appropriateness of scan coverage, image reconstruction, and contrast enhancement. The image QC staff can generate the quality assessment report after the QC check (
). The quality assessment report includes the quantitative QC results, qualitative QC results, overall quality to decide pass or fail, detailed information on any protocol deviations, queries for corrective action (if necessary), and query resolution.Image Viewer
The AiCRO image viewer was developed to provide a viewer platform, interfaces, and image processing tools [
]. Our viewer supports DICOM files obtained from common image modalities such as ultrasound, endoscopy, computed tomography, magnetic resonance, or positron emission tomography. The configuration function is provided to help the user quickly configure viewer setting customized for the trial with encrypted XML file. The time for image loading is minimized by using multithread processing, as this enables viewing images while others are loading, eventually reducing working time. We also incorporated a multiplatform function into the AiCRO image viewer, enabling both standalone and Web-based image viewing, using Java to develop platform-independent software that can be executed on Windows or Linux operating systems using Java runtime environment or on a Web-based viewer using Java Web Start.As there are already established image processing tools in most image viewers (
) [ ], we made all such tools available in the AiCRO image viewer; this viewer provides central image reviewers a range of tools for manipulating images in the AiCRO system. Measurement tools for region of interest (ROI) are also implemented, assisting reviewers in measuring key areas on uploaded images. Once measurements are made, the ROI image and measure value can be automatically saved and recorded into the imaging CRF in the AiCRO system. Since the ROI image and measure value are saved in the system, subsequent reviewers can retrieve this information for reference purposes and analysis of additional images. Other necessary reviewing functions include image contrast control and zoom. We also developed a multipotent imaging postprocessing software, called Asan-J software, that enables important postprocessing functions such as image registration, subtraction, semiautomatic segmentation, image classification, 3D rendering, and functional image analysis.Function | Detail |
Scrolling and positioning |
|
Metadata |
|
Information overlay |
|
Windowing |
|
Measurements |
|
Histogram |
|
Pseudocolor |
|
3D image viewer |
|
Electronic Case Report Form Data Management
All textual data including subject information and image analysis and review results in the AiCRO system adhere to CDISC standards for data formats for multicenter and multinational trials, a prerequisite for FDA submission. The FDA insists on the use of study data standards for data consistency, particularly in computer systems [
- ].A diverse and customized eCRF, also known as an imaging CRF including Response Evaluation Criteria in Solid Tumors (RECIST) 1.1, Immune RECIST, or Lugano classification, was developed to retain records of all data related to study images. Customized eCRFs were created for different study designs by anatomical location of tumor, tumor measurement, or type of drug or analysis, leading to inherent variations in data formats. We streamlined data formats according to the CDISC terminology for each therapeutic area, such as cardiology, neurology, or oncology, so that all data are collected using standardized CDISC terminology from the start to enable data management across the sites. These data collection process adhere to the Clinical Data Acquisition Standards Harmonization of CDISC standards [
].Collected data in the system need to be organized to standard structures for submission to regulatory agencies, as discussed in the CDISC standards, in the study data tabulation model [
]. Implementing an appropriate data standard for data collection and export allows for a variety of perspectives on clinical and nonclinical trial data. At the end of the trial, standardized data save time in terms of converting the data for statistical analysis and compiling a final report to submit to the FDA. Creation of standardized analysis datasets for metadata is included in the Analysis Data Model of CDISC standards [ ].AiCRO complies suitable CDISC terminology for all these processes (
), which are already set up for eCRF from the start of the trial and manage data from all sites for standardization. CDISC-controlled terminology transforms vocabulary used in clinical trials such as demographic information, adverse events, and medical examination from study protocols to CDISC codes or values. Consistency is critical when developing data integration systems, and all data in the AiCRO system are optimized for submission to the FDA in the United States and Pharmaceuticals and Medical Devices Agency in Japan without any additional processing.Electronic Signature
As the AiCRO system maintains electronic data, data validation is an important consideration for clinical trials. Data validation in an electronic system is mandated by regulations specific to electronic signatures. We thereby incorporated an electronic signature function into the system, as per FDA 21 CFR part 11, wherein electronic signatures must be “the equivalent of handwritten signatures, initials, and other general signings required by predicate rules” [
].Audit Trail
FDA 21 CFR part 11 [
] stipulates that all data are to be recorded with computer-generated, time-stamped audit trails representing date, time, editor, and any subsequent modifications. Thus, we provide for this in the AiCRO system. The guideline also emphasizes that, as in handwritten medical records, any changes to data must not obscure previous data, and the AiCRO audit trails are thus situated: Audit trails are generated whenever data are created, modified, or deleted during the clinical trial period and after trial closeout. These audit trail data can be extracted upon requests for inspection of records by other stakeholders such as pharmaceutical companies, CROs, or ministries of health.Query and Alarm
The query and alarm functions are designed to adhere to GAMP guidelines for corrective and preventive action (CAPA). CAPA outlines a process for the investigation, discussion, and correction of any problems as well as recognition and prevention of potential problems [
]. When designing a system suitable for use in clinical trials, the AiCRO system CAPA utilized the term “Query,” as it is commonly used in such trials. Generated queries from the AiCRO system fall into two categories: system problems such as image upload issues and clinical trial problems such as missing data.Appropriate data entry is key in clinical trials and usually managed via “edit check,” a process for the detection and correction of data when they are first entered [
]. The “Query” function is engaged during the QC process in reference to image quality or mismatch of imaging modality or protocol and during monitoring by the clinical research associate in cases of site data omission from the CRC or central reviewers. The function sends the generated query to the person responsible for the trial to allow him/her to resolve issues and is structured similarly to commonly used email systems.Role and Responsibility of Account
Accounts for accessing the AiCRO system are organized into different types depending on the individual’s role, such as site CRC, image QC, central reviewer, and many others. According to GAMP guidelines, definitions of the roles and responsibilities attached to each account are recommended for validation purposes, including the testing of computerized systems [
]. The AiCRO system administrator gives suitable permissions to different personnel and is responsible for overall account management over the study period. This function serves two purposes. From a clinical trial perspective, it blocks access to information that could affect imaging analyses among central reviewers, ensuring impartial review outcomes. From a computerized system perspective, this function allows the administrator to test the system for appropriate plans or test strategies. Parsing out permissions depending on the account enables improved system development and efficient control over clinical trial data.Computerized System Validation
For internal system validation, we organized the overall validation plan according to the V-model (
), which includes the purpose, process validation, test setting and methods, and schedule. All validation processes and results were documented.For installation qualification, the AiCRO’s client system was successfully installed in all test personal computers, regardless of the operating system such as Windows and Mac as well as Web-browsers including Chrome, Internet Explorer versions 10 and 11, and Firefox. The AiCRO’s server system was installed appropriately on our institutional server and Amazon Web Services Clouds in Seoul and China (Beijing).
For operational qualification, the system developer created test scripts to prove the proper functioning of all functional specifications and system operation. Subsequently, an independent tester executed test scripts in a predefined testing environment and recorded the test log and all results including test date, observations made, completeness of each function, problems/deviations encountered, and other information relevant to the operational qualification. These assessments found that all AiCRO functions could be operated without difficulty.
For performance qualification, we prepared a sample project with real data and different user accounts. An independent tester then evaluated module function using the sample project with different amounts of imaging data, work load sizes, and computing and network environments. All functions of the AiCRO worked appropriately under normal operating conditions of minimum required personal computer specifications, and the upload and download of several computed tomography/magnetic resonance imaging scans was performed at usual internet speeds of over 10 megabytes per second. However, during testing under challenging conditions, limiting factors that might cause an interruption or significant delay in activity were identified, which were machines with less than 4 GB RAM, uploading and downloading large image data for at least 3000 DICOM files at one time, internet speeds below 10 MB per second, and viewing large image data for at least 3000 DICOM files at once.
For design qualification, AiCRO validation is essential because each clinical trial project will have a different study design. We checked whether AiCRO was able to create a customizable data entry form and working algorithm per user specifications. Using a test project that simulated a sophisticated clinical trial for stroke, our imaging clinical research associate wrote the user specification, and our system developer created the customizable project in the AiCRO system. Thereafter, a test team, including a project manager, image analyst, and radiologist, performed user acceptance tests. The customized test project was successfully created, and its functional modules garnered acceptance from different kinds of users.
Finally, the AiCRO system was validated by Zigzag (Berkshire, United Kingdom), an external global quality assurance company. A professional quality manager from Singapore visited the core lab in Seoul for 3 days and reviewed all validation documents. This manager also required a demonstration of the process, including creation of a new customized project and associated functions. The quality manager verified that installation qualification, operational qualification, performance qualification, and design qualification were performed appropriately in accordance with the relevant regulations and guidelines.
Discussion
Principal Findings
During the 2000s, the FDA [
] was concerned about the objectivity and validity of imaging endpoints, since there were several critical issues such as reliability of imaging biomarker and results, and appropriateness of imaging data management with regard to GCP, FDA guidelines, and HIPAA. The FDA addressed this issue by developing the Clinical Trial Imaging Endpoint Process Standards Guidance for Industry. The first draft guidance was issued in 2011, with a revision in 2015 and finalization in 2018 [ ]. The guidelines emphasize adherence to regulations for imaging data collection, data transfer, and imaging analysis. At present, all stakeholders including pharmaceutical companies, imaging scientists and clinical trial professionals, and hospitals are expected to follow the 2018 FDA guidance to conduct clinical trials successfully.CTIMS thus needed to comply with FDA guidelines, which comprehensively cover the functional and regulatory aspects of imaging, thereby underscoring the importance of developing a CTIMS that was GxP-compliant. The appropriate use of medical imaging in clinical trials necessitates specific considerations including standardization of imaging acquisition, archiving, and QC; centralized independent blinded image review; and systems that can handle complex workflows while maintaining regulatory compliance.
Asan Image Metrics (Seoul, Korea), an academic imaging core lab, developed a CTIMS—the AiCRO system—which adheres to all regulatory requirements for medical imaging in clinical trials and enables stakeholders to access easily medical images obtained from different sites. It contains modules including deidentification, data transfer, data archive, eCRF, image viewing/analysis, data management according to CDISC, audit trail, query/alarm, complex workflow algorithm, and security. The platform thus ensures quality of results and minimizes data risks during and after clinical trial periods. In the last 2 years, we have implemented the AiCRO system on more than 20 domestic and international pharmaceutical multicenter clinical trials, and several audits and inspections have been conducted as well.
Strengths and Limitations
The main advantage of AiCRO is that it is an all-in-one system that meets virtually all regulatory and functional requirements for clinical trial imaging while maintaining considerable user flexibility. For example, if a user needs a new function for the specific trial, our development team can easily build or customize the necessary function according to user specifications, after which our quality assurance and data management teams offer continuous user and regulatory support for the new function. For example, in the cases of trials on rare diseases requiring sophisticated magnetic resonance imaging analyses, a dedicated imaging postprocessing module and disease-specific terminology can be added in the CDISC data format.
Although our system has several advantages, one of the limitations is that each site or hospital has its own security system containing firewalls to protect their patients’ information. Nowadays, many institutions strictly regulate the external transfer of medical data, and hospital network system firewalls are becoming stronger. Ideally, a CTIMS should strike a balance between security and function within the boundaries of regulations and institutional policies. Another limitation is that AiCRO is an independent system separated from other EDC system. Thus, in clinical trials, we need to link our system to other EDC systems. Otherwise, users have to use two different IT systems in a clinical trial.
Conclusions
We introduced a CTIMS, called AiCRO system, to conduct clinical trials more efficiently in accordance with regulatory requirements. In this paper, we discussed the development of the AiCRO system to meet a diverse array of regulatory requirements and the design of multiple modules for users to customize the system to the needs of their clinical trials. We also discussed the internal and external validation of AiCRO according to the GxP guidelines and FDA 21 CFR. The AiCRO system is an all-in-one platform enabling high-quality clinical trial imaging data, but further study is required to describe the results of implementation obtained for different types of trials and any necessary systemic improvements.
Acknowledgments
This research was supported by a grant from the National Research Foundation of Korea (#2017R1A2B3011475).
Conflicts of Interest
KWK has received research grants from the National Research Foundation of Korea during this study. Other authors declare there is no conflict of interest.
References
- Deserno TM, Deserno V, Haak D, Kabino K. Digital Imaging and Electronic Data Capture in Multi-Center Clinical Trials. Stud Health Technol Inform 2015;216:930. [Medline]
- Ohmann C, Kuchinke W, Canham S, Lauritsen J, Salas N, Schade-Brittinger C, et al. Standard requirements for GCP-compliant data management in multinational clinical trials. Trials 2011 Mar 22;12:85 [FREE Full text] [CrossRef] [Medline]
- O'Connor JPB, Aboagye EO, Adams JE, Aerts HJWL, Barrington SF, Beer AJ, et al. Imaging biomarker roadmap for cancer studies. Nat Rev Clin Oncol 2017 Mar;14(3):169-186 [FREE Full text] [CrossRef] [Medline]
- Park YR, Yoon YJ, Koo H, Yoo S, Choi CM, Beck SH, et al. Utilization of a Clinical Trial Management System for the Whole Clinical Trial Process as an Integrated Database: System Development. J Med Internet Res 2018 Apr 24;20(4):e103 [FREE Full text] [CrossRef] [Medline]
- Mankoff DA, Farwell MD, Clark AS, Pryma DA. How Imaging Can Impact Clinical Trial Design: Molecular Imaging as a Biomarker for Targeted Cancer Therapy. Cancer J 2015;21(3):218-224. [CrossRef] [Medline]
- Yankeelov TE, Mankoff DA, Schwartz LH, Lieberman FS, Buatti JM, Mountz JM, et al. Quantitative Imaging in Cancer Clinical Trials. Clin Cancer Res 2016 Jan 15;22(2):284-290 [FREE Full text] [CrossRef] [Medline]
- Johnson JR, Williams G, Pazdur R. End points and United States Food and Drug Administration approval of oncology drugs. J Clin Oncol 2003 Apr 01;21(7):1404-1411. [CrossRef] [Medline]
- US Food and Drug Administration. 2018 Apr 26. Clinical trial imaging endpoint process standards: Guidance for industry URL: https://www.fda.gov/media/81172/download [accessed 2019-04-01]
- Haak D, Page CE, Reinartz S, Krüger T, Deserno TM. DICOM for Clinical Research: PACS-Integrated Electronic Data Capture in Multi-Center Trials. J Digit Imaging 2015 Oct;28(5):558-566. [CrossRef] [Medline]
- ICH Expert Working. Integrated addendum to ICH E6 (R1): guideline for good clinical practice E6 (R2). 2015 Jun 11. URL: https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E6/E6_R2__Addendum_Step2.pdf [accessed 2019-04-01]
- Wang X, Liu BJ, Martinez C, Zhang X, Winstein CJ. Development of a novel imaging informatics-based system with an intelligent workflow engine (IWEIS) to support imaging-based clinical trials. Comput Biol Med 2016 Feb 01;69:261-269. [CrossRef] [Medline]
- Bendale A, Patel N, Damahe D, Narkhede S, Jadhav A, Vidyasagar G. Computer software validation in pharmaceuticals. Asian Journal of Pharmaceutical Sciences and Clinical Research 2011;1(2):27-39.
- Thakkar P, Balamuralidhara V, Pramod Kumar TM, Valluru R, Venkatesh MP. Use of computerized systems in clinical research: A regulatory perspective. Clinical Research and Regulatory Affairs 2011 Jul 23;28(3):55-62. [CrossRef]
- PIC/S. Good practices for computerised systems in regulated "GXP" environments. 2007 Sep. URL: http://www.gmp-compliance.org/guidemgr/files/PICS/PI 011-3 RECOMMENDATION ON COMPUTERISED SYSTEMS.PDF [accessed 2019-04-01]
- US Food and Drug Administration. Guidance for industry: computerized systems used in clinical trials. 1999. URL: https://www.fda.gov/inspections-compliance-enforcement-and-criminal-investigations/fda-bioresearch-monitoring-information/guidance-industry-computerized-systems-used-clinical-trials [accessed 2019-04-01]
- US Food and Drug Administration. Guidance for Industry Part 11, Electronic Records; Electronic Signatures. 1997. URL: https://www.fda.gov/media/75414/download [accessed 2019-04-01]
- Gamp 5: A Risk-based Approach To Compliant Gxp Computerized Systems. North Bethesda: International Society for Pharmaceutical Engineering; 2008.
- Ohmann C, Canham S, Cornu C, Dreß J, Gueyffier F, Kuchinke W, et al. Revising the ECRIN standard requirements for information technology and data management in clinical trials. Trials 2013 Apr 05;14:97 [FREE Full text] [CrossRef] [Medline]
- Korfiatis P, Kline T, Blezek D, Langer S, Ryan W, Erickson B. MIRMAID: A Content Management System for Medical Image Analysis Research. Radiographics 2015;35(5):1461-1468 [FREE Full text] [CrossRef] [Medline]
- Moore S, Maffitt D, Smith K, Kirby J, Clark K, Freymann J, et al. De-identification of Medical Images with Retention of Scientific Research Value. Radiographics 2015;35(3):727-735 [FREE Full text] [CrossRef] [Medline]
- National Electrical Manufactures Association. DICOM PS3.15 2019c - Security and System Management Profiles. 2019. URL: http://dicom.nema.org/medical/dicom/current/output/pdf/part15.pdf [accessed 2019-04-01]
- Shi B, Bai X, Yao C. An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE transactions on pattern analysis and machine intelligence 2017 Nov 1;39(11):2298-2304 [FREE Full text]
- dcm4chee.org. URL: https://www.dcm4che.org/ [accessed 2019-03-01]
- van Dam J, Omondi Onyango K, Midamba B, Groosman N, Hooper N, Spector J, et al. Open-source mobile digital platform for clinical trial data collection in low-resource settings. BMJ Innov 2017 Feb;3(1):26-31 [FREE Full text] [CrossRef] [Medline]
- Haak D, Page C, Deserno T. A Survey of DICOM Viewer Software to Integrate Clinical Research and Medical Imaging. J Digit Imaging 2016 Apr;29(2):206-215 [FREE Full text] [CrossRef] [Medline]
- CDISC. 2017. CDASH URL: https://www.cdisc.org/standards/foundational/cdash [accessed 2019-03-01]
- CDISC. SDTM URL: https://www.cdisc.org/standards/foundational/sdtm [accessed 2019-08-09]
- CDISC. ADaM URL: https://www.cdisc.org/standards/foundational/adam [accessed 2019-08-09]
- US Food and Drug Administration. 2003 Aug. Guidance for industry: part 11, electronic records; electronic signatures-scope and application URL: https://www.fda.gov/downloads/regulatoryinformation/guidances/ucm125125.pdf [accessed 2019-03-02]
Abbreviations
API: application programming interface |
CAPA: corrective and preventive action |
CDISC: Clinical Data Interchange Standards Consortium |
CRO: clinical research organization |
CTIMS: Clinical Trial Imaging Management System |
DICOM: Digital Imaging and Communications in Medicine |
eCRF: electronic case report form |
EDC: electronic data capture |
FDA: Food and Drug Administration |
GAMP: Good Automated Manufacturing Practice |
GCP: Good Clinical Practice |
HIPAA: Health Insurance Portability and Accountability Act |
ICH: International Conference on Harmonization |
IT: information technology |
NAS: Network-Attached Storage |
PACS: picture archiving and communication system |
QC: quality control |
RECIST: Response Evaluation Criteria in Solid Tumors |
ROI: region of interest |
Edited by G Eysenbach; submitted 08.04.19; peer-reviewed by J Lee, H Lin; comments to author 01.05.19; revised version received 05.07.19; accepted 22.07.19; published 30.08.19
Copyright©Youngbin Shin, Kyung Won Kim, Amy Junghyun Lee, Yu Sub Sung, Suah Ahn, Ja Hwan Koo, Chang Gyu Choi, Yousun Ko, Ho Sung Kim, Seong Ho Park. Originally published in JMIR Medical Informatics (http://medinform.jmir.org), 30.08.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on http://medinform.jmir.org/, as well as this copyright and license information must be included.