Introduction to the REAI project
The project's primary focus is on developing the AI model through deep machine learning algorithms using the RHInnO Ethics platform. A high-level workflow will be developed in consultation with key stakeholders to inform how data from the current RHInnO Ethics platform are captured and used to inform the decision-making algorithm of the new AI-powered RHInnO Ethics platform. This will inform the choice of large language and text models used to capture and process the various historical data available in the current RHInnO Ethics platform.
Historical data from the existing RHInnO Ethics platform will be used to conduct the initial training and validation of the AI model on various decision-making pathways. A system known as Optical Character Recognition (OCR) will be used to read the relevant information from the historical data. Additionally, systems known as BERT and Llama 3.2 will be used to cross-validate the applicants' responses against the reviewers' comments in the historical data to ensure consistency and accuracy. Specifically, BERT will be used for summarization of texts, while Llama 3.2 will be used for validation of responses. The use of Llama 3.2 will allow the generation of validated predictions on the best answers reviewers should give against applicants' responses in the application. Based on the validation results, further training may be necessary to fine-tune the system to improve model performance. It is important to note that pre-trained versions of BART and Llama2 models will be downloaded from Hugging Face and hosted in a private cloud environment. The models may further be fine-tuned using PyTorch on domain-specific data. Fine-tuning will focus on document summarisation, validation of questionnaire responses and question-answer generation.
Stakeholders’ engagement
To enhance the understandability and acceptability of the AI model, two consultative workshops will be conducted, where key stakeholders will be requested to give their opinions about the AI-powered functionalities. This will allow the model development process to be inclusive by involving diverse stakeholders from the African continent. Additionally, this will enable stakeholders to validate the model’s outputs and raise any concerns they may have. Stakeholders who will be involved in the consultative workshops will be requested to become champions for promoting the wide-scale use of the AI-powered platform. In addition, the involvement of representatives of various African RECs will provide a platform for sensitising RECs about the new platform and its capabilities in order to get as many RECs as possible to transition from the systems they use to the new AI-powered ethics review system. Efforts will be made to reach out to national regulatory agencies in different African countries and regional authorities such as the African Union and Africa CDC to sensitise them about the platform and seek their buy-in to adopt the AI-powered ethics review platform as the bonafide pan-African ethics review and oversight system.
Assess the effectiveness of the AI model in clinical trial decision-making and ethics review
To assess the effectiveness of the AI-powered RHInnO Ethics platform, 50% of the RECs that are currently using the cloud-based online RHInnO Ethics platform will be purposively selected and migrated to the new AI-powered RHInnO Ethics platform, while the other 50% will be left to continue using the current (non-AI powered) RHInnO Ethics platform. Using a consultative workshop and semi-structured interviews, representatives of RECs from each of the groups (those using AI-powered and those using Non-AI powered RHInnO Ethics platform), including administrators, chairpersons, and reviewers will be invited and consented to participate in an individual in-depth interview, to provide their feedback and experiences of using the two different platforms. This will aim to provide a comparative assessment of the efficiency of the two platforms (AI-powered and non-AI-powered platforms) to understand the perceived effects and value of the two platforms, specifically focusing on the quality, turn-around time and overall decision-making process in clinical research ethics review. The users' satisfaction, including user-friendliness, interactivity, interpretability and areas needing adjustment on the system, will also be identified.
In addition to the operation assessment, this objective will also aim to conduct a technical assessment of the platform to ensure optimum operation. This will involve assessing load times and implementing lazy loading, caching, and minification of static assets to ensure the system operates efficiently. In addition, data will continuously be updated to promote accuracy in the platform responses. Ongoing load balancing, autoscaling, and vertical scaling will also be implemented to promote the platform's resilience to high demand and traffic and its continuous use as it gains popularity.
Launching and scaling up the use of the AI-enabled ethics review platform.
This phase will involve disseminating the project findings to key stakeholders to sensitise them about the new AI-powered RHInnO Ethics platform and get new RECs to start using it. The process will involve using social media, including the launch of the project website and social media handles where information about the project will be published. Additionally, stakeholders will be invited to reflect on the future of RECs in the era of AI and the opportunities presented by the AI-powered RHInnO Ethics in Africa. A workshop report and peer-reviewed manuscript will also be key outputs of this project, where the wider public will be informed about the project.
Scaling UP
Adopting an AI-powered ethics review platform will require a stepwise approach involving key stakeholders' engagement in the project's conceptualisation and implementation. This will be covered through the co-creation workshops, where key stakeholders will be engaged to give their input about the AI-powered functionalities as part of the data collection and model training. The model development process will be inclusive and include diverse stakeholders from the African continent. Additionally, assessing the AI-powered functionalities will ensure the model can provide the necessary and expected outcomes. Those partners involved in the co-creation and testing of the models will eventually serve as champions who will sensitise others about the model. In addition, the stakeholder workshop, which will involve representatives of various RECs Africa and peer-reviewed publications, will provide a platform for sensitising RECs about the AI-powered platform and its capabilities to get as many RECs as possible to transition from the system they use to the new AI-powered ethics review system. Efforts will be made to reach out to national regulatory agencies in different African countries and regional authorities such as the African Union and Africa CDC to sensitise them about the platform and seek their buy-in to adopt the AI-powered ethics review platform as the bonafide pan-African ethics review and oversight system.
Anticipated Benefits
Research ethics is integral to strengthening the health system and global health. The proposed project aims to strengthen Africa's health system by integrating AI into an existing cloud-based online review system. This has the potential to enhance ethics review and the clinical trials decision-making and oversight processes in African RECs. Ultimately, an efficient ethics review system has the potential to save lives through robust review, efficient oversight and timely discovery of healthcare interventions.
Currently, the existing RHInnO Ethics platform is already used by over 32 RECs. It enables RECs to review proposals by reading each proposal, reviewing answers provided by the principal investigators and responding to specific questions posed to the reviewer before recommending whether to approve the proposal. Because there are no objective indicators to inform the REC members' recommendations, their decisions could be subjective and biased. Furthermore, REC members may suffer from unconscious bias, which could influence their approval decision. The proposed model will allow RECs to identify key ethics decision-making pathways, hence enabling the harmonisation of ethics review. Furthermore, the AI-powered RHInnO Ethics platform will have generic as well as customisable questions. By allowing RECs to adapt the platform to their individual needs and requirements while maintaining some generic questions, the platform will respond to the dynamic nature of the ethics review ecosystem while promoting harmonisation through the generic questions. Using a Cloud-based system will also enable central management of the system, hence making maintenance and technical support more efficient and cost-effective. This will allow RECs to have access to the platform without having to employ IT experts to manage the platform.
The project is run by three PIs, one male and two females. The principal investigator is a black Kenyan male who works as the CEO of EthiXPERT, while the two other co-PIs are white South Africans who work at EthiXPERT as COO and the board chairperson. This demonstrates cultural and gender diversity at the project management level. In addition, the targets research ethics committees. The existing RHInnO Ethics platform is currently used by over 32 RECs spread across eleven African countries in East, West and Southern Africa. These RECs are coordinated by a diverse staff of different genders and cultural backgrounds. RECs are predominantly multidisciplinary and multi-gendered. This project is, therefore, inclusive and accessible to all people regardless of their level of ability.
All RECs presently utilising the platform will be eligible to participate in the study. However, only historical data from RECs that consent to authorise read-only access to their historical data will be utilised. Before gaining read-only access to their historical data, all institutions hosting RECs that use the RHInnO Ethics platform will be given information about the project and an opportunity to provide written gatekeeper's permission. The REC chairpersons would serve as the initial gatekeepers. Only data from RECs with the relevant institutional permissions will be used to train the AI model. The researchers will reach out to the listed RECs below to obtain gatekeeper permission from at least ten of the RECs.
In seeking consent to use historical data to train the AI-powered RHInnO Ethics model, EthiXPERT shall assume full responsibility for ensuring that all activities related to the data shared from the RHInnO Ethics platform comply with POPIA/GDPR, including ensuring lawful data processing, anonymisation/pseudonymisation of personal data before processing it for AI model training and ensuring that the AI model development activities are conducted lawfully, and under the GDPR. To ensure data privacy and security, extracted text and responses will be tokenized for processing, ensuring no raw, sensitive data is stored unnecessarily. Only data essential for task execution will be used to minimise data usage. Finally, Input data (e.g., PDFs and questionnaires) and intermediate results will be discarded immediately after processing unless explicitly required for auditing purposes to ensure strict adherence to our Data Discarding Policy.
For more information on the funder, the Science for Africa Foundation, visit their website:
https://scienceforafrica.foundation/
For more information on the project, please contact the CEO, Mr Francis Kombe, at kombe@ethixpert.org.za