Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Psychotic disorders are serious mental health illnesses that lead to difficulties in distinguishing between the internal experience of the mind and the external reality of the environment¹, as well as a higher than average risk of self-harm and suicide². Under standard care, these disorders result in major public health impact with a significant health and economic burden on individuals, families and societies worldwide³. Early interventions in psychosis can improve outcomes of this mental disorder⁴. In particular, detection, prognostic assessment and preventive treatment of individuals who are at clinical high risk for developing psychosis (CHR-P)⁵ provides a unique potential to alter the course of the disorder, thereby improving the quality of life for many people and their families^3,6. CHR-P individuals are help-seeking young people presenting with attenuated symptoms and functional impairment⁷: their risk of developing psychosis is 20% at 2-years⁸ but it is higher in some specific subgroups^9,10. Despite some substantial advancements, the impact of preventive approaches in routine clinical practice is limited by the ability to detect most individuals who are at-risk¹¹. Current detection methods are based on help-seeking behaviors and referrals on suspicion of psychosis risk; these methods are highly inefficient in handling a large number of samples¹¹. Thus, the scalability of current detection methods to the vast majority of the at-risk population is quite limited¹². In fact, only 5% (standalone specialized early detection services) to 12% (youth mental health services) of individuals at-risk of developing a first psychotic disorder can be detected at the time of their at-risk stage by the current detection strategies⁶.

To extend the clinical benefits of the preventive approaches in a larger number of at-risk individuals, we developed an automated, lifespan-inclusive (i.e., across all ages), transdiagnostic (i.e., across different diagnoses)¹³, clinically-based individualized risk calculator, which can detect individuals at-risk of psychosis in secondary mental health care at scale, beyond those meeting CHR-P criteria¹⁴. This risk calculator used a Cox proportional hazard model to predict the risk of developing a psychotic disorder over six years from five routinely collected clinical variables selected a priori, in line with methodological guidelines¹⁵: age, gender, ethnicity, age-by-gender and primary index diagnosis. These clinical variables were selected based on a priori knowledge obtained from meta analyses^16,17, as recommended by the state-of-the-art methodological guidelines¹⁵. The number of predictors is limited to preserve the Event Per Variable ratio and minimize overfitting biases; including too many variables without a priori filter leads to overfitting problems and poor prognostic accuracy¹⁸. The method used to develop this model provides similar prognostic accuracy to automatic machine learning methods¹⁸. Parameters of the Cox model were estimated based on a retrospective de-identified cohort from the South London and Maudsley National Health Service Foundation Trust (SLaM)¹⁹. SLaM is a National Health Service (NHS) mental health trust that provides secondary mental health care to a population of 1.36 million individuals in South London (Lambeth, Southwark, Lewisham and Croydon boroughs), and has one of the highest recorded rates of psychosis in the world²⁰. All data used in the model development were extracted from the Clinical Record Interactive Search (CRIS) platform, a digital case register system, which provides researchers with retrospective access and analysis of anonymized clinical records¹⁹. The clinical information in CRIS is extracted from a bespoke Electronic Health Record (EHR) system, at SLaM, called electronic Patient Journey System (ePJS). SLaM is paper-free and ePJS represents the standard data collection platform for clinical routine. Thus, the transdiagnostic risk calculator leverages EHRs and has the potential to automatically screen large EHRs of patients accessing secondary mental healthcare, to detect those who may be at-risk of psychosis. The algorithm of this transdiagnostic risk calculator has been published previously^6,14,21. The transdiagnostic risk calculator has been externally validated in two NHS Foundation Trusts^14,21 and optimized²², demonstrating its adequate prognostic performance and generalizability across different populations.

According to methodological guidelines on the development of a risk prediction model^15,23, the next step after model development and validation is to implement the prediction model in routine clinical practice. Implementation studies are usually preceded by pilot or feasibility studies that address potential pragmatic limitations associated with the use of risk algorithms in clinical practice. For example, required data for running a calculator, such as age, gender and ethnicity, may not be available at the date of diagnosis or updated later. Effective methods for handling missing data and synchronizing frequent updates in real-time data streams should be considered to obtain the most reliable prediction results in an implementation. Furthermore, since the initial development of the risk calculator was based on retrospective cohort data, it is not known whether it can be used in a real-time data stream that is typical of a real-world clinical setting. Another challenge is ensuring that relevant clinicians receive the recommendations generated by the risk calculator within an appropriate time frame and within a shared and accepted communication pathway.

To overcome these limitations, we have completed a feasibility implementation study employing the individualized transdiagnostic risk calculator. The study included two phases: an in vitro phase that was conducted using data from the local EHR, without contacting clinicians or patients, and an in vivo phase, which involved direct contact with clinicians. The in vitro phase had two manifold aims: (i) to address implementation barriers according to the Consolidated Framework for Implementation Research (CFIR)²⁷ and (ii) to integrate the transdiagnostic risk calculator into the local EHR. Implementation barriers included the communication of risk outcomes to clinicians. In SLaM, all patients are invited to register for Consent for Contact (C4C), which indicates their willingness to be contacted for research, without affecting the quality of care. This reduces the ethical issues relating to contacting patients. Further to this, working groups with clinicians aided tailoring of how this information was communicated. During the in vivo phase (May 14th 2018 to April 29th 2019), all individuals (i) older than 14 years (ii) who were accessing any SLaM service (boroughs of Lambeth, Southwark, Lewisham, Croydon), (iii) receiving a first ICD-10 index primary diagnosis of any non-organic, non-psychotic mental disorder (with the exception of Acute and Transient Psychotic Disorders; ATPD), or a CHR-P designation and (iv) with existing contact details were deemed eligible. During the in vivo phase, new patients accessing SLaM each week were automatically screened for their psychosis risk, and those with having a risk greater than a certain threshold were detected. The research team then contacted the patients' responsible clinicians to discuss further recommendations and eventually suggest a further face to face assessment⁶. If those assessed were considered to meet CHR-P criteria, they were referred to specialist CHR-P services, such as Outreach and Support in South London (OASIS)²⁸. This would result in improved detection of individuals prior to the onset of a psychotic disorder and provide a significant opportunity for altering the course of the disorder. Crucially, this feasibility study involved the full integration of the calculator into the local EHR system, which is the topic of the current article. The full protocol of this feasibility study, including an overview of the plan for evaluating the proposed research, details on managing data security and ethical issues, has been presented in our previous work⁶. The current article, as a part of the feasibility study⁶, selectively focuses on presenting the technical implementation of a real-time psychosis risk detection and alerting system based on the local EHR data. More specifically, the aim of this study is to investigate the technical feasibility of this risk calculator in timely detecting at-risk patients as soon as they access a secondary mental healthcare service. The full results of the feasibility study, in terms of clinicians' adherence to the recommendations made by the risk calculator, will be presented separately. A comprehensive evaluation of the effectiveness of the proposed research, which requires randomized designs, is outside the scope of the current research program. To our best knowledge, this is the first method describing the implementation of a risk calculator based on live EHR data for early detection of psychosis.

Our approach to psychosis risk detection and alerting takes advantage of the CogStack platform. The CogStack platform is a lightweight, distributed, and fault-tolerant information retrieval and text-extraction platform²⁴. This platform consists of three key components: 1) the CogStack Pipeline that uses the Java Spring Batch framework to ingest and synchronize data from a pre-defined data source (both structured and unstructured EHR data in multiple formats such as Word, PDF files and images) to a predefined data sink in real time; 2) Elasticsearch, a search engine allowing for storage and querying of the full text of EHR data, as well as providing various application programming interfaces (APIs) to embed advanced analytics into the engine; and 3) Kibana, an interactive, web-based user interface that allows users to query data in Elasticsearch, build visualization dashboards and set alerting on anomalies or other patterns of interest from data. Moreover, CogStack incorporates the ability to alert clinicians to potential problems by Email and SMS (text), allowing clinicians to receive timely notifications about at-risk patients reported by the risk calculator.

We present a model of psychosis risk detection and alerting based on ePJS at SLaM, leveraging the CogStack platform. Compared with the CRIS platform that provides a mechanism for retrospective access to de-identified health records from ePJS on a weekly basis¹⁹, the CogStack platform at SLaM enables access to an identifiable EHR in real time, bringing the alerting closer to the point-of-care and the risk prediction in a prospective design, although both the CRIS and CogStack platforms use data sourced from ePJS in SLaM. In the section that follows, we provide details of the key steps in our approach, including preparing source data from the EHR, ingesting the source data into the CogStack platform to enable full-text search via Elasticsearch, running the psychosis risk calculator using a Python daemon thread, and setting interactive visualizations and real-time risk alerting via the Kibana user interface. Any researcher who aims to build a real-time risk detection and alerting system based on EHR data can follow the approach and its reference implementation. As we shall elaborate below, the proposed method exploits open-source, lightweight techniques with high flexibility and portability. This enables the risk calculator to be run in various locations and shows a high applicability to other risk estimation algorithms. Moreover, the method works as a straightforward approach to enhance the risk detection and alerting functionalities of an EHR embedded in a general healthcare system.