Development and validation of an artificial intelligence model for the early classification of the aetiology of meningitis and encephalitis: a retrospective observational study

Abstract

Background: Early diagnosis and appropriate treatment are essential in meningitis and encephalitis management. We aimed to implement and verify an artificial intelligence (AI) model for early aetiological determination of patients with encephalitis and meningitis, and identify important variables in the classification process.

Methods: In this retrospective observational study, patients older than 18 years old with meningitis or encephalitis at two centres in South Korea were enrolled for development (n = 283) and external validation (n = 220) of AI models, respectively. Their clinical variables within 24 h after admission were used for the multi-classification of four aetiologies including autoimmunity, bacteria, virus, and tuberculosis. The aetiology was determined based on the laboratory test results of cerebrospinal fluid conducted during hospitalization. Model performance was assessed using classification metrics, including the area under the receiver operating characteristic curve (AUROC), recall, precision, accuracy, and F1 score. Comparisons were performed between the AI model and three clinicians with varying neurology experience. Several techniques (eg, Shapley values, F score, permutation feature importance, and local interpretable model-agnostic explanations weights) were used for the explainability of the AI model.

Findings: Between January 1, 2006, and June 30, 2021, 283 patients were enrolled in the training/test dataset. An ensemble model with extreme gradient boosting and TabNet showed the best performance among the eight AI models with various settings in the external validation dataset (n = 220); accuracy, 0.8909; precision, 0.8987; recall, 0.8909; F1 score, 0.8948; AUROC, 0.9163. The AI model outperformed all clinicians who achieved a maximum F1 score of 0.7582, by demonstrating a performance of F1 score greater than 0.9264.

Interpretation: This is the first multiclass classification study for the early determination of the aetiology of meningitis and encephalitis based on the initial 24-h data using an AI model, which showed high performance metrics. Future studies can improve upon this model by securing and inputting time-series variables and setting various features about patients, and including a survival analysis for prognosis prediction.

Funding: MD-PhD/Medical Scientist Training Program through the Korea Health Industry Development Institute, funded by the Ministry of Health & Welfare, Republic of Korea.