Choosing the right model for your patient: An introduction to InsightRX Gemini

The success of model-informed precision dosing rests, in part, on a suitably predictive model. That’s why the InsightRX data science team carefully reviews, validates and improves population pharmacokinetic (PK) models for use in our Bayesian forecasting software. (Hughes 2021, 2023, 2024) However, just as a one-size-fits-all approach fails to find the right dose for each patient for drugs with narrow therapeutic indices, a one-model-fits-all approach doesn’t describe how every patient will respond to a particular drug.

For example, for vancomycin — a drug with high inter-individual variability and a narrow therapeutic range, for which AUC-guided Bayesian methods of dose adjustment have been recommended — there is growing evidence that obese patients or elderly patients with low serum creatinine are not always well described by published models developed for the general patient population (see Tong et al. 2020 and Hughes et al. 2024).

Several approaches have been proposed to handle this problem of describing tricky subpopulations. Colin et al. (2019) created a vancomycin meta-model using data pooled from fourteen different studies comprising 2554 individual patients. Covariate relationships were informed by knowledge of physiology and covariate effects selected to describe changes in PK arising from age, renal function, body weight, and co-morbidities. While this model performs consistently well across a wide range of vancomycin patients, we have found that for many subpopulations of patients it is outperformed by more specialized models (2021, 2023, 2024).

Another angle is to select between or combine multiple models using information about a specific patient to identify the right model for them. For example, Uster et al. (2021) created a modeling approach that weighted multiple model predictions or selected a model based on MAP Bayesian fit metrics. Their algorithms out-performed single-model approaches, however they require at least one drug level and therefore cannot be used for initial dosing. Selecting the right regimen from the start of therapy helps get patients to target faster, limiting its suitability for Bayesian MIPD end users operating at the point of care.

For several years, the recommendation at InsightRX has been to use a model auto-select feature to guide vancomycin model selection for adult patients. This auto-select tool automatically switches from a general model to an obese-specific model for patients with class III obesity. This feature has been used to improve patient care at nearly 800 hospitals to improve their vancomycin exposure target attainment, and is highlighted in our most recent client case study. However, a binary auto-select algorithm does not reflect the full diversity of patients, and other subpopulations, including the young and underweight, may benefit from more specialized models too.

We set out to create an algorithm to help clinicians identify the right model for their patient, an algorithm we’ve named InsightRX Gemini.

Designing InsightRX Gemini

With years of experience implementing popPK models for clinical decision support — and helping our network of clinicians treat over 1 million patients — we’ve discovered a lot about how to evaluate, validate and select models for use in precision dosing. We wanted to “bake in” all this knowledge into our algorithm, to complement data-driven insights into model performance. We distilled this experience into seven guiding principles:

1. Evaluate models as they’re used in practice: We assess models based on how they perform in real clinical settings, making predictions step by step without knowing future outcomes, just like in actual patient care. This ensures our models are practical and reliable in day-to-day use.
2. Prioritize population predictions: Models that perform well using population predictions (a priori) generally perform well once levels are available for Bayesian estimation of individual PK (a posteriori). Models tend to perform more similarly a posteriori versus a priori. We therefore built our algorithm based on a priori prediction performance.
3. Focus on key statistical metrics: We selected three broadly used metrics (El Hassani and Marsot, 2023, Hughes et al. 2023) for summarizing different aspects of model error:
   1. Error Size (RMSE): How far off predictions are, especially avoiding large mistakes.
   2. Bias (MPE): Whether predictions consistently miss in one direction, like always underestimating or overestimating.
   3. Accuracy: How often predictions fall within a safe, acceptable range.
4. Efficacy first: When models have a consistent bias, we prefer those that overestimate clearance. This helps prevent underdosing, a much more common problem than overdosing, particularly early in therapy before a level is available.
5. Tailored models for a better fit: Models developed from populations like the patients being treated provide more accurate dosing, as they better reflect the relationships between patient characteristics and drug behavior. Given two models that are otherwise statistically tied, we pick the one that best matches the patient’s characteristics.
6. Rely on scientific knowledge: Recognizing that vancomycin works in a complex, two-compartment system, we prioritize models that account for this complexity, ensuring more accurate dosing decisions.
7. Real-world experience matters: We rely on models that have been proven in real-world clinical settings, avoiding those that might seem good on paper but don’t hold up in practice due to issues like instability or unexpected variability.

We selected seven pharmacokinetic models that we know and trust (see principles 6 and 7 above), and evaluated each of these models in 384,000 treatment courses treated with vancomycin. We divided patients into 129 categories, based on their renal function, their BMI, their age and their sex. For each of these patient archetypes, we identified which models performed the best according to the principles described above. Additional information on the feature launch can also be found in our recent press release.

Algorithm Evaluation

We tested the performance of our algorithm on a hold-out data set of 145,000 treatment courses, comparing it to single-model approaches typically used in other clinical decision support software.

We found that, compared to widely-used, single model approaches, InsightRX Gemini’s auto-selected model consistently:

• Improved population prediction accuracy by as much as 50%
• Reduced bias by 85-88%, and
• Lowered error magnitude by 10-22%

We saw similar improvements for posterior dosing (i.e., forecasts made with MAP Bayesian PK parameter estimation after a sample has been collected), even though our algorithm was selected to minimize population predictions. This gives us further confidence that this algorithm is selecting appropriate models for each patient.

How does InsightRX Gemini compare to other model selection approaches in the literature?

InsightRX Gemini stands out from other model selection approaches in several key ways. First, it's built on a large and diverse data set (almost 800 hospitals!), which makes it highly reliable and broadly applicable across different patient populations. While some smaller subpopulations within our data had fewer patients, they were still larger than those in many model validation studies, allowing for robust analysis of model performance in these subpopulations.

Another advantage of InsightRX Gemini is its transparency and grounding in real-world clinical experience. Unlike some machine learning models that can be complex and difficult to interpret, InsightRX Gemini can be visualized as a decision tree, making it easy for clinicians to understand and apply. Although machine learning models can handle more varied data types, they often lack the intuitive clarity that InsightRX Gemini offers, while our approach still achieves a significant improvement in dosing accuracy.

Additionally, InsightRX Gemini is designed to meet clinical needs —it can be used right from the start of treatment, even before patient-specific data is collected, making it ideal for initial dosing decisions. Other methods, like model averaging, typically require drug levels or biomarkers to guide model selection (Uster et al. 2021). In contrast, InsightRX Gemini offers a simpler, more adaptable solution that works effectively from the outset. Future studies could evaluate how these more complex methods of model selection measure up against the strong performance of InsightRX Gemini.

Incorporating even better models or customizing them more precisely for specific patient groups, as we did with the Hughes (2024) model, could enhance its performance further. While it’s still an open question whether to focus resources on creating more specialized models or on developing more complex selection algorithms, our current approach strikes a strong balance. By using a straightforward model selection algorithm combined with the best-validated pharmacokinetic models, we’ve already achieved a significant improvement in model accuracy.

Want to learn more about InsightRX Gemini? Download a copy of our feature sheet below!