This consortium distributes a Nature published, custom BERT-style large language model called NYUTron that is built on 7.25 million clinical notes and 9.5 years of data. Our goal is to enable a community around NYUTron and develop use cases together to demonstrate real patient value.
About the Model
NYUTron, which was developed by NYU Langone researchers, reads a complete history and physical or discharge summary note to achieve state-of-the-art performance for estimating an inpatient’s risk of death, calculating length of hospital stay, 30-day readmission, comorbidity index, and insurance denials. The Nature publication has been accessed 86,000 times since its publication on June 7, 2023.
NYUTron is more performant compared to GPT. Internal studies found that GPT-4 alone cannot reach similar performances than NYUTron on the 5 tasks listed above. It is suspected that the domain-specific language of clinical notes helps the model identify textual drivers of risk.
NYUTron is also cost-effective compared to open source or GPT models. NYUTron is a small language model at 100-300 million parameters, compared to LLAMA2, which has 7 billion parameters, or GPT-4, which is suspected to be 16 expert models at 111 billion parameters each. For 24/7 hosting on the cloud, GPT-4 (if available) would likely run $500,000 per year, while NYUTron is estimated to cost less than $1,000 per year.
A core finding of the work is that we can accelerate the model lifecycle from weeks to days using a foundation large language model like NYUTron. Specifically, feature extraction is no longer necessary and foundation models can learn the relevant features for a task directly. In other words, we can train a model with medical notes such as a discharge summary and directly learn this patient’s readmission risk (as opposed to identifying individual features and then calculating risk).
About the Consortium
NYU Langone has organized the Consortium for Operational Medical AI (COMAI) to be comprised of non-profit hospitals and academic medical centers throughout the United States and internationally that can join together in a cooperative effort to use, study, and develop NYUTron; to develop a better understanding of clinical large language models and operational medical AI; and to facilitate improvement of the implementation and use of NYUTron and other clinical large language models for the purpose of quality improvement, safety, and health care efficiency for the ultimate benefit of patients and health care providers.
Reasons to Join
- Starter set of 5 models:
- predicting inpatient mortality at admission time
- predicting insurance denial at discharge
- predicting 30-day readmission at discharge
- predicting length of stay at admission time
- classifying Charlson Comorbidity at admission time
- Access to an inventory of use cases from contributing consortium members that run classification and prediction tasks on clinical text and notes
- Software infrastructure to build text-based models in hours vs months
- A community for best practices and co-learning
- Organizational best practices for building up expertise and personnel to build, develop, and deploy fine-tuned large language models
- Accelerate the LLM adoption of AI models in healthcare in the provider space with demonstrated value
Objectives
COMAI has been created to establish partnerships with non-profit or academic medical institutions to promote a mission of innovation in the clinical large language models and operational medical AI. As a member of COMAI, the member is expected to play an important role in the research, education, and innovation goals of COMAI including expanding and demonstrating the scientific, technological, and operational feasibility of innovative methodologies and applications governing clinical large language models.
Eligibility and Member Commitment
Any non-profit hospital or academic medical center is eligible. Membership in COMAI is $100,000 per year for a three-year term. Renewals and/or extensions are subject to the terms of the membership agreement.
Contact us in the next 30 days to access an upcoming exclusive information session!
For-profit organizations or government entities interested in learning more about COMAI can contact COMAI@nyulangone.org.
