🚨 Pulse of Innovation 🚨

Breaking news in the healthcare AI

Open AI: Brings in a new model called “Strawberry”

OpenAI has unveiled its latest AI model, o1, previously code-named "Strawberry." This model is designed to enhance reasoning capabilities in artificial intelligence. As reported by multiple sources, this new model series aims to tackle complex problems in science, coding, and mathematics by spending more time "thinking" before responding, mimicking human-like reasoning processes.

Enhanced Reasoning:

The o1 model demonstrates remarkable capabilities in complex problem-solving, particularly in STEM fields. In evaluations, it ranked in the 89th percentile on competitive programming questions (Codeforces) and placed among the top 500 students in the USA Math Olympiad qualifier (AIME). Its performance extends to scientific domains, exceeding human PhD-level accuracy on a benchmark of physics, biology, and chemistry problems (GPQA). This advanced reasoning ability allows o1 to tackle multifaceted issues, generate sophisticated algorithms, and excel at comparative analysis tasks like examining contracts or legal documents.

The o1 model's performance is particularly noteworthy in STEM fields, demonstrating its ability to solve complex problems and reason through challenging tasks.

Limitations:

• It is significantly more expensive, with input costs 3 times higher and output costs 4 times higher than GPT-4o in the API

• The model can be slower in processing queries, sometimes taking over ten seconds to answer complex questions. 

• o1 currently lacks features like web browsing and file analysis, which are available in other AI models. 

🧑🏼‍🔬 Bench to Bedside👨🏽‍🔬

Developments in healthcare AI research and innovations

Chai Discovery: Open AI’s platform for Faster and Accurate Drug Discovery

• Chai-1, a new multi-modal foundation model for molecular structure prediction that performs at the state-of-the-art across a variety of tasks relevant to drug discovery. Chai-1 enables unified prediction of proteins, small molecules, DNA, RNA, covalent modifications, and more.

• We covered AlphaFold 3 in this section last week, and Chai Discovery’s model performed on par with the AF3 (77% v. 76%).

• The most significant advantage of the Chai model over others (such as AF3) is that it does not require multiple sequence alignments (MSAs). The model can fold multimers more accurately (69.8%) than the MSA-based AlphaFold-Multimer model (67.7%), as measured by the DockQ acceptable prediction rate. Chai-1 is the first model to predict multimer structures using single-sequences alone (without MSA search) at AF3-Multimer level quality.

• Chai-1 can be used to predict antibody-antigen structures with high accuracy. It can also be prompted with new data, such as restraints derived from the lab, which can boost its performance. One of Chai-1's capabilities is epitope conditioning to double the accuracy of antibody-antigen structure prediction. This makes antibody engineering more feasible using AI.

• Chai-1 is free for individual and commercial use, so it can help scientists in academic labs and pharmaceutical companies improve protein identification and drug discovery.

🧑🏽‍⚕️ AI in Clinic 🏥

Developments in healthcare AI research and innovations

AirPods Pro 2 - FDA Authorized

On September 13, 2024, the FDA approved the Hearing Aid Feature (HAF) software for AirPods Pro 2, allowing them to function as over-the-counter hearing aids when paired with compatible iOS 18 devices. This approval is a first for the FDA and aims to help an estimated 30 million Americans with mild to moderate hearing loss.

Hearing Aid Feature Software

The Hearing Aid Feature (HAF) software will turn the AirPods Pro 2 into clinical-grade hearing aids. It will provide personalized dynamic adjustments to enhance ambient sounds in real-time. This innovative technology will apply the user's customized hearing profile across audio experiences, such as music, movies, and phone calls, without requiring manual adjustments. The software update is expected to be rolled out in fall 2024 and will be available in over 100 countries and regions, making it accessible to a wide range of users with mild to moderate hearing loss.

Accessibility

Air Pods Pro 2 offers a comprehensive hearing health experience, including active Hearing Protection, a scientifically validated Hearing Test, and a clinical-grade Hearing Aid feature. Users can take a convenient five-minute hearing test at home and receive an easy-to-understand summary of results, including an audiogram that can be shared with healthcare providers. This approach addresses the issue of untreated hearing loss, as research from the Apple Hearing Study revealed that 75 percent of people diagnosed with hearing loss haven't received necessary assistive support.

Reducing Stigma : Otolaryngologists hope Apple's entry into the hearing aid market will encourage more individuals with hearing loss to seek help, as AirPods Pro 2 offers a discreet and socially acceptable solution at a lower price point of $249, compared to thousands of dollars for other over-the-counter hearing aids.

🤖 Patient First, AI Second🤖

Ethical and Regulatory Landscape of Healthcare AI

Which is the best AI tool? Well, it depends.

• Multiple Health AI platforms have become available across different specialties, with the most substantial progress in radiology, which uses automated image analysis and related applications.

• With many available options, clinicians face the age-old dilemma of choosing the best available option. Instead of drugs, they now have to choose the AI tool.

• The biggest answer clinicians seek is the trustworthiness of the AI tool they are using so that it is accurate, reliable, and reproducible.

• A recent NEJM study attempted to evaluate the performance of several different radiology AI tools in detecting Diabetic Retinopathy and, by doing so, provided a framework for testing the trustworthiness of health AI tools.

• Key findings from the study:

  • Data Collection: A vast dataset of retinal images from the North East London NHS Diabetic Eye Screening Program (DESP) was used. This dataset included images from diverse patients, capturing different ethnicities, ages, and socioeconomic backgrounds.

  • ARIAS Evaluation: Eight AI tools were evaluated on their ability to detect DR accurately. The Automated retinal image analysis systems (ARIAS) were tested on their sensitivity, specificity, positive and negative predictive values, and likelihood ratios.

  • Population Subgroups: ARIAS's performance was assessed across different age groups, sexes, ethnicities, and socioeconomic statuses. This allowed for an evaluation of the systems' equity in performance across diverse populations.

  Challenges and Future Directions:

  • Technical Issues: Some ARIAS encountered technical difficulties when running offline or processing low-resolution images. This highlights the need for more standardized requirements and compatibility testing.

  • Standardization: A standardized API for ARIAS could streamline the evaluation process and facilitate wider adoption of these systems.

  • Cloud-Based Solutions: A cloud-based Trusted Research Environment (TRE) could reduce evaluation time and complexity and provide a more accessible platform for vendors to develop and test their ARIAS.

  • Cost-Effectiveness: Future studies should compare the cost-effectiveness of different ARIAS. This information is crucial for decision-makers when considering implementing these systems in healthcare settings.

This comparison of ARIAS at scale on a range of images with different characteristics, including a population of different ethnicities, wide age range, levels of deprivation, and spectrum of DR, provides the framework for transparent, equitable, robust, and trustworthy evaluation of clinical AI in screening to inform standards in health care before deployment.

This study shall act as a cornerstone for developing a health AI tool assessment framework.