🚨 Pulse of Innovation 🚨

Breaking news in the healthcare AI

Bendable CHIP: Multipurpose Microprocessor

Use semiconductors/microprocessors and other words a CHIP has become very critical in our daily lives, enabling technology from public infrastructure to personal devices including healthcare. An important limitation of these chips is that they are usually not bendable limiting their utility in certain fields. The recent article in the Nature describes a chip fabricated with indium gallium zinc oxide thin-film transistors on a flexible polyimide substrate which allows it to be a bendable chip at a very low cost. The practical application of this are limitless.

The CHIP Technology

This Flex-RV, a 32-bit RISC-V microprocessor based on an open-source 32-bit RISC-V central processing unit (CPU) extended with machine learning (ML) features fabricated with indium gallium zinc oxide (IGZO) thin-film transistors (TFTs), enabling an ultralow-cost and conformable microprocessor for emerging applications. The new non-silicon chip ultra-thin, programmable, open-sources, able to process data as fast as 60kHz, low power use of 6mW and above all bendable while still functioning. The manufacturing of the chip is also very cost effective.

Healthcare Use

Major use of the bendable CHIP could be in wearable technologies, single use implants such as brain-computer interface, single use test devices such as glucose monitors and chips.

Test Case in Healthcare

The chip comes with ML accelerator feature that allows special AI applications. The chip was tested for the ECG anomaly detection. They developed a Tiny Machine Learning (TinyML) model using Tensorflow28 to perform ECG anomaly detection using the ECG5000 time series classification dataset. Our tiny neural network consists of a one-dimensional convolution followed by a fully connected layer, an approach commonly used in simple time series classification.

This is a classic use of bendable chip technology in healthcare. Being a low cost chip with bending capacity can also be leveraged in the brain-computer interface where this thin chip can improve the data capture process while bent in tight brain areas.

🧑🏼‍🔬 Bench to Bedside👨🏽‍🔬

Developments in healthcare AI research and innovations

Standardizing Human Evaluations of Large Language Models in Healthcare: The QUEST Framework

• Generative AI is increasingly used in healthcare by both professionals and patients. However, the assessment of Large Language Models (LLMs) lacks strict guidelines, with most evaluations relying on automated metrics. Given the growing integration of LLMs in healthcare, it is crucial to standardise human evaluations to ensure safety and effectiveness.

• A recent article published in Nature Digital reviewed human evaluations of LLMs across various medical specialties. The study aimed to identify and analyse methods used in these evaluations to develop actionable guidelines for standardising human studies on LLMs - QUEST framework

  Key Findings:

1. Applications of LLMs: The highest use of LLMs was found in clinical decision support (28.1%), followed by medical education and examination (24.8%). Other applications included patient education, medical questioning, administrative tasks, and mental health support.

   

   Source: Nature Digital

2.  Specialties Evaluated: Radiology, Urology, and General Surgery were the most evaluated specialties. Other frequently studied fields included Plastic Surgery, Otolaryngology, Orthopedics, Psychiatry, Emergency Medicine, and General Medicine.

3. Study Design: Most studies involved sample sizes of around 100 participants, with only one study exceeding 2,000. Both qualitative and quantitative measures were common, though standardized checklists and recruitment protocols were rare. Likert scales were frequently used, but blinding techniques were often unclear.

4. Endpoints and Analysis: Most studies compared LLM outputs to human-generated responses, with some evaluating real-world versus controlled scenarios. Statistical analyses included t-tests, Cohen’s Kappa, and Intraclass Correlation Coefficients, while comparisons to benchmarks used t-tests, ANOVA, and Mann-Whitney U tests.

• The QUEST Framework:

  1. Quality of Assessment: Multidimensional analysis of LLM-generated information.

  2. Understanding and Reasoning: Evaluation of the model's logical reasoning.

  3. Expression Style and Persona: Clarity and empathy in communication.

  4. Safety and Harm: Detection of biases, errors, and fabrications.

  5.  Trust and Confidence: Assessing user trust in LLM outputs.

  The framework suggests three phases for designing evaluations:

  1. Planning: Define goals, user tasks, sample size, and evaluation criteria.

  2. Implementation and Adjudication: Execute the plan, collect data, and achieve consensus among evaluators.

  3.  Calculation and Scoring: Generate final scores based on the QUEST dimensions and compare them to quantitative evaluations.

• Conclusion:

  The QUEST framework offers a starting point for the standardisation of LLM evaluations in healthcare, though it may not be universally applicable across all specialties or align with institutional policies. Future studies can expand the framework to integrate automated assessments.

  

  Demonstration of QUEST Framework (Source: Nature Digital)

🧑🏽‍⚕️ AI in Clinic 🏥

Developments in healthcare AI research and innovations

AI - Jargon Buster

Deep Learning Model:

It is a form of machine learning(ML) that uses artificial neural networks which are very similar to the human brain. It learns from vast amounts of data and helps digital systems to learn and make decisions independently without human intervention.

Major areas of use: Genomics, Pathology and Radiology

Foundation Model:

ML model trained on a vast amount of data so it can be easily integrated for a wide variety of applications.

Example : Large Language Models(LLMs) which power chatbots like Chat GPT.

Human-in-the-loop:

A system that uses human and artificial intelligence so human intervention is possible to train and test the system’s algorithm to produce useful and transparent results. The major advantage is overcoming the shortcomings of both forms of intelligence.

Large Language Model (LLM)

ML capable of performing natural language processing tasks including generating and classifying texts and images, conversing, translating languages, predicting and summarising content. It uses a vast data set to achieve these skills.

In healthcare, these models assist in clinical decision support and patient- physician interactions.

Multimodal Model:

ML model that processes and combines different types of data like images, videos and text, to make more accurate determinations, draw insightful conclusions, or make precise predictions about real-world problems.

Examples: Data from an EHR , CXR and Radiologists written descriptions are analysed to arrive at a diagnosis.

🤖 Patient First, AI Second🤖

Ethical and Regulatory Landscape of Healthcare AI

Responsible AI: Addressing the Critical Nursing Data Gaps

• The U.S. Department of Health and Human Services (HHS) has awarded $2 million to the Columbia University to support the development of innovative methods to evaluate and improve the quality of healthcare data used by artificial intelligence (AI) tools. The funding will be used to address the challenge of insufficient knowledge of nursing practice, which can lead to inaccurate data signals and biases in AI algorithms.

• The proposed study aims to harness nursing knowledge to better capture the nuances of nursing data, resulting in more comprehensive, accurate, and transparent algorithms. The objectives include testing and validating different computational methods within a health care process modeling (HPM) framework, generating and validating knowledge graphs, extending multi-modal approaches to HPM-informed scalable computational processes, and building an open-source pipeline for sharing and reusing these processes.

• By combining data science methods with clinical knowledge, decision-making, and behavior, the study seeks to classify and make predictions about patients that are consistent with and can enhance the quality of the data captured. This will help to discover previously unknown patterns and improve the overall accuracy and effectiveness of AI-based healthcare tools.