In this work, we present an integrated framework employing multimodal Retrieval-Augmented Generation (RAG) combined with human-in-the-loop oversight to address a spectrum of applications in healthcare IT. This framework is designed to harness both textual and visual data, facilitating comprehensive information extraction for enhanced decision-making across various domains, including analytics optimization, knowledge base synthesis, and diagnostic (Dx) coding accuracy.

Our approach leverages the strengths of multimodal RAG to interpret complex datasets, incorporating visual and textual clues for a richer analysis. The inclusion of human-in-the-loop mechanisms further refines the system’s output, ensuring both scalability and precision in applications ranging from function calling in analytics platforms to the generation of dynamic, accurate knowledge repositories and the substantiation of Dx codes through precise ICD-10-CM mapping.

We detail the architecture’s design principles, emphasizing its modular nature that allows for scalability across different healthcare IT needs. Through case studies, we demonstrate the architecture’s effectiveness in improving operational efficiencies, enhancing the reliability of clinical data interpretation, and supporting comprehensive knowledge synthesis. This work underscores the potential of multimodal RAG systems in transforming healthcare IT by providing a robust, scalable solution for integrating diverse data types into actionable insights.