Many important healthcare applications like matching patients to clinical trials, applying the right clinical guidelines, differential diagnosis, clinical coding, patient registries, and real-world data curation depend on understanding the full history of a patient. For a complex patient this may include thousands of pages of texts, hundreds of medical images and lab results, and many thousands of FHIR or structured records. This keynote surveys the current state of the art of how different types of LLMs tackle this challenge:

• Document understanding and information extraction – extracting facts from unstructured text and images, resolving them to standard medical codes & terminologies, and de‑identifying multi‑modal data.
• Reasoning – reasoning LLMs are required to accurately handle uncertain, noisy, duplicate, or irrelevant data, as well as information that changes over time.
• Question answering – answering natural language questions about cohort creation, patient history, or instructing an agent to act, requires a deep understanding of the underlying data model, and a way to explain nuances and follow‑up questions to the user.

This keynote is intended for anyone building Generative AI solutions in healthcare that must reason over clinical data and wants to understand the current best‑in‑class models and architectures.