Organizations that develop or deploy Generative AI solutions in healthcare are subject to more than 70 national and state laws, regulatory rules, and industry standards. Once an organization establishes an AI Governance framework, its policies will include dozens of controls that must be implemented for each AI project. This session describes a subset of these controls that can be automated with current tools:
- Automated execution of medical LLM benchmarks during system testing and when monitoring in production, including coverage of medical ethics, medical errors, fairness and equity, safety and reliability – using Pacific AI
- Automating generation and executing of LLM test suites for custom solutions, including testing for robustness, bias, fairness, representation, and accuracy – using LangTest
- Automated generation of model cards, complying with transparency laws and including explained benchmark results – based on the CHAI draft model card standard
Ben Webster
at NLPLogix
Ben Webster is the Vice President of AI Solutions at NLP Logix in Jacksonville, Florida. He has been with the company for over 11 years, serving many clients across multiple industries and completing hundreds of machine learning projects (at differing levels of success). His focus in on creating prototypes to identify signal with new customers, and creating machine learning roadmaps to help facilitate the long term success of a client’s AI vision.
David Talby
CEO at Pacific AI
David Talby is the CEO at Pacific AI and John Snow Labs, helping companies apply artificial intelligence to solve real-world problems in healthcare and life science.
David is the creator of Spark NLP – the world’s most widely used natural language processing library in the enterprise. He has extensive experience building and running web-scale software platforms and teams – in startups, for Microsoft’s Bing in the US and Europe, and to scale Amazon’s financial systems in Seattle and the UK.
David holds a Ph.D. in Computer Science and Master’s degrees in both Computer Science and Business Administration. He was named USA CTO of the Year by the Global 100 Awards in 2022 and Game Changers Awards in 2023.
