Jean Feng, Ph.D. Department of Epidemiology and Biostatistics, University of California, San Francisco and UCSF-UC Berkeley Joint Program in Computational Precision Health

After a machine learning (ML)-based system is deployed in clinical practice, performance monitoring is widely recognized to be a crucial component to ensuring the safety and effectiveness of the algorithm over time. Nevertheless, designing an effective monitoring strategy is highly complex given the multitude of design decisions, including the data source (e.g. observational versus interventional data), the performance criteria tracked, the assumptions required by the procedure, and more. After reviewing existing approaches to designing clinical AI monitoring systems, we discuss the need for a systematic framework for designing post-market monitoring systems, the unique considerations in this setting, and the importance of causal thinking.