Applications for Stage I of the American Medical Informatics Association (AMIA) 2023 Artificial Intelligence Evaluation Showcase are open until December 9. View the informational PDF for additional details.

Participants in the AI Evaluation Showcase will evaluate an example of an AI system that is being used in health care. The purpose of this is to create a community of researchers and practitioners who can come to a consensus on the best practices for evaluating AI safety and effectiveness. To participate, applicants must form scientific teams. These teams will submit a paper, podium abstract, or poster proposal of their evaluation of an AI system. These evaluations will then be presented at AMIA conferences.

Evaluations are split into three stages. During Stage I: Technical Performance Studies, teams will submit a description of the AI system, the results of a study on its algorithm performance, and an outline of how they will complete the rest of their evaluation. Stage I submissions will be presented at the 2023 AMIA Informatics Summit in March.

During Stage II: Usability and Workflow Studies, teams will submit an evaluation addressing the usability and workflow aspects of their AI system by mid February. To participate in this stage, teams are required to either have participated in Stage I or include results of a technical performance study in their submission. These submissions will be presented at the 2023 AMIA Clinical Informatics Conference in May.

Finally, for Stage III: Health Impact Studies teams will summarize a comprehensive evaluation of their AI system. This summary will include results from their Stage I and II submissions, as well as new findings from further research. Stage III submissions are tentatively due by the start of June and will be presented at the 2023 AMIA Annual Symposium next November.

To learn more and apply for Stage I, please visit z.umn.edu/AMIA-AI-showcase.