Faculty Intelligence Meets Artificial Intelligence- Ai-driven Excellence in Programmatic Assessment for Interprofessional Education Across Health Professions
Learning Objectives: After attending this session, the learner will be able to:
1) Efficiently analyze assessment data through an AI platform
2) Provide feedback generated by a Large Language Model (LLM) to students
3) Utilize AI tools for educational improvement
Background: Interprofessional education (IPE) is crucial for preparing health professional students for collaborative healthcare practice, a requirement mandated by various accrediting bodies. Wayne State University's (WSU) Interprofessional Telehealth Team Visit (IPTTV) facilitates this by bringing together over 800 students and ten faculty members from seven different health professional programs. While IPTTV successfully delivers IPE experiences and collects rich quantitative and qualitative data from multiple assessment tools (Interprofessional Socialization and Valuing Scale (ISVS), McMaster-Ottawa Scale, Gap Kalamazoo, and reflective assignments), a significant challenge is the timely and efficient analysis and dissemination of this large dataset to inform programmatic improvements and provide students with feedback. Currently, limited faculty capacity prevents comprehensive data analysis and detailed feedback delivery, impacting opportunities for student growth and program refinement.
Objectives: This project, supported by a WSU Assessment Grant, seeks to enhance IPTTV assessment practices. The specific objectives are to achieve efficient analysis of quantitative data through AI support, provide meaningful and timely feedback to students, and train IPTTV faculty in the use of AI to enhance assessment processes. The goal is to create a sustainable plan for analyzing IPTTV data to drive continuous quality improvement and help participating programs meet their accreditation standards.
Methods: The project team recruited a statistician and an expert consultant in AI-powered data analysis to provide support. The team then identified an AI-powered data management platform for efficient data processing and analysis. The next steps included the development of AI algorithms to analyze quantitative assessment data. Subsequently, data from the IPTTV winter 2025 assessment instruments were analyzed. Finally, workshops will be conducted to train IPTTV faculty in the effective use of the identified AI-powered data management platform.
Results: Using Google AI tools (Vertex, API, Gemini, and Notebook LM), an automatically generated email provided post-event feedback to learners who participated in the IPTTV educational experience. The email summarized results from various assessments including the ISVS survey, ratings provided by teammates on individual and team performance, and evaluations from standardized patients on team performance. The report presented individual and team scores and provided benchmark comparisons against peers within their discipline and all participants or teams. Based on these comparisons, the email offered insights and suggested reaching out to a contact person for advice if scores were lower than benchmarks.
Expected Outcomes: 1) More efficient and comprehensive analysis of IPTTV assessment data, 2) students will receive timely, detailed, and actionable feedback, enhancing their learning and professional development, and 3) reduced faculty workload related to data analysis and sustainable assessment processes. By leveraging AI, the project enabled data-driven insights to refine the IPTTV curriculum and tailor educational strategies.
Active Learning Strategies: A demonstration of AI tools for assessment will be completed during this workshop inviting the audience to explore the discussed tools.