Synopsis
This poster examines how information integrity is negotiated in AI-mediated healthcare and what this reveals about the Ethics of Intelligence. Drawing on cross-national policy council workshops, it explores how stakeholders prioritise safeguards such as human-in-the-loop oversight, accountability, reliability, privacy, and security. Findings suggest integrity is sustained not merely through model performance, but through governance mechanisms and institutional responsibility. The study offers cross-sector lessons for AI governance in healthcare and higher education.
Abstract
As generative AI systems become embedded in high-stakes decision environments, questions of integrity can no longer be confined to technical performance metrics such as robustness, calibration, or accuracy (Kaur et al., 2022; Li et al., 2023). In healthcare contexts, AI-generated information increasingly informs clinical judgment, institutional governance, and policy formulation (Garcia-Vidal et al., 2019; Secinaro et al., 2021; WHO, 2021). Failures of information integrity in such environments may undermine patient safety, public trust, and institutional accountability, particularly where responsibility and oversight are insufficiently defined (Gerke et al., 2020; Sun & Medaglia, 2019). These developments raise central questions aligned with the conference theme of Ethics of Intelligence: how do emerging AI systems reshape responsibility, oversight, and ethical decision-making across professional domains?
This poster presents preliminary findings from the Yathiqu Trustworthy AI in Healthcare project, a cross-national mixed-methods study combining structured literature synthesis with participatory policy council design workshops conducted in three locations (Dubai, Boston, and San Francisco) (Khan, 2024).
A total of three policy council workshops were conducted using a structured, design-thinking-based methodology developed for this project. Each workshop was facilitated through guided multi-stage sessions led by the research team, ensuring consistency in delivery across sites.
Fifty-two stakeholders representing clinicians, AI developers, policymakers, administrators, pharmacists, patients, and civil society actors participated. Participants were distributed across professional categories, enabling comparative analysis of perspectives across clinical, technical, governance, and public domains.
The workshops employed a mixed qualitative and structured induction approach, including role-based definition exercises, characteristic ranking, group voting, persona clustering, and trade-off analysis. These activities generated both qualitative insights (narratives, interpretations) and structured data (rankings, consensus measures, prioritisation patterns).
The study builds on established trustworthy AI governance frameworks emphasising accountability, transparency, and human oversight (EU, 2019; Wing, 2021; WHO, 2021).
Across roles and locations, human-in-the-loop oversight emerged as the highest-ranked integrity safeguard. Participants consistently associated trustworthy AI not primarily with abstract notions of explainability or model accuracy, but with the ability of humans to interpret, question, and override AI outputs before action. Additional highly prioritised safeguards included accountability mechanisms, privacy and security protections, and system reliability. Technical attributes such as explainability were discussed as necessary but insufficient in isolation, reinforcing critiques that model-level properties alone cannot guarantee trustworthy outcomes (Zicari et al., 2021; Reinhardt, 2023).
Role-based differences were also evident. Policymakers and administrators emphasised auditability and institutional responsibility; clinicians foregrounded reliability and professional judgment; patients expressed concerns regarding opacity and loss of agency. These divergences suggest that integrity risks frequently arise at professional boundaries where AI-generated information is transferred, interpreted, and operationalised.
The findings support conceptualising information integrity as a negotiated socio-technical condition sustained through governance mechanisms, human agency, and institutional accountability (Stix, 2022; Wing, 2021). While situated in healthcare, the study offers cross-sector insights relevant to higher education, where AI systems increasingly influence research production, assessment practices, and institutional decision-making. By bridging empirical stakeholder evidence with AI governance theory, this work contributes to broader debates on the Ethics of Intelligence and institutional responsibility in AI-mediated environments.
Keywords
Information integrity; Ethics of Intelligence; Trustworthy AI; Human-in-the-loop oversight; Institutional accountability
References
EU (2019). Ethics guidelines for trustworthy AI. European Commission. https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai
Garcia-Vidal, C., et al. (2019). Artificial intelligence in clinical decision-making. eBioMedicine, The LANCET Discovery Science. 46, 27-29. https://www.thelancet.com/article/S2352-3964(19)30454-2/fulltext
Gerke, S., Minssen, T., & Cohen, G. (2020). Ethical and legal challenges of artificial intelligence-driven healthcare. Artificial Intelligence in Healthcare, 295–336. https://pmc.ncbi.nlm.nih.gov/articles/PMC7332220/
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2022). Trustworthy artificial intelligence: A review. ACM Computing Surveys, 55(2), 1–38. https://dl.acm.org/doi/10.1145/3491209
Khan, Z. R. (2024). Using AI Simulation for Building Trust (Yathiqu) in AI and AI Literacy. Dubai Future Foundation. https://dubairdi.ae/grant-initiatives-using-ai-simulation-for-building-trust-yatiku-in-ai-and-ai-literacy/
Li, B., Qi, P., et al. (2023). Trustworthy AI: From principles to practices. ACM Computing Surveys, 55(9), Article 177. https://dl.acm.org/doi/10.1145/3555803
Reinhardt, K. Trust and trustworthiness in AI ethics. AI Ethics 3, 735–744 (2023). https://doi.org/10.1007/s43681-022-00200-5
Secinaro, S., Calandra, D., Secinaro, A. et al. (2021) The role of artificial intelligence in healthcare: a structured literature review. BMC Med Inform Decis Mak 21, 125 (2021). https://pubmed.ncbi.nlm.nih.gov/33836752/
Stix, C. (2022). Artificial intelligence by any other name: A brief history of the conceptualization of “trustworthy artificial intelligence.” Discover Artificial Intelligence, 2, 26. https://doi.org/10.1007/s44163-022-00041-5
Sun, T. Q., & Medaglia, R. (2019). Mapping the Challenges of Artificial Intelligence in the Public Sector: Evidence from Public Healthcare. Government Information Quarterly, 36(2), 368-383. https://doi.org/10.1016/j.giq.2018.09.008
WHO (2021). Ethics and governance of artificial intelligence for health. World Health Organisation. https://www.who.int/publications/i/item/9789240029200
Wing, J. M. (2021). Trustworthy AI. Communications of the ACM, 64(10), 64–71. https://dl.acm.org/doi/10.1145/3448248
Zicari, R. V., et al. (2021). Z-Inspection®: A process to assess trustworthy AI. IEEE Transactions on Technology and Society, 2(2), 83–97. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9380498