RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting


RiskRAG Overview
Architecture overview of RiskRAG: a RAG pipeline retrieving risks from model cards and AI incident database, structuring and prioritizing them, mapping to concrete use cases, and linking mitigation strategies.

RiskRAG tackles the blind spots in today’s AI documentation by transforming vague, copy-and-pasted “model cards” into rich, context-aware risk reports. AI is already woven into our daily lives—from the phone in your pocket to the algorithms deciding your job application—yet, among 450,000 models on HuggingFace, 86% omit any mention of risks and 96% of the rest simply recycle identical warnings.

By mining thousands of human-written model cards and hundreds of real-world AI incident reports, RiskRAG surfaces both model-specific and use-specific hazards in clear, actionable language. For example, it can warn a newsroom that speech-to-text tools have misinterpreted accented voices into slurs, or flag privacy pitfalls when handling sensitive user data.

Beyond merely pointing out dangers, RiskRAG pairs each risk with targeted mitigation strategies—suggesting content filters to curb misinformation, recommending multilingual training data to avoid language bias, or advising differential privacy techniques to safeguard personal information.

User studies with developers, designers, and journalists confirm that RiskRAG’s reports are not only more informative but also more trustworthy and practically useful. Like a reliable nutrition label for food, RiskRAG equips AI practitioners with the detailed insights they need to build safer, fairer, and more transparent AI systems.



Publications

  • RiskRAG: A Data-Driven Solution for Improved AI Model Risk Reporting. CHI 2025 PDF

Code and data


We'll never share your email with anyone else.

N.B.: If you do not receive the instruction message within a few hours, please check your junk/spam e-mail folder just in case the email was moved there.