On assessing AI Risks and Opportunities
A Group Support System methodology for AI risk and opportunity assessments: a structured, collaborative approach that brings legal, technical, operational, and HR perspectives together. It reduces bias, hierarchy, and blind spots, turning debate into accountable decisions and actionable risk treatment across the AI lifecycle.
What we were trying to say.
Most AI risk assessments fail before they even start. Not because the frameworks are wrong, but because people apply them without thinking about where they are. A hospital, a startup, and a government agency face completely different risks from the same AI tool. Yet everyone reaches for the same generic checklist. That's the problem Vincent van Dijk and I wanted to dig into, and Vincent's sharp instinct for where theory meets organisational reality shaped a lot of how we framed it.
Drawing on research from Karlsruhe, we mapped out seven areas where AI assessment consistently breaks down: ethics, societal impact, legal fit, trade secrets, explainability, transparency, and the fact that AI systems change over time in ways an initial review won't catch. These aren't abstract concerns. They're the things that bite organisations later.
Our answer isn't another framework on top of the others. It's to start with your context (your use case, your risk appetite, your constraints) and then run AI against NIST's seven characteristics: validity, safety, resilience, transparency, explainability, privacy, and fairness. Context first, scoring second.
The examples we use are there to make the stakes real. Amazon's hiring tool didn't become biased on purpose. It learned from historical data that was already biased. ChatGPT's "DAN" and "Ranti" jailbreaks show that even well-designed AI can be steered outside its guardrails by a determined user. These aren't edge cases. They're the normal failure modes.
The other thing we argue is that this kind of assessment can't be done by one person, or by the team that built the system. Bias, blind spots, and hierarchy all distort a solo review. What we propose instead is a Group Support System: a structured way of bringing legal, technical, operational, and HR perspectives into the same room, reducing the tendency to defer to the most senior voice, and turning disagreement into an actual action plan rather than a compromise nobody owns.
Three things to take away.
- Figure out your organisational context before you start scoring anything: use case, risk tolerance, capabilities, limits.
- Make the assessment cross-functional and structured, not a self-review by the people closest to the system.
- Treat it as a lifecycle activity, not a one-time sign-off, and let the outcome drive a real decision: treat, own, outsource, or exit.
The method we're proposing is grounded in the literature, mapped onto NIST, and designed to produce something actionable rather than a report that sits in a drawer. Multiple perspectives, structured debate, decisions tied to someone's name. Vincent and I believe that's exactly the discipline AI governance is missing right now.
https://isaca.nl/an-exploration-of-ai-risk-collaborative-assessment-methodology/