Assessing AI Risk Before You Deploy It in Your Organisation
A step-by-step way to assess the risk of an AI project, from mapping impact and scoring likelihood and severity to choosing controls, based on NIST and OWASP.
The customer service team at an online store wants to launch an AI chatbot that answers return questions 24 hours a day. The demo looks great: fast, polite, fluent in Thai. Everyone wants to go live next week. But nobody has asked the important questions yet. What happens if the chatbot promises a refund the policy does not cover? What if someone types a trick instruction to make it reveal another customer's data? Assessing risk before using AI means asking these questions before your customers do.
What is an AI risk assessment?
An AI risk assessment identifies what harm an AI use case could cause, how often it might happen, how severe it would be, and how to reduce it. NIST AI RMF 1.0 splits this work into Map (understand the context and risks), Measure (analyse and assess) and Manage (treat and monitor), with Govern as the cross-cutting foundation.
NIST also lists the characteristics of trustworthy AI: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair. They make a good set of questions to start from.
Step 1: Map the context and who is affected
- Does the AI decide on its own, or does it recommend to a human who decides?
- Does the output reach customers or employees, or is it internal only?
- Does it use personal data?
- If it is wrong, who is harmed, and can the harm be reversed?
An international lens helps. The EU AI Act, which entered into force on 1 August 2024, classifies AI systems by risk level. The European Commission gives AI used for recruitment as an example of high risk, while chatbots must tell users they are talking to a machine. Most Thai businesses are not directly subject to it, but it is a useful way of thinking.
Step 2: Go through the common risks
Output quality risks
NIST AI 600-1, the Generative AI Profile, lists 12 generative AI risks, including confabulation (confidently made-up information), data privacy, information security, harmful bias and intellectual property.
Security risks
The OWASP Top 10 for LLM Applications 2025 ranks prompt injection first, followed by sensitive information disclosure. It also lists excessive agency, which means giving AI more permissions than it needs.
Business risks
Runaway costs, dependence on a single provider, and reputational damage when AI gets something wrong in front of customers.
Step 3: Score and prioritise
Use a simple table: score likelihood from 1 to 5 and severity from 1 to 5, then multiply. High scores need controls before launch; low scores may be acceptable but should be monitored. In the returns chatbot example, the risk of promising refunds outside policy might score likelihood 3 and severity 4, so it must be handled first.
Step 4: Choose controls
- Limit the AI to answering from approved policy documents only.
- Always hand money matters and exceptions to a human.
- Give the AI access only to the data it needs.
- Test with hard questions and trick prompts before going live.
- Keep logs so you can review what happened.
A risk written on paper can always be managed. The dangerous risk is the one nobody talks about.
Watch out for
Risk scores are estimates, not facts. Have several people score them, including the business owner, IT and legal, and reassess whenever you change the model, the data or the scope. For legal questions, consult a professional.
How to get started
- Pick one AI project that is about to go live.
- Answer the four context questions from Step 1.
- List risks using the NIST and OWASP headings, then score them.
- Assign a control and an owner to every high risk.
- Schedule a reassessment 30 days after launch.
If your organisation is not sure how ready it is, SCALE runs an AI Readiness Diagnostic workshop to assess this before you start; see scale.co.th. For the bigger picture, read AI Governance for SMEs, or for data questions, PDPA and AI.