AI Observability: Logging and Quality Checks for AI in Production

Once AI is live you need to see inside it: what to log, how to measure quality, how to keep personal data out of logs, with tools like OpenTelemetry and Langfuse.

An employee benefits chatbot had been live for two months. Everything seemed fine until an employee posted a screenshot in a LINE group showing the AI giving an answer about leave days that contradicted company policy. The team running the system wanted to know how often this happened, when it started, and whether the cause was the reference documents or the model itself. But there was nothing to look back at, because no logs had been kept from the start. This is why AI systems in production need observability, just like any other important software.

What is AI observability?

AI observability is the ability to see what an AI system is doing: what data it receives, what it answers, how many resources it uses and how good the quality is, so you can detect problems early and fix them precisely. Ordinary software usually fails loudly, for example a page goes down. AI often fails silently: it keeps answering normally, only the answers are wrong, and nobody knows unless you deliberately measure.

In NIST AI RMF 1.0, this work sits in the Measure function, which uses quantitative and qualitative tools to monitor risk, and the Manage function, which includes planning responses and communication when incidents happen.

What should you log?

Basics for every model call

• Time, user or session ID, and which feature was used
• Model name and prompt version
• Input and output tokens, response time and errors

Data for quality analysis

• Questions and answers (stored carefully; see Watch out for)
• Documents retrieved by the RAG system
• Tools called by an agent and the result of each step
• User feedback, such as thumbs up or down

So you do not have to invent a format, OpenTelemetry, an open observability standard, has semantic conventions for generative AI that standardise how to record things like model parameters, response metadata and token usage. According to the OpenTelemetry blog, capturing prompt and response content is opt-in.

How to check quality in production

1. Human spot checks

Have a subject expert sample conversations every week and score them against agreed criteria, such as accuracy, completeness and appropriate tone.

2. A permanent regression set

Keep a set of important questions with correct answers, and rerun it every time you change the model, prompt or documents to see what got worse.

3. AI-assisted scoring

LLM-as-a-judge means asking another model to evaluate answers against criteria. It lets you check more volume, but compare it with human scores regularly.

4. Metrics and alerts

For example, thumbs-down rate, hand-off-to-human rate, average response time and daily cost. Alert immediately when a number moves unusually.

Tools you can use

Besides connecting OpenTelemetry to your existing monitoring, there are specialised tools such as Langfuse. Langfuse's documentation describes it as an open-source, self-hostable platform with tracing, prompt versioning, LLM-as-a-judge evaluation, and dashboards for quality, cost and latency.

AI without logs is like an employee whose work nobody ever checks. You only find out it is wrong when a customer tells you.

Watch out for

• Logs are sensitive data: user questions may contain personal data. Mask it before storing, limit who can access logs and set retention periods. Read more in PDPA and AI.
• More is not always better: logs nobody reads are pure cost. Assign someone to review them every week.
• AI scores are not the whole truth: treat them as signals, and keep humans in the loop for important decisions.

How to get started

1. Pick one AI system that is already live.
2. Start logging the basics: model, prompt version, tokens, latency and errors.
3. Add a feedback button for users and set rules for masking personal data in logs.
4. Build a test set of the 20 to 50 most important questions.
5. Have the team review the dashboard and sample conversations every week.

If you have not assessed risk before launch, start with Assessing AI risk before deployment. And if you want to grow a career in AI and data, browse roles at Plants.work Jobs.