Common Technical Interview Questions for AI Roles, and How to Approach Them
Common AI interview questions, from LLM basics, RAG and evaluation to system design and ethics, with how to answer so interviewers see how you think.
A candidate answers "What is RAG?" fluently, like reciting a textbook. Then the interviewer asks, "If your RAG system gives a wrong answer, how would you find the cause?" and there is a long silence. AI interviews today don't test how well you remember definitions; they test whether you can think through real problems.
What is a technical AI interview?
It's an interview that tests fundamental understanding, system design and judgement in applying AI to real work. For roles such as AI Engineer, ML Engineer or AI Solutions Architect, there is usually a Q&A round, a coding round and a system design round. The questions below are common patterns, but every company asks differently.
Category 1: LLM fundamentals
"Explain tokens and the context window to a non-technical person."
Approach: use simple language, then connect it to real consequences: long documents beyond the context must be split, and the number of tokens affects cost.
"Why do LLMs hallucinate?"
Approach: explain that the model generates likely text from patterns it learned rather than looking up facts, then suggest ways to reduce it: answer from attached documents, require cited sources, and allow "I don't know" when information is missing.
"How do fine-tuning and RAG differ, and when would you use each?"
Approach: RAG suits knowledge that changes often and needs citable sources; fine-tuning suits adjusting the style or behaviour of answers. Many systems use both.
Category 2: RAG and LLM applications
"If RAG gives a wrong answer, how do you debug it?"
Approach: break it into stages. Did it retrieve the right documents (retrieval)? Do those documents contain the answer? Did the model use them correctly (generation)? Then fix one point at a time, for example by adjusting chunking, adding reranking or revising the prompt.
"What special challenges do Thai documents bring?"
Approach: Thai has no spaces between words, mixes spoken and written language, and has several spellings for transliterated words. You need to test word segmentation and search with users' real questions.
Category 3: Evaluation and production
"How do you measure the quality of an LLM system?"
Approach: talk about test sets with reference answers, clear criteria, combining human review with LLM-assisted grading, and monitoring quality after launch. Good answers usually include an example from your own project.
"If API costs exceed the budget, what would you do?"
Approach: first measure where the cost comes from, then propose options such as smaller models for simple tasks, caching and shorter context, while checking that quality still meets the bar.
Category 4: System design
"Design an AI assistant that answers customer questions for a bank."
Approach: start by asking questions back: who are the users, how sensitive is the data, when must it hand off to a human? Then lay out components such as RAG, guardrails, authentication, logging and evaluation. Interviewers score the reasoning behind trade-offs more than a pretty diagram.
Category 5: Ethics and judgement
"Have you ever decided not to use AI for a task? Why?"
Approach: answer with reasons about risk, personal data under the PDPA, or accountability when AI is wrong. This question tests whether you know the limits of the tools you use.
Things to watch out for
- Don't memorise answers. Interviewers often dig deeper until they can tell whether you really understand.
- Don't use AI to help you answer during the interview unless the company allows it.
- If you don't know, say so and explain how you would find out. That beats guessing.
Interviewers aren't looking for someone who knows every answer, but for someone who knows how to find it.
How to prepare
- Read the job description and note the technologies and work the role emphasises.
- Choose 1 or 2 of your own projects and prepare to explain the problem, your decisions and the results. If you don't have any yet, see 10 AI Portfolio Projects.
- Practise answering the questions in this article out loud, timing 2 to 3 minutes each.
- Practise system design by sketching on paper and explaining every trade-off.
- Rehearse with someone else or with AI and ask for feedback. See How to Practise Interviews with ChatGPT and Claude.
To practise with questions tailored to real roles, try Plants.work's Interview Lab, and when you're ready, search AI roles on the jobs page.