Context Engineering: The Skill That Comes After Prompt Engineering
Learn what context engineering is, how it differs from prompt engineering, and the techniques AI teams use to feed models the right information.
You already write solid prompts: a role, context, a clear output format. But once you connect AI to company documents, let a chat run long, or give it several tools, the answers start to slip. It forgets what you said at the start or grabs the wrong piece of information. Most of the time the problem is not the wording of the prompt. It is everything the model sees on each turn, and that is the domain of context engineering.
If you want a refresher on the basics first, read What is Prompt Engineering. This article builds on it.
What is context engineering?
Anthropic's engineering article "Effective context engineering for AI agents" (September 2025) describes context engineering as "the natural progression of prompt engineering". Prompt engineering is about writing and organising instructions. Context engineering is about curating everything that ends up in the model's context window: the system prompt, chat history, retrieved documents, tool descriptions, tool results and examples. And it happens again every time something is passed to the model.
Why more context is not always better
Anthropic uses the term context rot: as context grows, the model's ability to accurately recall information from it decreases. It happens across all models, to different degrees. So treat context as a finite resource with an "attention budget" that every new token draws down. The guiding principle in the article is to find the smallest possible set of high-signal tokens that makes the desired outcome most likely.
Four parts to design
1. A system prompt at the right altitude
Anthropic warns about two failure modes: long, brittle if-else rules on one side, and vague guidance that forces the model to guess on the other. The recommendation is to start with a minimal prompt, test it, then add instructions or examples based on the failures you actually observe.
2. Tools that do not overlap
Tool descriptions consume context too. If several tools have overlapping jobs, the model gets confused about which one to use. Good tools have a clear purpose, are robust to errors and return compact results.
3. Canonical examples
Instead of stuffing every edge-case rule into the prompt, pick a few diverse examples that represent the real task. Examples communicate more than long rule lists.
4. Retrieved data
Decide whether to preload all documents or let the AI pull only what it needs. That is the next section.
Techniques for long tasks and AI agents
- Just-in-time retrieval: keep lightweight references such as file paths, links or stored queries, and let the agent load content when needed. The trade-off is some speed. Some systems use a hybrid: retrieve part up front, explore the rest at runtime.
- Compaction: when a conversation nears the context window limit, summarise what matters and start a new window with that summary. The lightest form is clearing old tool results you no longer need.
- Structured note-taking: let the agent write progress notes outside the context, such as a to-do file, and read them back when needed.
- Sub-agents: hand subtasks to other agents working in clean contexts, and have them return only a short summary to the main agent.
Example: an HR benefits assistant
Imagine the HR team at a Bangkok company building a chatbot for leave and benefits questions. The first version pasted the entire employee handbook into the prompt every time. It was slow and sometimes answered from an outdated policy.
The redesigned context worked like this:
- A shorter system prompt stating the role, the scope, and that every answer must cite the policy section.
- The handbook split into sections, with the AI retrieving only the sections relevant to the question.
- Metadata such as the effective date attached to each document.
- Long chats summarised into the points already covered instead of sending the full history.
AI is only as smart as the information in front of it allows. Your job is to choose what belongs there.
Things to watch out for
- Good context does not eliminate hallucination. Still check answers that drive decisions.
- Personal data or company secrets placed in context must follow your organisation's data policy first.
- Compaction can drop important details. Test that summaries keep what you need.
How to start
- Pick one AI task you already run and list everything sent to the model on each turn.
- Remove what the model does not use, then test whether answers get worse.
- Replace large data dumps with retrieval of only the relevant parts.
- Add instructions or examples only where you see the model actually fail.
Context engineering is a skill that teams building AI agents and AI workflows increasingly need. To see where it shows up in real roles, browse AI jobs on Plants.work, or plan your learning path with the AI Career Assistant.