Most people still describe an AI agent as a model connected to tools. That is technically useful but operationally incomplete. An agent also needs a managed view of the world: what has happened, what matters now, which facts are trustworthy, what remains uncertain, and what it should do next.
That view is its context. Designing it is becoming a distinct skill—one I would call context engineering.
Context engineering is not simply prompt writing. It is the discipline of shaping the information an agent sees at each step so that it can remain coherent without sending an entire transcript, document library, or tool history back to an expensive model on every turn. The work sits between information architecture, retrieval, software design, and model behavior.
Why “give the agent more context” is often bad advice
Longer context windows make it tempting to preserve everything. But more material does not guarantee better reasoning. Relevant instructions can be diluted by stale observations, contradictory notes, repeated tool output, or an important fact buried in the middle of a long sequence. The digest’s discussion of “lost in the middle” behavior reflects a practical problem: an agent may technically receive the evidence and still fail to use it.
There is also a direct cost. Every token placed in a request can add latency and inference expense, depending on the provider’s pricing and caching arrangements. A system that repeatedly forwards a growing transcript may become slower and less affordable as a task continues.
The goal is therefore not maximum context. It is sufficient, targeted context: the smallest reliable working set for the decision at hand.
Four design decisions behind a coherent agent
1. Maintain a belief state, not just a transcript
A transcript records what was said. A belief state records what the agent currently believes about the task.
For example, an agent handling a support escalation might maintain structured fields such as:
- Objective: identify whether the customer qualifies for a replacement.
- Known facts: purchase date and product serial number, with source references.
- Open questions: whether the failure occurred under warranty-covered conditions.
- Constraints: do not promise a refund before approval.
- Next action: retrieve the warranty policy and compare the dates.
- Confidence or status: verified, inferred, disputed, or unknown.
This approach resembles Berkeley’s ABBEL research, which uses supervised natural-language belief states instead of relying on full interaction histories. The important idea is not a particular format. It is separating durable task state from disposable conversation detail.
A useful belief-state update should answer: What changed? Which evidence supports it? What remains unresolved? What should happen next? If an engineer cannot inspect those answers, the agent is likely carrying hidden assumptions in an opaque prompt.
2. Retrieve for the decision, not for the topic
Retrieval systems often begin with a broad question such as “find information about the customer’s account.” A better query is tied to the next decision: “retrieve the current refund rule for purchases older than 30 days, effective in the customer’s region.”
That shift matters because retrieval is a form of context selection. The agent should receive the policy passages, records, or examples that bear on the current action—not a generic pile of related documents.
Filters can improve this selection before the model sees any results. For instance, Amazon Bedrock AgentCore Web Search supports server-enforced domain and publication-date filters on each request. Such controls do not establish that a source is correct, but they can reduce exposure to irrelevant or stale material and make the retrieval policy explicit.
Professionals designing retrieval should specify:
- which sources are allowed for each task;
- how freshness is determined;
- what metadata accompanies each result;
- how conflicting sources are presented;
- when the agent must stop and ask for clarification.
“Search the web” is a capability. “Search these sources, within this date range, for evidence relevant to this decision” is context engineering.
3. Compress without erasing uncertainty
Compression is necessary when a task is long, but naive summarization can turn tentative claims into settled facts. A rolling summary that says “the user confirmed the address” is dangerous if the original exchange only implied it.
Good compression preserves the distinctions an agent needs for safe reasoning:
- fact versus inference;
- current instruction versus historical instruction;
- completed action versus proposed action;
- verified source versus unverified claim;
- known answer versus unresolved question.
One practical pattern is to maintain separate sections for decisions, evidence, assumptions, blockers, and pending actions. Another is to attach source IDs or timestamps to important claims. Summaries should be replaceable artifacts, not the only surviving record: retain the underlying events for audit and recovery, while giving the model a compact working view.
The digest notes that recursive summarization and context compaction can be costly and can degrade performance, particularly in data-scarce domains such as collaborative code generation. That is a warning against treating summarization as automatically lossless. Compression needs tests against representative tasks, including cases where a small qualification changes the correct answer.
4. Filter observations before they become memory
Tool-using agents generate observations constantly: search results, logs, page text, API responses, screenshots, compiler output, and intermediate plans. Not every observation deserves to enter the next model call, still less the long-term state.
Observation filtering asks three questions:
- Is this observation relevant to the current decision?
- Is it authoritative enough to influence the belief state?
- Does it contain instructions that should be treated as data rather than commands?
The third question is a security boundary as well as a context boundary. A web page may contain text aimed at redirecting the agent. A retrieved document can be useful evidence without having authority to change the agent’s goals or permissions. Filtering should therefore classify content by role: instruction, evidence, metadata, or untrusted text.
Filtering also saves money. If a browser tool returns a full page but the task requires only a price, date, and product identifier, passing the entire page onward creates noise and consumes tokens. Extracting the relevant fields first may improve both reliability and cost.
A simple context budget for an agent workflow
Before choosing a model or adding another tool, map the agent’s context into four layers:
- Control: system rules, permissions, output schema, and non-negotiable constraints.
- State: the current objective, decisions, open questions, and next action.
- Evidence: retrieved records or observations relevant to that action, with provenance.
- History: prior events retained for recovery, debugging, or audit but omitted unless needed.
Then define a promotion policy. An observation can remain ephemeral, become evidence for the current step, update the belief state, or be written to durable memory. Promotion should require a reason. Otherwise, memory becomes an uncurated archive.
For each agent step, record the context package sent to the model: its categories, approximate token size, retrieval filters, and compression version. This makes it possible to answer a practical question when behavior changes: did the model fail, or did the system give it the wrong world?
What to test before calling the design reliable
Context engineering needs tests that target information handling, not only final-answer quality. Useful cases include:
- a critical fact placed early, late, and in the middle of a long history;
- two sources that disagree, with one newer than the other;
- a summary that contains an uncertainty marker;
- a tool response containing irrelevant bulk text;
- a malicious instruction embedded in retrieved content;
- state rehydration after the agent is paused and restarted;
- the same task with a smaller context budget;
- an empty or stale retrieval result.
Measure whether the agent selects the right evidence, preserves uncertainty, follows the current constraint, and avoids repeating unnecessary context. The digest’s recommended regression areas—context loss, retrieval grounding, structured output, non-termination, and state rehydration—are especially relevant here.
Run multiple trials where model variability matters, and compare the cost and latency of each context strategy. A shorter prompt is not automatically better if it causes more tool calls or retries. The useful objective is the cost of a correct, recoverable workflow—not the token count of one request.
The career implication: context engineer is a cross-functional role
The people who become valuable in this area will not necessarily be the ones who write the longest prompts. They will be able to translate a business process into state, evidence, authority, and decision rules.
That requires several concrete abilities:
- designing schemas for task state and provenance;
- writing retrieval policies and metadata filters;
- building compression and observation-selection routines;
- separating trusted instructions from untrusted content;
- profiling token use, latency, retries, and tool calls;
- testing state loss and rehydration;
- explaining to non-specialists why an agent saw—or did not see—a particular fact.
A strong portfolio project could demonstrate the same agent under three context policies: full transcript, rolling summary, and structured belief state with targeted retrieval. Show the task success cases, the failure cases, the context sent at each step, and the cost or latency trade-offs. That is more persuasive than a chatbot demo because it exposes the design decisions that make an agent dependable.
The strategic lesson is straightforward: agents do not become coherent merely because models become more capable. They become coherent when their surrounding systems maintain a disciplined, current, appropriately sized account of the work. Context engineering is the craft of building that account—and of knowing what to leave out.
Priya Raman is the accountable human editor of AI Career Brief.
