Overview

Context engineering is the practice of controlling what an AI model sees, when it sees it, and in what format, across every step of a multi-turn interaction. It extends far beyond writing a single prompt: it encompasses memory management, retrieval strategies, , compression, and agent orchestration.

Two identical models using the same API can produce radically different results depending on the context they receive. A scheduling assistant with access to calendar data, contact preferences, and availability tools will outperform one that only receives the user's raw text, even though the underlying is the same. The context determines the output quality.

The Context Window

A is the fixed-size buffer that holds everything an can consider during a single call. Instructions, conversation history, retrieved documents, tool descriptions, and the user's current request must all fit within this limit. LLMs have no persistent memory between calls. If information is not in the context window, the model cannot access it.

A useful analogy: the context window functions like RAM in a traditional computer. The LLM itself is the CPU, powerful but stateless. All relevant data must be loaded into the window (RAM) before each operation. Insufficient or incorrect data leads directly to incorrect output.

Token Mechanics

Context windows are measured in , subword units produced by a . Tokens do not map directly to words: common words like "the" are a single token, while longer or rarer words are split into multiple tokens (e.g., "unhappiness" → "un" + "happiness"). Source code is particularly token-dense. A single line of Python typically consumes 15–25 tokens due to symbols, indentation, and variable names.

Modern Context Window Sizes

Anatomy of Context

A typically contains seven distinct component types. All are measured in the same unit () and all draw from the same fixed budget:

Context Rot

Context rot refers to the degradation of model performance as context length increases. This is an inherent property of -based architectures: the computes pairwise relationships between all tokens. As the token count grows, each individual relationship receives proportionally less weight, diluting the model's ability to attend to critical information.

This degradation is not model-specific. It has been observed across GPT, Claude, Gemini, and Llama model families. Practical effective context is often significantly smaller than the advertised maximum window size.

The Four Context Failures

Beyond general context rot, four distinct failure modes commonly occur in production AI systems — including -driven poisoning. Each has a different root cause and requires a different mitigation strategy:

Strategy: Write

The Write strategy persists information outside the in external storage (files, databases, key-value stores) so the model can retrieve it later without consuming active context space.

Scratchpads

A stores intermediate results during a task: partial decisions, constraints discovered mid-conversation, or intermediate calculations. These notes persist in structured state rather than in the raw message history, preventing important details from being lost during compression or truncation.

Long-Term Memory

Long-term memory systems store persistent user facts across sessions: preferences, past decisions, organizational context, and learned constraints. Products like ChatGPT, Cursor, and Claude implement memory features that write facts to external storage and selectively load them into future sessions. Anthropic's recommended pattern is structured note-taking through files (e.g., NOTES.md, todo lists) that survive context resets and compaction.

Strategy: Select

The Select strategy controls which information enters the for a given turn. Rather than loading everything available, the system retrieves only what is relevant to the current task.

RAG (Retrieval-Augmented Generation)

systems embed documents into a and retrieve the most semantically similar chunks at query time. Only the top-ranked results enter the context window, keeping the knowledge base accessible without consuming the entire budget.

Dynamic Tool Selection

When an AI system has access to many , exposing all of them simultaneously increases context confusion. Dynamic tool selection analyzes the user's query and exposes only the 3–5 most relevant tool definitions. Research demonstrates this improves tool selection accuracy by approximately 3x compared to loading all tools (arXiv:2411.15399).

Just-in-Time (JIT) Context Loading

Instead of pre-loading large datasets, JIT systems maintain lightweight references (file paths, query strings, URLs) and retrieve full content only when needed. Claude Code uses this pattern to navigate large codebases. It stores file references and loads specific files incrementally using search tools, rather than ingesting the entire repository.

Strategy: Compress

The Compress strategy reduces count while preserving the essential signal within the context. This applies when relevant information exists but consumes too many tokens in its raw form.

Trimming

Trimming applies rule-based removal: drop messages older than N turns, remove tool call results after they have been processed, or truncate content beyond a token threshold. This is the simplest approach and is effective for preventing unbounded context growth in long-running sessions.

Summarization

Summarization uses the model itself (or a smaller model) to distill a long conversation or document into its key decisions, constraints, and unresolved items. The full history is then replaced with this condensed summary. Claude Code implements this as "auto-compact," triggered automatically at 95% context capacity, it summarizes the entire conversation trajectory while preserving architectural decisions and unresolved issues.

Prompt Caching

does not reduce token count, but achieves an equivalent economic effect. Repeated prompt segments (system instructions, tool definitions) are cached server-side and reused at a fraction of the cost. Anthropic charges 0.1x the base input price for cache reads, reducing a $5/M-token system prompt to $0.50/M on subsequent calls. Cache write costs are 1.25x (5-minute TTL) or 2x (1-hour TTL).

Strategy: Isolate

The Isolate strategy splits work across separate boundaries, ensuring that no single agent's window becomes overloaded with unrelated information.

Sub-Agents

are specialized AI instances, each with their own clean context window, spawned to handle specific subtasks. A complex task such as "refactor the codebase and update documentation" can be decomposed into two sub-agents: one receives only code files and style guides, the other receives only documentation and API change logs. Each operates with a focused context. Upon completion, sub-agents return compressed summaries to the parent agent, not their full execution trace.

Sandboxing

Sandboxing isolates execution environments from the context window. Code agents execute code in a separate runtime, store results as variables, and return only the relevant output (return values, error messages). The full execution trace, intermediate print statements, and stack frames remain outside the context, preserving budget for higher-value information.

Prompt Engineering vs. Context Engineering

focuses on crafting the instruction text itself: techniques like chain-of-thought reasoning, role assignment, and few-shot examples. These techniques remain important but represent only one component of a larger system.

encompasses the entire information architecture around the model: what data is loaded, when it is loaded, how it is formatted, and how it is managed across multi-turn interactions. The central question shifts from "how should I phrase this prompt?" to "what does the model need to see right now to succeed at this specific task?"

Production Patterns

Production AI systems typically combine all four strategies. The table below maps how three widely-used tools implement each strategy: