AI Writes the Code, Engineers Build the Context: Why Context Engineering Is the Most Valuable Skill in the AI Era
Click to enlarge
Ask an AI coding agent to fix a bug today and it will happily generate a patch in seconds. Whether that patch is actually correct, safe, and consistent with the rest of the codebase depends on almost nothing the model itself controls. It depends on what the engineer put in front of it: which files were loaded, which past decisions were summarised, which tools were exposed, and which noise was kept out. That discipline now has a name, context engineering, and in 2026 it has quietly become the single most valuable skill separating teams that ship reliable AI systems from teams that ship expensive demos.
From Prompt Engineering to Context Engineering
For the first couple of years of the LLM boom, the dominant skill was prompt engineering, finding the right words and structure for a single instruction to get a good one-shot response. That skill still matters, but it was built for a world of isolated chat turns, not autonomous agents. Anthropic's Applied AI team put this shift into words clearly in its own engineering write-up: prompt engineering is about writing and organising instructions, while context engineering is the broader set of strategies for curating and maintaining the optimal set of tokens, an LLM sees during inference, everything beyond just the prompt, including tool outputs, memory, retrieved documents, and message history.
The distinction is not academic. A single well-crafted prompt can carry a one-shot classification task. It cannot carry a 15-step coding agent that reads a file, runs a test, gets an error, searches the web, and edits three more files before it is done. At every one of those steps, something new enters the model's working memory, and if nobody is deliberately managing what stays and what gets dropped, the system degrades in ways that have nothing to do with the underlying model's raw intelligence.
Why This Became Urgent: Context Rot Is Real and Measured
It is tempting to assume that bigger context windows solve this problem outright. They don't. Chroma's widely cited 2025 technical report, authored by Kelly Hong, Anton Troynikov and Jeff Huber, ran 18 frontier models, including Claude 4, GPT-4.1, Gemini 2.5 and Qwen3, through 194,480 calls and found context rot, across every model tested, even on tasks the same models handled perfectly when given a short input. The researchers named this failure mode context rot. It is not the same as hitting a hard token limit; a model with a 200,000-token window can start losing reliability well before that window is even close to full.
One especially useful finding from that report involved a direct comparison: focused prompts averaging around 300 tokens consistently outperformed full, unfiltered prompts averaging around 113,000 tokens on the same underlying task. Adding more information did not add more capability, it diluted the model's attention across content it didn't need. This is the empirical foundation underneath the entire context engineering discipline, and it is why the field has increasingly moved away from the old assumption that a bigger context window is automatically a better one.
The Four Things Anthropic Says to Get Right
Anthropic's engineering guide breaks effective context engineering down into a handful of concrete levers rather than vague advice, and this framework is a genuinely useful mental checklist for anyone building agents right now.
Table
| Lever | What It Means in Practice |
|---|---|
| System prompts | Clear and specific, structured with headers or XML, but never padded with rigid, over-engineered instructions that break on edge cases |
| Tool design | A small set of clear, non-overlapping tools with well-scoped purpose; overlapping or bloated toolsets confuse the model about which tool to call |
| Just-in-time retrieval | Pull in files, memory, or data only when the agent actually needs it for the next step, rather than pre-loading everything up front |
| Compaction and offloading | Summarise long histories, move raw tool output to external storage, and keep only compact, high-signal references in the live context window |
2 columns · 5 rows
None of these are exotic ideas individually. What makes context engineering a distinct discipline is treating all four as one continuous design problem across a multi-step task, rather than optimising a single prompt and hoping the rest takes care of itself.
Sub-Agents: Isolating Context Instead of Sharing Everything
One pattern shows up repeatedly across production systems built by different teams: instead of one large agent carrying an ever-growing history, an orchestrator agent breaks work into subtasks and spawns focused sub-agents, each with its own clean context. A sub-agent reads twenty files, runs a search, or executes a test suite, then returns only a short, relevant summary to the orchestrator, which never has to see the raw output itself. Chroma researcher Kelly Hong walked through this exact pattern in a public talk, and Anthropic describes a comparable architecture in its own context engineering, where an orchestrator spawns parallel subagents to explore different facets of a query and returns only the essential findings upward.
The company behind the AI agent Manus arrived at nearly identical conclusions independently, describing how a typical agentic task in their system can require around 50 tool calls, and how naively appending every tool result to a growing conversation history causes exactly the kind of context bloat that triggers rot. Their engineering notes on this, published as lessons from building Manus, are one of the clearer public accounts of what breaks first when context management is treated as an afterthought rather than a core design decision.
The Multi-Agent Trap: Why Cognition Pushed Back
Not everyone agrees that splitting work across many agents is automatically the right answer, and this disagreement is itself instructive. Cognition, the team behind the AI software engineer Devin, published a widely discussed post arguing that naive multi-agent setups often make context problems worse, not better, because multi-agent context sharing problems can outweigh the benefits. Their conclusion was not that sub-agents are wrong, but that context sharing between agents has to be engineered as carefully as context inside a single agent, otherwise you have simply moved the rot problem from one place to another.
Model Context Protocol: The Plumbing Layer Underneath
Good context engineering also depends on how an agent actually reaches external data and tools in the first place, and this is where the Model Context Protocol, or MCP, fits in. Anthropic open-sourced MCP in November 2024 as a standard way to connect AI applications to external systems like file systems, databases and business tools, replacing what used to be a custom integration for every single data source. The official MCP documentation describes it as a universal, open protocol for two-way connections between AI applications and the systems where their data actually lives, and it has since been adopted well beyond Anthropic's own products across the wider agent ecosystem.
MCP does not by itself solve context engineering. A well-designed MCP server can still return a bloated, unfiltered response that floods an agent's context window exactly the way a poorly designed custom integration would. What MCP changes is the plumbing: it gives teams a standard shape for tool and resource access, which makes it far easier to apply the same context discipline, minimal, well-scoped, clearly described tools, consistently across every integration an agent touches.
Karpathy's Framing: Software 3.0 and the Context Window as RAM
Andrej Karpathy, who helped popularise the term context engineering after endorsing it in mid-2025, has since offered one of the clearest mental models for why this matters. Speaking at Sequoia Capital's AI Ascent 2026, he described a three-stage evolution: Software 1.0 was explicit hand-written code, Software 2.0 was learned neural network weights, and Software 3.0 is prompting itself, where the program is the text sitting in the model's context window and the LLM is the interpreter that runs it. In this framing, the context window plays the role RAM plays in a traditional computer, and the engineer's real job becomes operating system management, deciding what gets loaded into that limited working memory at each step, not writing more code by hand.
This is also why Karpathy later drew a distinction between casual vibe coding, where a developer accepts AI-generated output with minimal scrutiny, and what he called agentic engineering, a more disciplined practice built on spec writing, diff review and evaluation loops around autonomous agents. Context engineering sits underneath both, but it becomes non-negotiable the moment you move from a single chat exchange to an agent that is expected to work independently across many steps.
Practical Techniques Worth Learning Right Now
- Compaction: periodically summarise long conversation or tool-call history into a compact form instead of letting it grow unbounded
- Offloading: store large raw outputs like full file contents or search results outside the model's live context, and pass back only a reference or short summary
- Just-in-time retrieval: fetch a document, file or memory record only at the step where it is actually needed, rather than front-loading everything at the start of a task
- Sub-agent isolation: give a sub-agent its own clean, narrowly scoped context for a specific piece of work, then return only the distilled result to the orchestrating agent
- Tool minimalism: expose a small number of clearly scoped tools rather than a sprawling toolkit, since an oversized tool list itself consumes context and increases the odds of the wrong tool being called
LangChain's own engineering team has documented several of these techniques concretely, including a hands-on repository that context engineering patterns, built directly on top of the failure modes identified in the Chroma research. It is one of the more practical starting points if you want to see these ideas implemented in working code rather than described in the abstract.
Common Mistakes Teams Are Still Making
- Treating a larger context window as a substitute for actual context curation, when Chroma's research shows degradation happens well before the window fills up
- Appending every tool call and raw output to a single growing history instead of summarising or offloading it
- Building an oversized, overlapping toolset for an agent instead of a small number of clearly scoped tools
- Splitting work across multiple agents without a deliberate plan for how context and decisions are shared between them, the exact trap Cognition's team has written about publicly
- Writing one large, rigid system prompt meant to cover every possible situation, instead of clear, adaptable instructions supported by good tool and retrieval design
Why This Matters for Engineers, Not Just AI Researchers
This shift matters well beyond people building AI products full time, because context engineering is increasingly what separates a developer who gets consistent, reliable output from an AI coding agent from one who gets impressive demos that fall apart on real codebases. Anthropic's own developer documentation on prompt engineering strategies is still a useful starting point for the fundamentals, but the skill that increasingly determines outcomes on multi-step, real-world tasks sits one layer above that: deciding what information an agent should and shouldn't see at each point in its work. As LangChain's Lance Martin put it in his own early write-up on the topic, context engineering became a distinct discipline specifically because agents pull in context from many tool calls, not just a single user prompt, and managing flowing agent context in building agents that actually work in production.
For students still choosing between core CSE and an AI/ML-focused engineering branch, this same context-curation instinct is increasingly part of the job description either way, and it's worth weighing alongside curriculum and placement data using our BTech CSE vs CSE AI&ML comparison.
The broader research community has reached a similar conclusion from another direction entirely. Independent writer Drew Breunig's widely referenced essay on how how agent context fails catalogues specific, recurring failure patterns, like context poisoning and context distraction, that engineers now need to recognise by name in the same way they once learned to recognise race conditions or memory leaks. That is, in many ways, the clearest signal that context engineering has moved from a buzzword to an actual engineering discipline: it now has its own named failure modes, its own diagnostic vocabulary, and its own emerging best practices, rather than just being a vague call to write better prompts.
Where This Goes Next
None of this replaces the value of a good engineer's judgement, it relocates it. The rigour that used to go into writing clean functions and clear interfaces is now going into deciding what an agent gets to see, when, and in what form. Teams that treat this as a genuine engineering discipline, with its own patterns, failure modes and tooling, are the ones shipping agents that hold up on real, messy, multi-step work. Teams that treat it as an afterthought are the ones re-explaining, in increasingly frustrated lan
guage, why their AI system worked perfectly in the demo and fell apart in production.
If you're mapping out an engineering path toward this kind of work, our B.Tech & M.Tech College Predictor is a useful starting point for shortlisting colleges with strong AI/ML and CS programmes based on your JEE rank.
Share this article
Copy
Frequently Asked Questions
What is context engineering in AI development?
Context engineering is the discipline of curating and managing everything an AI model sees during inference, including system instructions, tool outputs, retrieved data, and conversation history, rather than focusing only on the wording of a single prompt. Anthropic describes it as the natural progression of prompt engineering for multi-step, agentic systems.
How is context engineering different from prompt engineering?
Prompt engineering focuses on crafting the wording and structure of a single instruction for a one-shot response. Context engineering covers the full information environment an agent operates in across many steps, including tools, memory, retrieved documents, and history, not just the initial prompt.
What is context rot?
Context rot is a measurable phenomenon, documented in Chroma's 2025 technical report, where an LLM's output quality degrades as input length increases, even when the model is nowhere close to its maximum context window. All 18 models tested in the study showed this pattern to some degree.
Does a larger context window solve context rot?
No. Chroma's research found that performance degradation begins well before the context window is full, and that focused, shorter prompts often outperform much longer, unfiltered ones on the same task. A bigger window increases capacity, not automatically reliability.
What is the Model Context Protocol and how does it relate to context engineering?
The Model Context Protocol, or MCP, is an open standard Anthropic released in November 2024 for connecting AI applications to external data sources and tools through a consistent interface. It doesn't automatically solve context engineering, but it makes it easier to apply consistent, well-scoped tool design across integrations.
What is agentic engineering and how does it relate to context engineering?
Agentic engineering is a term associated with Andrej Karpathy describing a disciplined practice of orchestrating AI agents through spec design, diff review and evaluation loops, as opposed to casual vibe coding. Context engineering is a foundational layer beneath it, since reliable agent orchestration depends on carefully managing what each agent sees at every step.
Why are sub-agents used in context engineering?
Sub-agents let a task be broken into smaller pieces, each handled with its own clean, narrowly scoped context. A sub-agent processes raw data or tool output and returns only a compact summary to the main orchestrating agent, which prevents the main agent's context from becoming bloated with unnecessary detail.
Confused About College Admissions?
Get expert advice on college selection, admission chances, and career path in a personalized counselling session.
Book a Counselling Slot
Select Date
Pick a Slot