Context Engineering: The Skill That Replaced Prompt Engineering

Sometime in early 2026, a quiet shift happened in how serious builders talk about AI. The phrase "prompt engineering" started getting an eye-roll in senior engineering rooms — not because writing good prompts stopped mattering, but because everyone realized it was only ever one piece of a much bigger puzzle. The piece people are now scrambling to understand is called context engineering. And almost nobody has a clean explanation of what it actually means.

The prompt engineering era, honestly assessed

Prompt engineering had a real moment. Between 2023 and early 2025, learning to write clearer, more structured prompts made a genuine difference. Chain-of-thought prompting, role instructions, few-shot examples — these weren't tricks, they were real techniques that improved outputs noticeably.

But something uncomfortable happened as people moved from chatbots to agents — systems that run tasks, use tools, remember things, and operate across multiple sessions. The prompt kept getting better. The agent kept failing. Not during demos. In production. After real users started using it. When they referenced something from last week. When the knowledge base grew past 10,000 documents. When two users had contradictory preferences and the system couldn't tell them apart.

That's the insight that context engineering is built on. An LLM doesn't think independently. Every response it produces is a function of exactly what's inside its context window at that moment — the system prompt, whatever text you've included, conversation history, documents you've retrieved, tool call outputs, user preferences you've injected. Feed it the wrong things, leave out the right things, or bury the critical information in position 75,000 — and you'll get a worse response, every time, regardless of how good the underlying model is.

The model hasn't changed. Your context did. That's the whole problem.

What "context" actually contains

When people hear "context window," they usually picture a single text blob with the conversation history in it. That framing is about three years out of date. In a real agent system running in 2026, the context window at any given inference step contains several distinct layers — each of which can be managed well or poorly

1. 1

   System instructions

   The standing rules, persona, and constraints. Most teams treat this as static. The better teams update it dynamically based on what the agent is currently doing.

2. 2

   Working memory

   The current conversation turn, including tool calls and their outputs from earlier in this session. This fills up fast. Managing it carefully — summarising or trimming stale turns — is most of the work.

3. 3

   Retrieved knowledge

   Documents, code snippets, or data pulled from a vector store or search index based on the current query. The naive version pulls the top-K chunks by embedding similarity. The careful version also factors in recency, relevance type, and whether the chunk is likely to land in the "forgotten middle" of a long context.

4. 4

   User and session state

   Preferences, past decisions, role, and any facts established in previous sessions. This is what most agents don't have — and it's why they feel like strangers every time.

5. 5

   Tool outputs

   Structured results from APIs, calculators, search, databases. These need to be formatted correctly and placed at the right position in the window — not dumped raw into the middle where the model will likely ignore them.

Why agents break in production and almost nobody diagnoses it correctly

Here's the pattern that plays out constantly. A team builds an agent. It works beautifully in development — clean outputs, logical behaviour, impressive demos. They ship it. Real users interact with it for a few weeks. Then the failure reports start coming in.

The agent starts hallucinating facts it was never given. It forgets things the user told it three sessions ago. It contradicts advice it gave the same user last Tuesday. When the knowledge base grows, it starts inventing policies that don't exist in any document. The first instinct of most teams is to blame the model or rewrite the prompt. Both are almost always wrong.

There's a specific phenomenon worth naming: the "lost in the middle" problem. Research published in 2023 and confirmed repeatedly since shows that language models attend strongly to information at the beginning and end of the context window, and measurably less to information buried in the middle. This means a 128k context window does not give you 128k of equally useful space. The architecture of attention is uneven. If you're shoving critical instructions or key retrieved documents into position 60,000, you're doing context engineering badly even if you technically have room

A 128k context window does not give you 128k of equally useful space.

What context engineers actually do

The job title doesn't exist yet, not in any standardised way. But the practice does, and a recognisable set of skills is starting to crystallise around it. The clearest description I've found: context engineering is the practice of deciding what information goes into the context window, in what format, in what order, and how much of it — for each specific inference step

That involves a surprising number of concrete decisions. Which documents to retrieve — and just as importantly, which not to. How to summarise a long conversation history into something compact that preserves the semantically important turns. Where to inject the retrieved content in the window. Whether to include user preferences inline or as a separate block. How to format tool call outputs so the model can parse them correctly and they're not ignored. How to compress old session data into a memory summary rather than re-injecting the full raw transcript.

Prompt engineering vs. context engineering — they're not opposites

One thing worth being direct about: context engineering doesn't make prompt engineering irrelevant. Writing clear, specific, well-structured prompts still produces better outputs than sloppy ones. The relationship is more like this — prompt engineering is what you do inside the context window. Context engineering is how you build and manage the context window itself. Both matter. The order of importance just shifted.

The teams that are most frustrated right now are the ones who became genuinely good at prompting and assumed that would keep compounding. It did, up to a point. That point is roughly "your agent starts doing things across multiple sessions with real users." Past that point, context engineering is the lever, and prompting is just one component of it.

The tools most builders are actually using right now

The memory problem nobody talks about enough

There's a specific failure mode that context engineering has to solve that pure context window management can't fix, and it's worth naming clearly: the agent doesn't remember. Not because the context is too short — because nothing is storing the right things between sessions. Emphasis: false

LLMs are stateless by design. Every inference call starts completely fresh. This is not a flaw, it's what allows them to scale to millions of concurrent users. But it means that if you want your agent to remember that a user prefers concise answers, or that they're debugging a specific production issue, or that they told you last week they'd already tried the obvious solution — you have to build the infrastructure that stores and retrieves that information. The model doesn't do it for you. It can't.

The $6.27 billion agent memory market in 2026 is essentially the ecosystem that has grown up to solve this problem. Vector databases, memory extraction pipelines, session state managers, profile stores — these are all parts of the infrastructure layer that context engineering runs on. Knowing they exist is useful. Knowing how to compose them correctly is the actual skill.

What to actually read and build if you want to get good at this

A few honest recommendations, not a comprehensive list.

• Read Andrej Karpathy's original thinking on LLM context — his explanations of attention and what the model actually "sees" are the best foundation you'll find.
• Build one agent with explicit context logging. Every inference, write the full payload to a file. Read 20 of them. You'll spot patterns in five minutes that would take weeks to find otherwise.
• Read the "lost in the middle" paper. It's short, it's practical, and it will change how you structure every context window you build going forward.
• Try Mem0 on a simple agent that spans two sessions. The difference in behaviour when persistent memory is working correctly is immediately obvious.
• Read the LangGraph documentation on per-node memory. Even if you're not using LangGraph, the mental model is worth having.