What Is a Context Layer?

The firm I work for builds what we call an executable context layer. I spend a fair amount of time explaining what that means. The name does not help much on its own: depending on who you ask, “context” can mean a chat history, a collection of documents, or almost everything an organisation knows.

I find it easier to start with a question an agent might be asked. Consider a desk head at a bank:

Can we do another $40M of 1-year EUR/USD forwards with Halden Capital today?

Halden and the figures here are fictional. The answer depends on what the bank means by exposure, its current position, the effect of the proposed trade, and the policy governing any excess.

In this example, Halden has $226M of potential future exposure against a $250M limit. The approved calculation says the proposed trade would add $30M, taking it $6M over. Halden’s rating makes it eligible to request a temporary excess under the bank’s current policy, with approval from the chief risk officer’s delegate.

So the answer is: it would exceed the limit, but there is an approval route. The agent can offer to prepare that request. It cannot treat the existence of an exception policy as permission to execute the trade.

Getting there requires several pieces of information held in different places. Each has a plausible wrong version somewhere in the bank. A context layer helps the agent find the right definitions and the tools to get the figures, subject to the access rules of the person it is working for. The rest of this post follows that question through the system.

The context window

A model’s training gives it general knowledge, including knowledge of finance. It does not give it today’s Halden exposure or tell it which internal policy the bank approved last week. The application must make those facts available when the model works on the request.

The context window is the working material available to a model call. It can include instructions, tool descriptions, selected memory, the user’s request, and previous messages and tool results. An agent application may retain extensive state between calls, but it still has to decide which of that state belongs in the next call.

A simple agent loop asks the model to choose a next step, executes any permitted tool call, adds the result to the conversation, and invokes the model again. The application can also summarise or remove earlier material as the task progresses.

The diagram below follows a simple run with the transcript retained. Its first tool call asks the context layer for a map: which definition of exposure applies, where to find the live figure, and which tools can answer the question. That ordering is a design choice in the agent’s instructions.

The animations throughout this post are scripted illustrations of the same fictional case. Their answers and token counts explain the mechanisms; they are not recordings of model runs. The first figure counts the context supplied across turns, including cached input.

How much workflow belongs in the agent?

There is a design choice here that I think of as thick versus thin agents. In a thick agent, the application defines much of the workflow: look up the counterparty, fetch these fields, run this calculation, then apply this decision path. In a thinner agent, the application supplies a goal, constraints, and a small starting set of tools. The agent discovers more of what it needs as it works.

I keep getting asked where we encode the workflow in a thin agent, or what orchestrates it. The way I describe it is that the workflow is an emergent property of the context the agent is presented with. Given the goal, instructions, available capabilities, and results so far, the model chooses what to do next. Each result changes the context for the following decision.

For Halden, the live exposure and proposed trade’s add-on show it would go over the limit. The current policy and rating reveal an approval route, and the available tools give the agent a way to prepare a request. That sequence can emerge during execution without a Halden-specific workflow being encoded in the harness.

We still have to encode the business rules: what exposure means, which policy applies, who may approve an excess, and which operations are available. The agent works out the sequence from that context. The harness enforces the boundaries as it goes.

Building a thick agent takes time because the team has to anticipate and implement its paths through the work. For the Halden question, that includes the normal limit check, the temporary-excess route, missing ratings, stale figures, and whichever exceptions the desk encounters next. A policy change can require changes to the workflow and its tests. A situation nobody encoded can leave the agent stuck, or following a path that no longer fits.

That work accumulates as the number of agents grows. Going from ten to a thousand specialised agents can mean hundreds more workflows to build, test, and maintain, even where they share libraries. I mean scale here in engineering effort: how quickly we can support another kind of work and keep it correct as the organisation changes.

With thinner agents, the same harness can serve many tasks. We can support a new task by updating the context layer, for example by describing a calculation and providing a governed tool to run it. The harness stays the same. We still have to connect the underlying systems and evaluate how the agent handles the task, but agents can discover the same shared policy instead of each carrying a copy. That is a large part of the appeal for me: fewer versions of the same business rule embedded in separate agent implementations.

These are ends of a spectrum. A fixed workflow can still retrieve context dynamically, and a flexible agent still needs enforced boundaries. Giving the agent more freedom to choose its next step makes it more dependent on accurate tool descriptions and results.

Tool results can steer the agent

A tool result becomes part of the model’s input. It may arrive as a distinct tool message or a structured object, but the model still interprets its contents when deciding what to do next.

A note on Halden’s credit file might point to the temporary-excess policy. That could help the agent discover an option it would otherwise miss. An old note could point to a superseded policy, and a malicious note could try to persuade it to skip approval altogether.

Those are different kinds of text with different authority. Retrieved content should be treated as evidence to evaluate, not as permission to override the agent’s instructions. The application also needs to enforce access and approval checks when a tool is called.

Try the variants below. The point is how a small change in a returned record can alter the next step. A policy note should lead to checking the authoritative policy.

Millions of items, a handful needed

A bank may hold millions of records an agent could potentially use: communications, trades, positions, policies, client files, risk calculations, and approvals. Connectors and protocols such as MCP provide ways to discover and call tools over those systems. We still have to determine which sources apply to this request and what the caller is allowed to see.

For the Halden question, the relevant material can be reduced to seven items: the exposure definition, live exposure, limit, proposed trade’s exposure add-on, credit rating, current excess policy, and the delegate authorised to approve the request.

The figure shows a conceptual narrowing from millions of items to those seven. In an implementation, access restrictions must apply during discovery as well as retrieval: even a search result or description can reveal something the user should not know.

One of the seven items has to be computed. The exposure added by a proposed trade depends on its terms and the applicable calculation. The agent needs to discover how to obtain that result, including the required inputs and the meaning of the output.

Cost, latency, accuracy

A large tool result retained in the conversation contributes to the input on every subsequent call, until it is summarised or dropped. Caching can reduce the cost, but selecting what the agent needs still matters. Retrieval also takes time, so we need to measure the complete task, including calls to the context layer.

A larger window does not guarantee better use of the information. Lost in the Middle found that performance varied with the position of relevant information in the input. Chroma’s Context Rot study found that increasing input length could degrade performance even with task difficulty held constant. We need to test how well the agent uses the context we supply.

For Halden, removing old policies could prevent conflicting approval routes. Removing the current exception policy would leave the answer incomplete.

The race below runs the same five turns twice: once with everything that matched the search, once with only what the task needs.

Which exposure answers this question?

The limit in this example is defined against live potential future exposure. A current value net of collateral of $142M answers a different question. Yesterday’s utilisation of $214M is out of date. The applicable figure is $226M.

Using the overnight figure gives $214M plus $30M: $244M, apparently within the limit. The calculation misses this morning’s activity. Supplying the live figure fixes that part, but without the current policy and rating the agent may still miss the approval route.

The interactive example lets you change those inputs. It shows why I would inspect the assembled context early in an investigation: which definition was supplied, when was the figure measured, and which policy version was treated as current?

What a context layer does

By a context layer, I mean the system that helps an agent work out what a request means and find what it needs to act on it. For Halden, that includes resolving the exposure definition and getting access to the bank’s live risk system under the caller’s permissions.

In the approach we are building, that involves three connected parts.

An ontology describes the organisation’s concepts and their relationships to sources, policies, and tools. For this task, it connects “credit limit” to the applicable exposure measure and identifies where that measure can be obtained. It also records who owns the definition and which version applies.

A governed tool interface lets the agent discover and invoke capabilities. It checks whether this user, acting through this agent, may make the particular call with the supplied arguments. Reading exposure, calculating an add-on, requesting an exception, and approving that exception are separate capabilities with separate permissions. Calls and their outcomes are recorded.

Data gateways provide controlled access to the underlying systems. Where the source supports it, queries can run in place with row and column restrictions. Results need enough provenance to identify the source, time, and relevant version or calculation.

The agent can begin by stating its goal without already knowing every internal system or tool name. The context layer returns a small map of meanings, sources, and permitted capabilities. The agent then fetches or computes what it needs and can ask for more if the first result leaves a gap.

This is what I mean by executable: the map leads to operations the agent can actually perform. For Halden, it leads to live exposure and a calculated add-on, then to an approval request if the user chooses to proceed.

The context map is itself an access-controlled result. A restricted resource can leak through its name or description before the agent ever opens it. The caller’s permissions apply from the first search through to execution.

Keeping it correct

Building the initial map is only part of the job. Someone must own the definition of exposure, connect it to the appropriate source, and approve the policy relationship. When the bank changes its excess policy, the new version needs an effective date and an owner. The old version may still be needed to explain a historical decision, even though it no longer governs today’s request.

That maintenance has to reach cached context and tool descriptions as well as source documents. Otherwise the bank can update its policy correctly while an agent continues to work from yesterday’s map. If the layer cannot establish which policy applies or whether an exposure figure is fresh enough, the agent needs that uncertainty explicitly so it can stop or ask for clarification.

We also need to reconstruct what happened. For the Halden answer, I would want to see the definition selected, the exposure timestamp, the calculation used for the proposed trade, the policy version, and the access decisions. If the answer is wrong, I need to be able to follow it back to the source or decision that caused it.

Too little context leaves the agent without the basis for a decision. Too much can add delay and competing interpretations. The final illustration makes that tradeoff visible, although the right amount depends on the task: a credit-limit check and a review of a disputed policy need different material.

Back at the desk, the trade would take Halden $6M over its limit. The agent has found the current exception policy and can prepare a request for the CRO’s delegate. The desk head can decide whether to proceed with that request.

For a broader view of the category, see Sanjeev Mohan’s Context Is Everything. So What Is It Made Of?.