AI’s biggest enterprise risk is not model training. It is context transfer

A company connects an AI vendor to its codebase, support history, product analytics and internal documents. Then a forward-deployed engineer arrives to help turn the prototype into production.
Everyone calls this an AI implementation.
Look at it from another angle and it is something more consequential: one organisation is exposing the machinery of how it thinks to another organisation whose business is learning how work can be automated.
That does not make the vendor malicious. It does mean that “Does the provider train on our prompts?” is no longer a sufficient risk question.
The deeper question is: what operational context crosses the boundary, who can learn from it, and what becomes difficult to recover once the platform is embedded?

The accepted story: the model is just another tool

Enterprise technology has always required vendors to see parts of a customer’s operation. Consultants inspect processes. Cloud providers host data. Support engineers receive logs. SaaS products store workflows.
So it is reasonable to treat an AI provider as another supplier governed by contracts, permissions and security controls.
There is real evidence for that view. OpenAI says business and API inputs and outputs are not used for model training by default. Anthropic makes the same default commitment for Claude Enterprise. Microsoft describes its current Frontier Tuning work as occurring inside the customer’s compliance boundary.
Those commitments matter. Claims that every enterprise prompt is secretly poured into a foundation model ignore both public policy and contractual reality.
But “not used for training” answers only one narrow question. It does not describe the entire learning surface created when a model company helps build software inside a customer’s business.

The model vendor is moving up the stack

The AI industry is not stopping at inference APIs.
OpenAI’s description of its Forward Deployed Engineering team says engineers work with strategic customers from discovery and system design through production rollout. The role embeds with customer teams, contributes code, codifies reusable patterns and returns eval-driven feedback that can influence product and model roadmaps.
AWS now offers forward-deployment engineering through its Generative AI Innovation Center, including embedded teams that build custom systems and perform domain adaptation. Microsoft’s Frontier Tuning programme similarly pairs forward-deployed engineers with a company’s workflows, tools, evaluation criteria and business conventions.
This is valuable precisely because generic models are not enough. Production AI needs the part no benchmark contains:

• which exceptions matter;
• which internal systems are authoritative;
• how experienced employees resolve ambiguity;
• where a workflow can tolerate error;
• which outcomes the organisation rewards;
• what “good” looks like in that particular business.

That knowledge is often more defensible than the source code. Competitors can reproduce a feature. They cannot easily reproduce years of accumulated judgment about customers, edge cases and trade-offs.
When an AI implementation captures that judgment, the integration is touching intellectual property even if no patent, repository or training dataset changes ownership.

The redistribution is of context, not necessarily ownership

Calling this the largest redistribution of IP in history is provocative, but the phrase points at something real if we define the transfer correctly.
The transfer is not necessarily theft. It is not necessarily legal ownership changing hands. It is the movement of operational context from private human networks into systems operated, supported or influenced by a small number of AI platforms.
Think of enterprise knowledge as a ladder:

1. Data: documents, tickets, code and transactions.
2. Workflow: the sequence in which people and systems act.
3. Exceptions: the cases where the documented workflow fails.
4. Judgment: how experts choose between imperfect options.
5. Evaluation: the evidence the company uses to decide whether an outcome is good.
6. Strategy: which capabilities matter enough to build, buy or abandon.

Traditional software often captured the first two layers. Agents and forward-deployed implementations increasingly need all six.
This produces what I would call the context-transfer gradient:

The closer a vendor gets to your exceptions, evaluations and strategic decisions, the less useful “we do not train on your data” becomes as a complete description of the risk.

Again, that statement does not imply misconduct. A vendor can honour every privacy commitment and still gain legitimate product insight from deployment patterns, feature requests, failure modes and aggregated demand. The customer can also become dependent on a proprietary implementation that encodes its own process in a form it cannot easily move elsewhere.
The main risk is therefore not only data leakage. It is asymmetric learning and dependency.
The customer teaches the system how its business works. The platform learns what enterprise software needs next. But the reusable platform improvements belong to the platform, while the customer may be left with an implementation whose model, orchestration and support layer it does not control.

Why model companies are also building software

This helps explain why companies that build models are also moving into applications, coding tools, agents and implementation services.
Software is distribution, but it is also observation.
An API request reveals whether a model can answer a prompt. A production application reveals why the answer matters, what happens next, which failure stops adoption, and how a human corrects it. An embedded engineer sees the causal chain around the model.
The value loop looks like this:

model capability → customer workflow → production failure → evaluation signal → platform improvement → deeper adoption.

Every step can be legitimate and contractually controlled. The strategic advantage still compounds. A company operating both the model and the application layer can see where intelligence becomes useful, not merely where it scores well.
That is why the software layer is not a distraction from the model business. It is where abstract capability encounters economic reality.

Open weights complicate the conspiracy story

The original concern contains an assumption that needs correcting: hyperscalers do release models with accessible weights.
Google provides Gemma as an open-weight family that can be downloaded, tuned and run on customer-controlled hardware. Microsoft describes Phi as an open model family that supports local and edge deployment.
So the market is not cleanly divided into closed hyperscalers and independent open-source defenders. Large platforms can publish open weights while still benefiting from cloud hosting, tooling, distribution and enterprise integration.
Open weights also do not automatically solve the context problem. A company can run a model locally but send telemetry to a hosted observability service, rely on a proprietary agent harness, expose repositories to a coding assistant, or hire an external team to encode its workflows.
The relevant unit of control is the entire system, not the model file.
Open weights reduce one dependency: control of inference. They do not by themselves determine who can access prompts, tools, evaluations, traces, support cases or the knowledge embedded in the orchestration layer.

Why not let specialist vendors do this work?

The strongest response is that this is simply how specialisation works.
Most companies should not build foundation models, agent runtimes, security layers and deployment infrastructure themselves. External experts see repeated patterns and can deliver better systems faster. Customers benefit when lessons from many implementations improve the platform. Strict isolation would make every organisation repeat the same mistakes.
That is true.
The mistake is not using a platform or a forward-deployed team. The mistake is treating the engagement as ordinary software procurement when it operates more like a joint reconstruction of the company’s decision-making system.
The answer is governance, not paranoia.

A context-boundary review

Before giving an AI platform broad access, a technical leader should be able to answer six questions.

1. What knowledge is entering the system?

Inventory more than files and database fields. Include prompts, tool calls, traces, human corrections, evaluation datasets, exception handling and support conversations.

2. Who can observe each layer?

Separate model processing, automated logging, vendor support, embedded engineers, subprocessors and internal administrators. “The vendor” is not one access path.

3. What is each layer allowed to influence?

Distinguish model training, product analytics, abuse monitoring, support, roadmap feedback and aggregated service improvement. Do not compress these into a single yes-or-no question about training.

4. Which parts can remain under customer control?

Consider local inference, private networking, customer-managed keys, isolated evaluation environments, redacted traces and self-hosted orchestration. Use architecture to reduce trust requirements rather than asking contracts to carry all of them.

5. Can the encoded knowledge leave with you?

Export prompts, tools, evaluation suites, traces, memory, workflow definitions and fine-tuning artifacts in usable formats. An exit plan that exports only documents does not recover the intelligence layer built around them.

6. What learning exchange is acceptable?

A platform should be allowed to improve, and a customer should know which signals support that improvement. Make the exchange explicit. Price it, constrain it or opt into it deliberately.
This review will not eliminate dependency. It makes dependency visible enough to choose.

The real Trojan horse is convenience without a boundary

There is no evidence that forward-deployed engineers are a coordinated Trojan horse for stealing customer IP. That framing is too simple and too accusatory for the facts we have.
The more credible danger is quieter.
AI makes it extraordinarily convenient to translate a company’s private operating knowledge into machine-readable context. The same process that creates value can also centralise dependency, expand access and move critical judgment into a platform the company does not own.
That is not a reason to reject AI vendors. It is a reason to recognise what is being exchanged.
The next generation of enterprise architecture will not be defined only by where data is stored or where models run. It will be defined by who controls the context that teaches agents how the business works.
The companies that understand this will still use hyperscalers, frontier models and forward-deployed engineers. They will simply stop pretending they are purchasing intelligence by the token.
They will treat context as capital—and design the boundary before the transfer begins.

← All writing