Governing what you didn’t build

A decade ago, the software your organisation ran was, broadly, software your organisation chose, configured and understood. Today a large and growing share of it is something stranger: a foundation model — trained by a handful of labs, on oceans of data no customer ever audits, exhibiting behaviours its own makers cannot fully explain. You did not build it. You may not even know exactly what is inside it. And yet you have wired it into your product, your service desk, your decisions.

This is the defining compliance problem of the moment, and most boards have not named it. We have decades of practice governing things we built and bought as finished goods. We have almost none at governing a borrowed intelligence that learns, drifts, and occasionally invents — and that reaches your customers under your brand, not the vendor’s.

The comforting story is that the model provider carries the risk. The law, and a growing line of cases, say otherwise. Accountability flows downstream, to the organisation that deployed the thing — and it does not stop to read the vendor’s disclaimer.

Be clear about what has actually happened. The capability that now drafts your emails, answers your customers and triages your tickets does not live in code your engineers wrote and can read. It lives in the weights of a model trained at vast expense by an external lab, on a corpus you cannot list, optimised by methods you cannot audit. You license the output of that process and point it at your business. In every meaningful sense, you have hired a brain whose education you did not supervise.

That borrowed brain brings borrowed risks. It can be confidently wrong — “hallucinate” — in ways no traditional software does. It can carry biases absorbed from its training data. It can drift as the vendor updates it beneath you, changing behaviour you had come to rely on. None of these are reasons not to use it; the value is real and the direction of travel is settled. They are reasons to govern it deliberately — because the one thing you cannot license out is responsibility for what it does in your name.

You can license the model. You cannot license out the responsibility for what it does in your name.

Picture the AI supply chain as a line. At one end, the model provider trains the foundation model. In the middle, an integrator may fine-tune or wrap it into a product. At your end, you — the deployer — put it in front of a customer or a decision. The model travels down that chain; the accountability travels with the brand the user sees, which is yours.

The law is catching up to this fast. Under the EU AI Act’s Article 25, the moment you put your name on a system or change what it is used for, you take on the obligations of a provider — you cannot hide behind the lab that trained it. And the courts did not wait for the statute: in Moffatt v. Air Canada (2024), a tribunal flatly rejected the airline’s argument that its chatbot was “a separate entity,” holding the company liable for the wrong answer its AI gave a grieving customer. The disclaimer did not save it. Yours will not save you.

Three comfortable beliefs collapse on contact with reality. The first: “it’s the vendor’s model, so it’s the vendor’s problem.” But the customer met your brand, not theirs — and that, as Air Canada learned, is what a tribunal looks at. The second: “the contract indemnifies us.” An indemnity may move some money after the fact; it does not move the regulatory duty, the reputational hit, or the harm to the person on the other end. The third: “the disclaimer covers us.” Small print has never licensed an organisation to give people wrong, biased or harmful answers.

Underneath all three is a category error: treating a foundation model as a product you bought rather than a supplier you are responsible for. You would not let an unvetted subcontractor speak to your customers in your name with no oversight and no record. A deployed model is exactly that subcontractor — faster, cheaper, and far harder to interrogate after the fact.

The good news, again, is that this is not uncharted territory. We know how to govern third parties we depend on but do not control; we simply have to apply that discipline to AI. Due diligence before you deploy: what is this model, what is known about its training and limits, what does the provider document? Evaluation against your own use case, not the vendor’s benchmark — test it on your hardest, most sensitive cases before a customer does. Monitoring after launch, because the model drifts and your usage changes. And a human fallback for the high-stakes path, so a wrong answer is caught before it lands.

This is also where the new rules help rather than hinder. The EU AI Act obliges model providers to maintain technical documentation for the deployers downstream — so demand it, and write your contracts to get the model cards, evaluations and change-notifications you need to discharge your own duty. Governance you cannot evidence is governance you do not have.

You don’t need to have trained the model to be in control of it. You need these five in place before it touches a customer.

Do this and the borrowed brain becomes a managed supplier rather than an unexamined risk — powerful, useful, and answerable to a human who owns the outcome.

You didn’t build it, and you can’t fully see inside it. You still answer for it. Govern it like you mean it.