An AI center of excellence is a committee, not a control plane
· 6 min read · Blog
An AI center of excellence sets policy and reviews proposals. It cannot stop an agent writing to a production system. Where the boundary actually sits.
If you're responsible for AI at a large company, an AI center of excellence probably looks like the answer to governance. Put one team in charge of the budget line, the model gateway, the API keys and the architecture review board, and you have real controls in one place.
A CoE is a good start. On its own, it isn't enough.
Those controls decide which AI gets bought, which models can be called and who gets a key. They don't see what an agent does next: the tools it calls, the systems it writes to, the actions it takes. A CoE can hold every lever it's supposed to and still miss the agents that matter. It was set up as if AI were a procurement problem, and it's now being asked to control what AI does in production.
That matters because of who answers for it. The KPMG AI Quarterly Pulse Survey for Q2 2026 asked 204 US C-suite and business leaders at companies with $1 billion or more in revenue where accountability sits for AI-informed or executed business decisions. 34% said a named C-suite executive, 32% the CEO or executive committee and 14% a business unit leader. Only 10% said a centralized AI governance or risk committee.
For 80% of the 204 leaders surveyed, accountability sits with the CEO, the executive team or a named leader, not a dedicated governance committee. If the CoE can't see what agents do in production, neither can the person accountable for them.
That runtime job is what a control plane does, and it's what this article covers.
What is an AI center of excellence?
IBM's definition is one that we will build upon:
"An artificial intelligence center of excellence (AI CoE) is an operational structure dedicated to promoting the adoption, optimization, and governance of AI across an entire organization." - IBM, published 11 December 2024, updated 5 June 2026
In reality, the CoE gets executive sponsorship, forms a cross-functional team, establishes standards and policies, runs a use case intake pipeline, offers shared platforms and enablement, and then reports business value back to management. It also sits right at the heart of a broader AI adoption framework.
One thing worth noting for any CISO reading this is that the CoE is the policy decision-making body - the group deciding what is allowed. The body responsible for implementing that decision to a running system is a completely separate entity, and the two aren't always linked.
What can an AI center of excellence actually enforce?
Even though it owns key choke points, it doesn't need any additional rules. The most important axis isn't centralized vs. decentralized - it's whether there exists a required choke point.
The four things that are worth auditing against your own setup are these:
Funding - Does a brand-new AI project need its budget approval routed through the CoE?
Keys - Does the CoE possess the model gateway and API credentials?
Architecture review - Is submitting to the review board mandatory or practically optional?
Deployment pipeline - Can something get to production without going through the gate owned by the CoE?
Microsoft spells out the enforcing relationship quite clearly on the page that's ranking right now under this term:
"Platform teams enforce consistent governance, manage dependable deployments and help make sure that the delivery is secure over all workloads."
A centrally controlled CoE with no real technical bottleneck posts some recommendations. A federated one with a required gateway will enforce the rules. A federation just distributes delivery - not control.
The handoff everyone recommends
Let's start with what most organizations agree on. Oracle talks about least-privileged agent access and time-sensitive guardrails that "will block harmful content and prevent misuse." AWS states that the CoE "establishes and enforces best practices and governance operations," all set up via "a central platform."
IBM says the CoE "may determine approved architectures and shared services." Microsoft outlines the transfer process in three stages, with a trigger, a destination, and a stated condition before starting. So all the major vendors basically agree on this.
However, none of them provide information on who actually funds the construction of the enforcement mechanism - nor what it is intercepting, what will happen to agents currently running, and how one could even test that all possible areas are covered.
Out of the twelve published guides we reviewed, which included Microsoft, IBM, Oracle, AWS, and Deloitte, none of them said what happens to the agents already running.
The reason the actual transfer rarely finishes is very specific: it's coverage, not authority. The marginal cost of creating an agent is virtually zero. When a business unit decides to leverage Microsoft 365, Salesforce, or ServiceNow under their current company license, it doesn't raise a funding request - meaning it never reaches the funding gate.
The architecture review board then reviews whatever gets presented, but there's no mandate to present anything. Bypass routes include entitlements the company already holds:
Copilot Studio right within an existing Microsoft tenant
Claude on a company laptop
ChatGPT on a company laptop
None of these get caught by a procurement gate since there is literally nothing left to buy. As one Copilot architect said on LinkedIn back in August 2026, the very first question isn't whether to let agents run - most organizations are fine with letting them run by default. It's more about whether anyone has the slightest idea how many agents there are and what they can reach.
This is essentially the framework that SR 11-7 used for model risk back in 2011 - model owner, an independent validation, and internal audit and over half of this audience is probably running it already. AI governance is essentially recreating a much weaker version of a control model that they've long owned.
So who owns enforcement?
Microsoft actually gets this right: platform teams have ultimate responsibility for the enforcement point. What remains is that the actual transfer is mentioned as a bullet point but never actually taken further than that, so nearly nobody has even completed it.
The structure that still exists after a second line review, and maps onto the three lines of defense- is:
Decision rights - The CoE.
Control ownership - Platform engineering as the first line. Build and run the enforcement point.
Standard-setting and independent challenge - Second-line risk or the CISO, depending on the firm. In a highly regulated organization, the CISO cannot both create the enforcement controls and then independently verify them - that really blurs the first/second-line divide.
Assurance - Internal audit, third line. Please don't leave this out. It's the one thing that will actually identify an uncontrolled agent in a very heavily regulated company.
Named accountability - Where necessary, a senior manager with a documented statement of responsibilities. Under SMCR in the UK, a role label doesn't meet that requirement.
The test is far more useful than an organizational chart: the person responsible for enforcing controls is whoever can show an auditor that a non-compliant agent simply couldn't act.
Our view on the actual look and feel of that gate in practice is:
An agent is never going to be an acceptable approver of its own actions and will write out conditions based on blast radius and reversibility rather than uniform consistency.
A global blanket approval per write would be seen as a step backward for control and would be rejected by operations long before risk even gets a look at it.
Common questions
What is an AI center of excellence?
An AI center of excellence (AI CoE) is an operational structure that organizes AI initiatives across an organization, sets the bar, manages use-case intake, and provides common platforms and specialized knowledge. It ensures AI capabilities align with your company's strategic goals, rather than developing each solution in isolation.
What is a center of excellence?
A center of excellence is a cross-functional team that focuses scarce skills and sets the standards for a particular method across the whole organization. The model has been around for decades \- it was used for analytics, RPA, and shared services before someone finally applied it to machine learning.