The State of AI Infrastructure

Why most companies extract nothing from artificial intelligence, and what the ones who do are doing differently

Picture of Emma Merrill

Emma Merrill

OpenTeams

Based on the NBER Working Paper 34836, a representative survey of nearly 6,000 senior executives across the United States, United Kingdom, Germany, and Australia.

"We have been trading our autonomy for convenience...we need to be aware of what we’re doing. Own your data. Own your result."

Executive Summary

Artificial intelligence adoption among firms is nearly universal. Measurable returns from it are not. Roughly seven in ten companies now use at least one form of AI. Yet across the first representative international survey of firm-level AI use, nearly nine in ten report no measurable improvement in output per employee, and more than nine in ten report no change in workforce size.
This is a problem of architecture. The firms that do see returns share a single defining characteristic: they have moved beyond general-purpose, off-the shelf AI tools and toward AI they own, govern, and have tailored to their specific operations. This piece explains what that transition requires, and offers a practical framework for executives who are ready to make it.

The Core Findings at a Glance

69%

of firms now use AI

89%

report no measurable productivity gain from it

4:1

money spent vs. money returned

Section 1: AI Adoption vs. AI Impact

What firms are doing with AI

AI adoption is nearly universal and continues to rise. The U.S. leads at 78%, followed by the United Kingdom at 71%, Germany at 65%, and Australia at 59%, for an employment-weighted average of 69% across the four countries. Within three years, 75% of firms expect to use AI in some form.
The most common application today is generating text via a general-purpose chatbot, used by 41% of firms. The average executive personally uses AI for approximately 1.5 hours per week, roughly eighteen minutes per working day. That figure, while low, has been rising: in the U.K. alone, executive AI use increased by approximately 50% between early and late 2025, from 0.9 to 1.4 hours per week.

What firms are getting from AI

Firms were asked how AI adoption had affected the volume of sales per employee, the study’s proxy for labor productivity, over the previous three years. Across all four countries, 89% reported no material impact. The weighted average productivity gain across all surveyed firms was 0.29%; less than one-third of one percentage point over three years.
Employment figures follow the same pattern. More than 90% of firms reported that AI had not materially changed their headcount. The small residual effects vary by country and are nearly symmetric: slightly negative in the U.S. (-0.09%) and U.K. (-0.14%), slightly positive in Germany (+0.07%) and Australia (+0.32%). In aggregate, AI has not yet moved the labor market in any measurable direction.
89% of firms report no measurable productivity improvement from AI. The average gain, across all surveyed firms, is 0.29%, cumulative, over three years.

The expectations gap

What makes these figures especially significant is what executives expect next. Looking forward three years, the same respondents forecast a 1.4% productivity gain. That is a five-fold increase from what has been achieved over the past three years. U.S. executives are the most optimistic, projecting a 2.25% gain, followed by the U.K. at 1.86%, Australia at 0.92%, and Germany at 0.87%.

Those expectations may prove correct. Or they may not. The central question this piece addresses is what distinguishes the minority of firms already seeing results, and whether the majority can realistically close the gap.
Metric Past 3 Years Next 3 Years (Expected)
AI Adoption Rate (avg., 4 countries)
69%
75%
Executive personal AI use
1.5 hrs/week
– –
Productivity impact (avg., all firms)
+0.29 cumulative
+1.4%
Employment impact (avg., all firms)
~0% cumulative
-0.7%
Firms reporting no productivity impact
89%
~40% (expected)
Firms reporting no employment impact
90%+
– –

Section 2: Who Sees Results?

The 89% figure marks AI’s failure to deliver as a bimodal distribution. A small minority of firms are reporting meaningful gains; the vast majority are reporting nothing at all. This raises the question, what do the minority have that the majority lack?
The NBER study enables segmentation by firm size, productivity, compensation, leadership age, industry, and country. When researchers isolated which firm characteristics actually predict positive AI outcomes, four variables emerged as independently significant.

The four predictors of AI success

Scale

Larger organizations report higher AI impact. The mechanism is not mysterious: bigger firms have more proprietary data, deeper internal technical capability, and the resource base to build AI that fits their specific context rather than accepting whatever a general-purpose product offers. Scale also creates the operational volume needed to measure AI’s effect rigorously, a point we return to in Section 3.

Productivity baseline

High-performing firms get more from AI. This finding is counterintuitive only if you assume that AI can compensate for operational dysfunction. It cannot. AI systems require clean, structured data and well-defined processes to function reliably. The firms with the strongest pre-AI operations are also the firms best positioned to give AI something useful to work with.

Compensation levels

Firms that pay more attract talent capable of moving beyond off-the-shelf tools. Average wages here serve as a proxy for internal engineering and analytical depth. This is the capability required to customize models, evaluate outputs, and integrate AI into core workflows rather than simply purchasing a subscription and hoping for the best.

Leadership Age

The single most robust predictor across all analysis: firms led by younger executives adopt AI faster, use it more intensively, and project larger returns. The study notes that CEOs use AI personally more than CFOs. Leadership age is a proxy for organizational culture: the willingness to treat AI as a strategic capability to be built.

The pattern these predictors share

Each of these four characteristics points to the same capability to build, run, and govern Ai that the organization owns and controls, rather than accepting what a general-purpose vendor provides. Firms with scale, deep talent, strong data, and forward-looking leadership are better positioned to customize AI to their context. They can train models on proprietary data. They can integrate AI directly into workflows. They can observe what the AI is doing and hold someone accountable when it fails.
The firms succeeding with AI are the ones deploying it with more control, more specificity, and more accountability.
This matters for the reason the study highlights in its look-ahead data. Today, 41% of firms use AI primarily for generating text. Three years from now, the most anticipated application is data processing using machine learning. The shift from the first to the second requires a fundamentally different level of organizational commitment: proprietary data pipelines, custom model development, internal system integration, and ongoing technical management. The firms that have already built those capabilities will have a substantial head start.

Section 3: A Framework for Meaningful AI Ownership

The evidence points toward a conclusion that is easy to state and difficult to operationalize: firms that own their AI infrastructure outperform those that do not. But “ownership” means different things depending on a company’s size, data assets, and technical maturity. What follows is a structured framework for executives who are ready to move from adoption to outcomes.
We organize AI ownership across three dimensions: data, model architecture, and governance. Each dimension involves a spectrum of commitment, rather than a binary choice. And critically, each has a realistic entry point for firms that are not global technology companies.

Dimension 1: Data Ownership

Data is where most firms should start, and where most firms have more to work with than they realize. General-purpose AI tools are already trained on publicly available information. The only input that can produce AI that genuinely outperforms generic alternatives for your specific context is data that exists nowhere else: your customer records, operational logs, transaction histories, and domain expertise. This is a real competitive advantage. It cannot be replicated by a competitor who purchases the same subscription.

But ownership of data is not the same as availability of data. Executives considering a serious AI investment should begin with three concrete steps:

  • Audit what proprietary data actually exists and where it lives. Most organizations discover that their data is more fragmented, inconsistently formatted, and legally encumbered than they assumed.
  • Distinguish between data you have and data you can use. Privacy regulations, contractual restrictions, and legacy systems architectures frequently mean that only a fraction of available data is actually deployable for AI training or retrieval.
  • Build collection pipelines that are continuous, rather than historical. The compounding competitive advantage that proprietary data creates only materializes if capture is ongoing. A one-time data extraction is a starting point, rather than a strategy.

Dimension 2: Model architecture

One of the most important sentences in the original brief is also the least developed: you do not need to build a large language model from scratch. That is true. But the spectrum between “use a chatbot” and “build a foundation model” is wide, and different points on it produce very different levels of ownership and differentiation. Executives need a map.
Context engineering on a commercial model
The lowest-commitment option. You are using a vendor’s model exactly as designed, providing detailed instructions to shape its outputs. Ownership is minimal. Differentiation is almost zero. The barrier to entry for a competitor is a few hours of experimentation. This is appropriate for exploration and low-stakes internal applications, but it does not constitute a strategic AI capability.
Retrieval-augmented generation (RAG)
A commercial model is connected to your proprietary data at inference time, without retraining the underlying model. The AI retrieves relevant internal documents, records, or knowledge before generating a response. This approach meaningfully leverages your proprietary data without the cost and complexity of model training. It is achievable for most mid-to-large firms within months, with a focused team. RAG represents the realistic entry point for firms serious about differentiation.
Fine-tuning an existing open-source model
An existing model, from the growing ecosystem of open-source alternatives, is distilled, quantized, and then further trained on your proprietary data. The result is a model that reflects your organization’s specific knowledge, processes , context, and priorities whether the fine-tuning is done via post-training iterative token prediction on your training data, or using specific prompt / result pairs.This requires more expertise and meaningful data volume, but it is not out of reach for firms with several hundred employees and access to a data team.The differentiation can be substantially higher than RAG alone.
Full model development
Building and training a model architecture from scratch. This is only relevant for a very small number of organizations: large technology companies, countries or coalitions, organizations with extraordinary data scale and unique AI requirements, and research institutions. For almost every company reading this, it is the wrong starting point.
The appropriate point on this spectrum depends on firm size, data volume, technical capability, and use case specificity. A 200-person professional services firm and a 50,000-person industrial manufacturer face genuinely different tradeoffs. What does not vary is the directional principle: the further along this spectrum a firm moves, the more differentiation it creates and the harder that position is to replicate.

Dimension 3: Governance and infrastructure ownership

This is the dimension the original brief treats most superficially, and it may be the most consequential. Ownership of AI infrastructure means more than choosing which model to use. It means building the organizational systems required to observe, control, and be accountable for what your AI actually does.
Observability
Can your organization see what your AI is producing, when it fails, and why? Most firms using off-the-shelf tools cannot answer this question. They have purchased outputs without any visibility into the process generating them. This is not merely a philosophical concern: when an AI system produces an error in a customer-facing context, or surfaces biased results, or hallucinates a fact in a research document, someone needs to know, and that requires instrumentation. Owned AI deployments can build that instrumentation in. General-purpose, AI subscriptions typically cannot.
Environmental control
When an employee pastes proprietary information into a commercial chatbot, where does that data go? Most executives do not know. Many assume it is protected. The terms of service for most general-purpose AI products can tell a different story, that can lead to unintended and potentially disastrous consequences for your proprietary data and intellectual property. Firms that own their deployment environment can enforce a clear answer to this question, and they must, especially in regulated industries.
Accountability structures
Who is responsible when your AI produces a wrong answer? In organizations using generic tools, the honest answer is usually nobody. Governance ownership means defining accountability before a failure occurs: establishing who reviews high-stakes AI outputs, what the escalation path is when the system behaves unexpectedly, and how errors are logged and learned from. These are organizational design questions as much as technical ones.

The perception gap: a governance warning

The NBER study surfaces a finding that deserves dedicated executive attention. When employees at the surveyed firms were asked identical questions about AI’s future impact on employment, their answers diverged sharply from their executives’. Senior leaders expect AI to reduce their workforce by 0.7% over the next three years (1.2% in the U.S. specifically). Employees, responding through the U.S. Survey of Working Arrangements and Attitudes, expect AI to increase employment by 0.5%.
That is a 1.2 percentage point gap in expectations about one of the most consequential decisions an organization will make, and it exists between people in the same companies. This is a governance problem. Organizations without transparent AI strategies, observable AI systems, and shared accountability frameworks will produce exactly this kind of misalignment, and will face its consequences when AI-driven workforce decisions are announced without shared context.

Sequencing: What to Do First

The framework above describes three dimensions of AI ownership. It does not prescribe a sequence. For most organizations, the following order reflects the evidence on where value is generated and where failure is most common.
  • Start with a data audit. Before committing to any specific model architecture, understand what data you have, what you can legally and technically use, and where the gaps are. This step reveals more about your realistic AI options than any vendor conversation will.
  • Choose one high-value use case and build toward RAG. Pick a domain where the cost of generic AI performance is real and observable, customer service accuracy, internal knowledge retrieval, compliance review, and build a retrieval-augmented system against your proprietary data in that domain. Measure rigorously.
  • Instrument before you scale. Before expanding to additional use cases, build the observability and governance infrastructure required to understand what the first use case is actually doing. Scaling broken systems is expensive.
  • Evaluate fine-tuning when you have data volume and measured outcomes. Once you have a working RAG deployment with clear performance data, you have the evidence base needed to evaluate whether fine-tuning would produce meaningful gains in your context.
For firms without internal ML expertise, none of these steps require building everything in-house. AI ownership does not mean AI self-sufficiency. A company can own its data strategy, its governance framework, and its accountability structures while working with an external partner to build the underlying technical system. What matters is that the organization retains control of the assets that produce differentiation.

What Ownership Gets Wrong

An honest treatment of the AI ownership thesis requires acknowledging where it fails. Organizations that pursue deeper AI ownership without discipline encounter:
  • Over-investment in infrastructure that becomes obsolete. The AI model landscape is moving faster than most enterprise technology cycles. Significant investment in a custom deployment architecture can become a liability when the underlying model ecosystem shifts. The mitigation is modularity: own the data layer and the governance layer; treat the model layer as more replaceable.
  • Data governance that creates legal exposure. Assembling proprietary training data without rigorous review of privacy law, data licensing, and contractual terms is a common path to regulatory risk. The legal function should be involved in the data audit before any model development begins, not after.
  • Fine-tuned models that encode historical biases. Models trained on your operational data will reflect whatever patterns existed in that data, including discriminatory ones. This is particularly acute in hiring, lending, and customer service applications. Bias evaluation is not optional; it is a prerequisite for deployment.
  • Ownership theater. The most common failure mode: organizations announce an AI infrastructure initiative, hire a small team, purchase some cloud compute, and declare ownership without actually changing how AI integrates into operations. The test is not whether the organization has built something; it is whether the AI system is observable, accountable, and meaningfully differentiated from what any competitor could purchase tomorrow.

A Diagnostic: Where Does Your Organization Stand?

The following questions are designed to help executive teams identify their current position on the ownership spectrum and their most important next step.

Data Ownership

Can you identify the three largest proprietary data assets your organization holds?
Do you know what proportion of that data is currently structured and accessible for AI use? Has your legal team reviewed your data assets for AI training eligibility?
Has your legal team reviewed your data assets for AI training eligibility?

Model Architecture

Are your AI tools currently connected to any proprietary internal data at inference time? Could a competitor replicate your current AI capability by purchasing the same subscription? Have you evaluated the RAG spectrum against your highest-value use case?

Governance and observability

Do you know where your employees’ prompts and data go when they use commercial AI tools?
Can you measure the error rate of your current AI deployments?
Have you communicated your AI workforce strategy to employees, not just to leadership?

Conclusion

The executives in this study expect AI to deliver five times the productivity gain over the next three years that it delivered over the last three. Some of them will be right. The ones who will be right are the ones building the infrastructure (data, model, governance) that makes AI specific to their context rather than generic to everyone’s.
The window for building that infrastructure is not indefinite. The firms that are doing it now are accumulating proprietary training data, workflow integration depth, and institutional knowledge about what works in their specific environment. Those assets compound. Each quarter of structured data collection, each measured use case, each governance decision made and documented makes the position harder to replicate.
The barrier to meaningful AI ownership is not technical complexity. For most organizations, the real barriers are clarity about what data they hold, discipline about where to start, and the organizational will to build accountability structures before they are needed. None of those are technology problems.
Seven in ten companies are already using AI. Will your organization own what makes it worth having?
Based on NBER Working Paper 34836 | ~6,000 Senior Executives | 4 Countries

Share:

Related Articles

Share Your Feedback

Share your feedback on Collab and Nexus. We read every response.

Collab (Desktop App)

Nexus (Intelligence Hub)

Closing