The AI gold rush has a cost hangover. Global tech giants have announced $740 billion in capital expenditure for 2026 alone - a 69% jump over 2025 - yet there remains no clear evidence of AI driving broad increases in productivity. At some point, the cost of all those GPUs and cloud instances can outstrip the salaries of the very people they were meant to replace. Welcome to the great AI CAPEX reckoning - where we shift from scaling at all costs to scaling with a business case. [morganstanley.com] [finance.yahoo.com] What you’ll learn:
- Where AI stops making economic sense: Why the “Catanzaro Threshold” - when compute spending per employee overshoots labour costs - is a red flag, and how to spot it early.
- Why ~77% of tasks resist automation: A sector‑by‑sector look at the barrier, the kinds of work still cheaper for humans, and the ~23% of genuine “Viable Wins” (with methodology caveats).
- How to audit and right‑size your AI investments: A Rational Automation Framework with proposed metrics and decision criteria for choosing small language models or heuristics over expensive frontier AI.
1. The Unit Economics of Displacement
Takeaway: When your GPU fleet becomes your priciest headcount, you’ve crossed the wrong line. “For my team, the cost of compute is far beyond the costs of the employees,” Nvidia VP of Applied Deep Learning Bryan Catanzaro recently told Axios. That inflection point - let’s call it the Catanzaro Threshold (a conceptual label, not a formal standard) - is when each additional pound spent on AI infrastructure yields less value than a pound spent on talent. [techspot.com] What does the compute side actually cost? Based on current Blackwell‑generation hardware, a single 8‑GPU server runs approximately $350,000 in CapEx; a 576‑GPU mid‑size cluster totals roughly $35.2 million ($61,000 per GPU); and a 24,576‑GPU hyperscale deployment reaches approximately $1.2 billion ($47,000 per GPU). GPUs account for 60–70% of total cluster CapEx, with networking at 10–25% and the remainder split across storage, cabling, management infrastructure, and software. Inside each server, the HGX GPU board alone costs $200,000–$300,000+, with CPUs, RAM, NVMe storage, network adapters, and cooling bringing the per‑server total to $250,000–$400,000 at B200/B300 generation pricing. And that’s before energy, facilities, and the MLOps engineers to keep it all running. [amcompute.com] Token‑based pricing compounds the problem. Because charges rise with every request, costs spike unpredictably as more teams adopt tools - harder to forecast than standard software licences. Uber’s CTO Praveen Neppalli Naga admitted mid‑year that the budget “I thought I would need is blown away already,” largely due to surging token costs from AI coding tools. [techspot.com] [finance.yahoo.com] The uncomfortable question: are organisations actually saving money, or just shifting the expense from payroll to cloud bills? When compute becomes a metered, recurring operating cost rivalling headcount, the savvy move is to pause and re‑balance - ensuring each pound of compute is earning its keep in measurable productivity gains. —
2. The “77% Barrier” Map
Takeaway: For over three‑quarters of vision‑related tasks studied, humans remain the cheaper option. A rigorous 2024 study by MIT FutureTech, the Productivity Institute, and IBM’s Institute for Business Value analysed 420 computer vision tasks, surveying 5–9 workers per task. The core finding: only 23% of worker wages being paid for vision tasks would be economically attractive to automate. In the remaining 77%, it was cheaper to keep humans on the job. More strikingly, while 36% of all US non‑farm jobs have at least one task exposed to computer vision, just 8% have a task that is economically attractive to automate. [theregister.com][ide.mit.edu] The barrier is largely one of minimum viable scale: most computer vision systems are cost‑effective only when offered as a cloud‑based service across entire sectors or even the whole economy, not deployed within a single organisation. The researchers estimated that even with a 50% annual cost decrease, it would not be until 2026 that half of all vision tasks reached machine economic advantage; at a more conservative 10% annual decrease, that threshold would not arrive until after 2042. [ide.mit.edu] Methodology caveat: these percentages relate specifically to computer vision tasks, not all work. But the underlying logic - high fixed costs amortised over limited task volume - applies broadly. Below is an analytical map (judgement, not source data) of where the barrier bites hardest:
| Sector | Why Humans Remain Cheaper (~77%) | The ~23% Viable Wins |
|---|---|---|
| Public Sector & Government | High‑context casework, accountability requirements, low‑frequency complex decisions. Custom AI rarely hits minimum viable scale in individual departments. | High‑volume document classification, simple FAQ chatbots, routine form processing. |
| Healthcare | Clinical edge cases, empathy, legal liability, physical tasks (wound care, patient handling). | Radiology image screening (under clinician supervision), administrative transcription and scheduling. |
| Manufacturing & Logistics | Fine motor dexterity, unstructured environments, small‑batch variability. A bakery with five bakers paying $48,000 each saves only ~$14,400/yr on ingredient inspection - far below deployment cost [ide.mit.edu]. | High‑volume quality inspection on consistent production lines; repetitive warehouse routing in large hubs. |
| Professional Services | Low‑frequency, bespoke judgement (legal drafting, IT architecture). Senior expertise is still cheaper for non‑routine work. | Document‑heavy triage (e.g. bulk contract review, financial report scanning), boilerplate code generation. |
3. The $740 Billion Paper Trail
Takeaway: Capital is surging, but much of it looks defensive - and returns are unproven. Morgan Stanley Research reported in March 2026 that major global technology companies have announced $740 billion in CapEx for the year - a 69% increase over 2025. Yet this spending surge has coincided with no widespread evidence of AI displacing jobs, according to the Yale Budget Lab, and no clear evidence of AI improving broad productivity. In parallel, the tech sector has seen more than 92,000 layoffs in 2026 so far across nearly 100 companies, outpacing last year’s total of roughly 120,000. [morganstanley.com][finance.yahoo.com] Where is the money flowing? A significant share appears to be defensive CapEx - spending driven by fear of falling behind competitors rather than proven ROI:
- Data centre and GPU build‑outs: stockpiling capacity for workloads that may or may not materialise.
- AI tooling and model licences: costs that have, in cases like Uber’s, blown past annual budgets within months. [techspot.com]
- Experimental “skunkworks” projects across departments, many without a clear business case.
Enterprises face mounting pressure to demonstrate measurable gains - companies are being pushed to show productivity metrics that tie AI spending to business results. In the UK public sector, this scrutiny is formalising: the Central Digital and Data Office has worked with HM Treasury to develop assessment criteria for the spending review, with the stated aim of “ensuring investment in AI technology provides a significant return on investment”. There is also “ongoing work to analyse the potential cost and value of AI for civil service functions”. [techspot.com] [publictechnology.net] The countervailing view: Morgan Stanley analysts estimate AI tools could boost bank productivity by 20–50% over five to ten years, with an estimated 18% improvement in pre‑tax income once AI is fully embedded across workflows. The opportunity is real - but it is long‑term and conditional, not guaranteed by CapEx alone. [morganstanley.com] —
4. The Rational Automation Framework
Takeaway: Measure, focus, and pick the cheapest tool that works. To avoid contributing to an infrastructure bubble, treat every AI initiative with the same investment discipline you would apply to a major procurement. Below is a proposed audit checklist - the metrics are illustrative, not industry‑standard terms, but designed to force honest accounting:
Proposed Audit Metrics
| Metric | What It Measures | When to Worry |
|---|---|---|
| Inference‑to‑Income Ratio (proposed) | AI compute spend ÷ revenue (or output value) attributable to that AI system | Rising over consecutive quarters - you’re spending more per unit of value delivered. |
| Human‑Efficiency Parity (proposed) | Cost per task via AI vs. cost per task via human worker | AI cost per task > human cost per task = you haven’t reached parity. Don’t scale until you do. |
When to Choose SLMs or Heuristics Over Frontier Models
According to a 2026 industry comparison, small language models (SLMs) can be 5–20× cheaper to run than large language models (LLMs), with self‑hosted costs of $500–$2,000/month versus $5,000–$50,000/month for equivalent LLM API usage at scale. SLMs also offer sub‑second inference, on‑premises data control, and fast, inexpensive fine‑tuning. Use the following decision logic: [lucas8.com]
- Is the task narrow and well‑defined? → SLM fine‑tuned on domain data will likely match frontier accuracy at a fraction of the cost. [lucas8.com]
- Must data stay on‑premises? → SLM on private infrastructure is often the only compliant option. [lucas8.com]
- Is response latency critical? → Smaller models serve real‑time applications faster. [lucas8.com]
- Is the problem solvable with rules? → A regex, heuristic, or traditional algorithm costs effectively nothing in compute. Not every workflow needs a language model.
If the answer to all four is “no,” a frontier LLM via API is likely justified. Many teams are settling on a hybrid approach: frontier AI for complex, unpredictable queries and SLMs or heuristics for high‑volume, well‑defined tasks. [lucas8.com]
Governance guardrail
Bake in budget caps, MLOps monitoring, and contingency plans. One engineer recounted how an overzealous AI agent “destroyed his database” and network through overuse. Good governance catches cost spirals and failure modes weeks before they surface in quarterly reports. [finance.yahoo.com]
What to Do Monday Morning
- Map your AI spend. Get a single‑page breakdown of all AI‑related expenditure - cloud bills, model licences, project costs. You cannot bridge a gap you cannot see.
- Identify one quick win and one quick kill. From that map, find one project delivering measurable value (scale it) and one that has been in “pilot” with no clear ROI (freeze it). Build momentum with the board.
- Institute a “Value Gate.” Require every new AI proposal to state which KPI it will improve, by roughly how much, and within what timeframe. No green light without it.
- Pressure‑test your model choices. For each deployed or planned AI system, ask: could a smaller model, a heuristic, or a human‑in‑the‑loop process achieve the same outcome more cheaply? If yes, switch.
- Upskill and partner. Invest in FinOps‑for‑AI skills across your technology and finance teams. And where an external perspective would help, bring one in.
Devsultants LLP specialises in helping organisations navigate exactly this kind of reckoning. Our discovery and de‑risking engagements pinpoint where AI genuinely moves the needle - and where it shouldn’t be applied. We bring deep expertise in data enrichment, AI/RAG, cloud strategy and cost optimisation, DV/SC‑cleared delivery for security‑sensitive public‑sector programmes, and business case development for emerging technology. In an age of exuberance, we’re your sober second opinion. Modernise, but don’t mechanise for its own sake. The winners of this phase will be those who deploy AI deliberately, cost‑effectively, and with evidence. It’s time to bridge the gap between compute and value.
