AI project failure is not new. It looks a lot like digital transformation failure.
23 August 2025 · Jenny Watkins · Best Practice · 9 min read

If you feel that AI pilots are everywhere yet results are scarce, you are not imagining it.
MIT Media Lab’s Project NANDA has just published The GenAI Divide: State of AI in Business 2025 by Aditya Challapally, Chris Pease, Ramesh Raskar & Pradyumna Chari. The analysis of 300 plus enterprise initiatives & interviews with leaders finds that although adoption of tools like ChatGPT & Copilot is high, transformation is low.
The headline is stark: 95 percent of organisations are getting zero return, with only 5 percent of enterprise tools reaching production and delivering measurable impact.
1. High AI failure rates mirror digital transformation rates
The report’s numbers are eye-catching & useful for board conversations:
- Over 80 percent of organisations have explored or piloted general LLM tools and nearly 40 percent report deployment, yet these mainly boost individual productivity rather than P&L outcomes. Meanwhile, 60 percent evaluated enterprise-grade AI, only 20 percent reached pilot, and just 5 percent reached production.
- The authors describe a GenAI Divide: widespread experimentation without structural change. Seven of nine sectors show little or no disruption.
- There is a steep pilot-to-production chasm. Chatbots are easy to try but fail in critical workflows because they lack memory & customisation.
- External partnerships are roughly twice as likely to reach deployment as internal builds.
If you put those figures alongside the broader transformation record, the resemblance is uncanny. Boston Consulting Group’s multi-year studies report that roughly 30 to 35 percent of digital transformations achieve their objectives, while McKinsey has long suggested that around 70 percent of major transformations fall short. In other words, AI projects are not uniquely risky. They are another category of digital change, and they fail for many of the same reasons.
2. Why AI projects fail for the same reasons as other change programmes
The report’s qualitative findings are revealing. When organisations explain stalled AI pilots, the top barriers include user resistance, model quality concerns in context, poor user experience & weak executive sponsorship. Crucially, the authors argue that the core barrier is learning: most systems do not retain feedback, adapt to context, or improve over time.
As a result, staff happily use consumer tools for simple tasks, but they do not trust enterprise tools for high-stakes work. Users prefer AI for quick drafting & summaries yet strongly prefer humans for complex multi-week work.
Those complaints rhyme with classic transformation pain:
- Unclear strategy & value – investment bias towards visible top-line use cases where attribution is easy. Budgets often over-index on sales & marketing, even though back-office automation yields better ROI.
- Data that does not support ambition – tools struggle without contextual learning, memory & integration with organisational data & workflow. When underlying data is unstructured, duplicated or siloed, AI performance is capped.
- Change management gaps – users resist new tools that do not fit day-to-day work. Leaders underestimate the steps needed to embed new ways of working and to build trust in automated decisions.
All of this will be familiar to any membership body that has lived through a CRM upgrade, website rebuild or LMS rollout.
3. Intercloud9’s view: poor execution, not poor technology
At Intercloud9 we believe the high failure rate of AI projects is largely a failure of execution. Three themes recur across our client work and sector research.
a) Strategy: start with a concrete problem, an impact hypothesis & a measure
Too many AI initiatives begin with the tool and reverse into the problem. Start instead with a specific, measurable pain point:
- What exactly are we trying to improve? For example, reducing the average time-to-first-response in member support, or cutting manual reconciliation time in finance.
- What is it costing now in £s & member experience?
- What will success look like in 90 days and 12 months, with baselines & an accountable owner?
Our AI Maturity Model is explicit that non-profits should align AI to mission outcomes and focus on narrow, high-value use cases. It emphasises that AI adoption should be measured against strategic objectives, not technology novelty. Use it to frame the business case, the outcome measures & the decision rights before writing a single prompt.
b) Data & architecture: you cannot finesse rubbish in, rubbish out
The sector’s data reality is hard to ignore. Many organisations maintain data in disparate or legacy systems, with weak governance & inconsistent structure. That undermines retrieval, context & model performance. Our model & guidance call out the exact problems you see on the ground: fragmented sources, lack of metadata and data stewardship & the need to standardise before you scale.
Your peers tell the same story. In our 2025 AI survey of the membership sector, two thirds reported significant concerns about data privacy, a large majority said transparency & explainability are very or extremely important, and only a small minority felt their organisation was doing a great deal to prepare staff. In short, the data is not ready, and the governance is not yet convincing. You will not get reliable AI outcomes until those foundations improve.
c) Change management & governance: the bar is higher for AI, and rightly so
Change is hard in any case, but AI amplifies the governance requirement for three reasons:
- Opacity & error modes – generative models can be useful & wrong at the same time. Hallucinations, outdated training data & brittle prompts are not just quality issues, they are governance issues. Staff need to know when to trust and when to escalate, and members need to trust the outcomes.
- Member & public trust – charities & professional bodies hold a higher bar on fairness, safeguarding & duty of care. AI outputs can affect eligibility, access to services or perceived fairness, so transparency & explainability are essential.
- Regulatory & ethical scrutiny – GDPR, charity governance & procurement rules already apply. AI magnifies these obligations because automated decisions & profiling introduce new risks that boards & trustees must oversee.
Our maturity model therefore embeds governance & ethics as a first-class dimension at every stage. Practical measures include an AI principles statement, Data Protection Impact Assessments for pilots, an ethics & oversight group, staff training & transparency reporting as you scale.
4. Why generative AI has not yet delivered big operational efficiencies
Generative AI is excellent at language tasks, summarisation & pattern hints. It is less mature for tightly controlled, audit-heavy back-office processes that demand repeatability, whilst simultaneously handling variation, traceability & complex decision making. The dividing line is not intelligence, it is memory, adaptability & integration.
Most enterprise tools today do not learn from your feedback or retain context in a way that reduces toil over time. They are wrappers that require you to re-teach the tool for every task. Until tools learn, remember and slot into your data & workflow, operational efficiency gains will be sporadic.
The best current cases are in well-bounded tasks where vendors have built tightly integrated, learning-capable systems that cut external spend, reduce cycle times and remove manual steps. Examples include call summarisation & routing, document automation & narrow code generation.
5. Why Agentic AI for customer service is not yet producing results at scale
Many organisations start with customer service because it is visible and feels low risk. Results often disappoint for three practical reasons:
- Hard to move from pilot to operations: Chatbots handle FAQs, but struggle with real member cases. They lack memory across conversations, do not deal well with edge cases, and rarely integrate cleanly with CRM systems, knowledge bases & entitlement rules.
- Low trust after small errors: Minor mistakes quickly undermine confidence. Staff are happy to use consumer AI for drafting, but they reject enterprise tools that feel clumsy in live workflows, especially if the system cannot learn and the user experience slows them down.
- Chasing the wrong ROI: Budgets often favour front-office demos because results are easy to show. The better returns are usually in back-office work where you can remove manual steps, reduce outsourcing and shorten cycle times.
The takeaway for membership organisations is simple: do not start with a generic, public-facing chatbot. Start with a narrow, high-value workflow that you control end-to-end, where you can integrate deeply, learn continuously and measure a real cost or experience improvement.
6. What to do next: a practical path for UK non-profits & membership bodies
- Frame a business problem, not a tool – choose one workflow with an executive sponsor & a financial or member-value baseline. Write the 90-day impact hypothesis and define the metric & data you will use.
- Run an AI readiness pulse – assess people, processes, technology, data & governance using a lightweight version of our AI Maturity Model, then focus on the handful of gaps that will block your chosen workflow.
- Make data usable – identify the minimum data set that drives the workflow. Clean it, add metadata, fix identifiers, and put a feedback loop in place so the model can learn from corrections. Define what good looks like as you progress from nascent through integrated.
- Pick a partner who will customise and learn – as discussed in the report, external partnerships are often more likely to reach deployment, especially when the vendor will adapt to your processes and be held to business metrics rather than model benchmarks.
- Design for memory & improvement – require persistent memory or an explicit learning loop. If your vendor cannot show how the system retains context and reduces rework over time, keep looking.
- Bake governance in from day one – publish a short AI principles statement. Complete a DPIA for the pilot. Set up an oversight group that includes a trustee. Define human-in-the-loop checkpoints & clear escalation routes.
- Earn trust with quick, visible wins – start at the workflow edge where error costs are low but value is clear. Scale only once users attest that the tool makes their work easier and the data shows it.
- Measurement is key – track the operational KPI & the spend you eliminate. Focus on retiring manual steps and reducing costs.
- Educate continuously – prioritise training, clear tooling choices & help with data quality.
7. How Intercloud9 can help
We specialise in practical AI adoption for membership & non-profit organisations, with a focus on execution, data & trust. Here are three ways we can support you:
- AI Maturity Model assessment – a rapid diagnostic across people, processes, technology, data & governance to establish your current stage & the small number of actions that will move you forward. The model is designed for trustee-led organisations with limited capacity & a high duty of care to members & the public.
- Targeted pilot design & delivery – we help you select and shape a use case with a measurable baseline, prepare the minimum viable data layer, and partner with vendors who can customise and learn. Our approach is to start narrow, integrate deeply and prove value before scaling.
- Governance & change enablement – we work with you to publish an AI principles statement, embed DPIAs, set up a light-touch ethics & oversight group, and build a training plan that staff actually use, aligned to your CPD frameworks.
If you want a packaged route into this work, our AI Readiness Support Options combine assessment, action planning & the critical enabling steps. The Starter, Intermediate & Advanced packs cover maturity assessment, roadmap, awareness sessions, leadership coaching, data landscape analysis, data governance & infrastructure readiness.
You can explore our thinking & frameworks here.
Final thought
The report confirms what many of you already suspect. AI is not underperforming because the models are useless. It is underperforming because most initiatives are not designed and executed as end-to-end changes in service of a clearly priced problem.
The organisations that win will be the ones that treat AI like any other serious transformation: pick a real pain point, fix the data that matters, integrate with the workflow, learn from feedback, and build trust through transparency.