If you feel that AI pilots are everywhere yet results are scarce, you are not imagining it.

MIT Media Lab’s Project NANDA has just published The GenAI Divide: State of AI in Business 2025 by Aditya Challapally, Chris Pease, Ramesh Raskar and Pradyumna Chari. The analysis of 300 plus enterprise initiatives and interviews with leaders finds that although adoption of tools like ChatGPT and Copilot is high, transformation is low.

The headline is stark: 95 percent of organisations are getting zero return, with only 5 percent of enterprise tools reaching production and delivering measurable impact.

1. High AI failure rates mirror digital transformation rates

The report’s numbers are eye-catching and useful for board conversations:

• Over 80 percent of organisations have explored or piloted general LLM tools and nearly 40 percent report deployment, yet these mainly boost individual productivity rather than P&L outcomes. Meanwhile, 60 percent evaluated enterprise-grade AI, only 20 percent reached pilot, and just 5 percent reached production.
• The authors describe a GenAI Divide: widespread experimentation without structural change. Seven of nine sectors show little or no disruption.
• There is a steep pilot-to-production chasm. Chatbots are easy to try but fail in critical workflows because they lack memory and customisation.
• External partnerships are roughly twice as likely to reach deployment as internal builds.

If you put those figures alongside the broader transformation record, the resemblance is uncanny. Boston Consulting Group’s multi-year studies report that roughly 30 to 35 percent of digital transformations achieve their objectives, while McKinsey has long suggested that around 70 percent of major transformations fall short. In other words, AI projects are not uniquely risky. They are another category of digital change, and they fail for many of the same reasons.

2. Why AI projects fail for the same reasons as other change programmes

The report’s qualitative findings are revealing. When organisations explain stalled AI pilots, the top barriers include user resistance, model quality concerns in context, poor user experience, and weak executive sponsorship. Crucially, the authors argue that the core barrier is learning: most systems do not retain feedback, adapt to context, or improve over time.

As a result, staff happily use consumer tools for simple tasks, but they do not trust enterprise tools for high-stakes work. Users prefer AI for quick drafting and summaries yet strongly prefer humans for complex multi-week work.

Those complaints rhyme with classic transformation pain:

• Unclear strategy and value – investment bias towards visible top-line use cases where attribution is easy. Budgets often over-index on sales and marketing, even though back-office automation yields better ROI.
• Data that does not support ambition – tools struggle without contextual learning, memory, and integration with organisational data and workflow. When underlying data is unstructured, duplicated or siloed, AI performance is capped.
• Change management gaps – users resist new tools that do not fit day-to-day work. Leaders underestimate the steps needed to embed new ways of working and to build trust in automated decisions.

All of this will be familiar to any membership body that has lived through a CRM upgrade, website rebuild or LMS rollout.

3. Intercloud9’s view: poor execution, not poor technology

At Intercloud9 we believe the high failure rate of AI projects is largely a failure of execution. Three themes recur across our client work and sector research.

a) Strategy: start with a concrete problem, an impact hypothesis and a measure

Too many AI initiatives begin with the tool and reverse into the problem. Start instead with a specific, measurable pain point:

• What exactly are we trying to improve? For example, reducing the average time-to-first-response in member support, or cutting manual reconciliation time in finance.
• What is it costing now in £s and member experience?
• What will success look like in 90 days and 12 months, with baselines and an accountable owner?

Our AI Maturity Model is explicit that non-profits should align AI to mission outcomes and focus on narrow, high-value use cases. It emphasises that AI adoption should be measured against strategic objectives, not technology novelty. Use it to frame the business case, the outcome measures and the decision rights before writing a single prompt.

Explore the framework here.

b) Data and architecture: you cannot finesse rubbish in, rubbish out

The sector’s data reality is hard to ignore. Many organisations maintain data in disparate or legacy systems, with weak governance and inconsistent structure. That undermines retrieval, context and model performance. Our model and guidance call out the exact problems you see on the ground: fragmented sources, lack of metadata and data stewardship, and the need to standardise before you scale.

Your peers tell the same story. In our 2025 AI survey of the membership sector, two thirds reported significant concerns about data privacy, a large majority said transparency and explainability are very or extremely important, and only a small minority felt their organisation was doing a great deal to prepare staff. In short, the data is not ready, and the governance is not yet convincing. You will not get reliable AI outcomes until those foundations improve.

c) Change management and governance: the bar is higher for AI, and rightly so

Change is hard in any case, but AI amplifies the governance requirement for three reasons:

• Opacity and error modes – generative models can be useful and wrong at the same time. Hallucinations, outdated training data and brittle prompts are not just quality issues, they are governance issues. Staff need to know when to trust and when to escalate, and members need to trust the outcomes.
• Member and public trust – charities and professional bodies hold a higher bar on fairness, safeguarding and duty of care. AI outputs can affect eligibility, access to services or perceived fairness, so transparency and explainability are essential.
• Regulatory and ethical scrutiny – GDPR, charity governance and procurement rules already apply. AI magnifies these obligations because automated decisions and profiling introduce new risks that boards and trustees must oversee.

Our maturity model therefore embeds governance and ethics as a first-class dimension at every stage. Practical measures include an AI principles statement, Data Protection Impact Assessments for pilots, an ethics and oversight group, staff training, and transparency reporting as you scale.

4. Why generative AI has not yet delivered big operational efficiencies

Generative AI is excellent at language tasks, summarisation and pattern hints. It is less mature for tightly controlled, audit-heavy back-office processes that demand repeatability, whilst simultaneously handling variation, traceability and complex decision making. The dividing line is not intelligence, it is memory, adaptability and integration.

Most enterprise tools today do not learn from your feedback or retain context in a way that reduces toil over time. They are wrappers that require you to re-teach the tool for every task. Until tools learn, remember and slot into your data and workflow, operational efficiency gains will be sporadic.

The best current cases are in well-bounded tasks where vendors have built tightly integrated, learning-capable systems that cut external spend, reduce cycle times and remove manual steps. Examples include call summarisation and routing, document automation and narrow code generation.

5. Why Agentic AI for customer service is not yet producing results at scale

Many organisations start with customer service because it is visible and feels low risk. Results often disappoint for three practical reasons:

• Hard to move from pilot to operations: Chatbots handle FAQs, but struggle with real member cases. They lack memory across conversations, do not deal well with edge cases, and rarely integrate cleanly with CRM systems, knowledge bases and entitlement rules.
• Low trust after small errors: Minor mistakes quickly undermine confidence. Staff are happy to use consumer AI for drafting, but they reject enterprise tools that feel clumsy in live workflows, especially if the system cannot learn and the user experience slows them down.
• Chasing the wrong ROI: Budgets often favour front-office demos because results are easy to show. The better returns are usually in back-office work where you can remove manual steps, reduce outsourcing and shorten cycle times.

The takeaway for membership organisations is simple: do not start with a generic, public-facing chatbot. Start with a narrow, high-value workflow that you control end-to-end, where you can integrate deeply, learn continuously and measure a real cost or experience improvement.

6. What to do next: a practical path for UK non-profits and membership bodies

• Frame a business problem, not a tool – choose one workflow with an executive sponsor and a financial or member-value baseline. Write the 90-day impact hypothesis and define the metric and data you will use.
• Run an AI readiness pulse – assess people, processes, technology, data and governance using a lightweight version of our AI Maturity Model, then focus on the handful of gaps that will block your chosen workflow.
• Make data usable – identify the minimum data set that drives the workflow. Clean it, add metadata, fix identifiers, and put a feedback loop in place so the model can learn from corrections. Define what good looks like as you progress from nascent through integrated.
• Pick a partner who will customise and learn – as discussed in the report, external partnerships are often more likely to reach deployment, especially when the vendor will adapt to your processes and be held to business metrics rather than model benchmarks.
• Design for memory and improvement – require persistent memory or an explicit learning loop. If your vendor cannot show how the system retains context and reduces rework over time, keep looking.
• Bake governance in from day one – publish a short AI principles statement. Complete a DPIA for the pilot. Set up an oversight group that includes a trustee. Define human-in-the-loop checkpoints and clear escalation routes.
• Earn trust with quick, visible wins – start at the workflow edge where error costs are low but value is clear. Scale only once users attest that the tool makes their work easier and the data shows it.
• Measurement is key – track the operational KPI and the spend you eliminate. Focus on retiring manual steps and reducing costs.
• Educate continuously – prioritise training, clear tooling choices and help with data quality.

7. How Intercloud9 can help

We specialise in practical AI adoption for membership and non-profit organisations, with a focus on execution, data and trust. Here are three ways we can support you:

• AI Maturity Model assessment – a rapid diagnostic across people, processes, technology, data and governance to establish your current stage and the small number of actions that will move you forward. The model is designed for trustee-led organisations with limited capacity and a high duty of care to members and the public.
• Targeted pilot design and delivery – we help you select and shape a use case with a measurable baseline, prepare the minimum viable data layer, and partner with vendors who can customise and learn. Our approach is to start narrow, integrate deeply and prove value before scaling.
• Governance and change enablement – we work with you to publish an AI principles statement, embed DPIAs, set up a light-touch ethics and oversight group, and build a training plan that staff actually use, aligned to your CPD frameworks.

If you want a packaged route into this work, our AI Readiness Support Options combine assessment, action planning and the critical enabling steps. The Starter, Intermediate and Advanced packs cover maturity assessment, roadmap, awareness sessions, leadership coaching, data landscape analysis, data governance and infrastructure readiness.

You can explore our thinking and frameworks here.

Final thought

The report confirms what many of you already suspect. AI is not underperforming because the models are useless. It is underperforming because most initiatives are not designed and executed as end-to-end changes in service of a clearly priced problem.

The organisations that win will be the ones that treat AI like any other serious transformation: pick a real pain point, fix the data that matters, integrate with the workflow, learn from feedback, and build trust through transparency.