When AI fails in social protection, it’s rarely technical: Lessons from the AI Hub Clinic

The promise of artificial intelligence in social protection is increasingly difficult to ignore – faster targeting, reduced fraud, and improved service delivery. But before considering any application, it is critical to ask this question: is your institution actually ready? 
 
That was the central theme of the first session of the AI Hub Clinic Series, hosted by the AI Hub for Social Protection under the Digital Convergence Initiative (DCI). Held on April 14, 2026, the session brought together practitioners and experts from Canada and Uzbekistan to explore what AI readiness really means, the risks of premature applications, and detecting early warning signs.  

Readiness is an institutional question, not a technical one

The session made it clear that harms caused by AI in social protection are rarely technical failures, but institutional shortcomings. The very same AI systems that can affect positive changes to social protection delivery, can also cause new harms. The capacity of implementing institutions can determine the outcome. Acknowledging this in advance means addressing the following three pillars of readiness: 

• Data readiness: Is the data accurate, complete, and representative especially for marginalised populations? 

• Legal and regulatory frameworks: Is there explicit legal authority for automated or AI-assisted decisions? 

• Governance and organisational capacity: Are the people, processes, oversight structures, and infrastructure to operate AI responsibly? 

These are not boxes to tick after procurement, but preconditions for adopting AI responsibly, to be continually addressed throughout the AI lifecycle. 

The session drew on a variety of real cases to highlight the cost of adopting AI in social protection without the proper readiness. For instance, Australia’s Robodebt scheme issued over 500,000 unlawful automated debt notices and ended in a AU$1.8 billion class action settlement. In the Netherlands, an algorithmic childcare benefits system wrongly accused more than 26,000 families of fraud – disproportionately migrants – and led to the resignation of the government. In Michigan, USA, an automated unemployment fraud system achieved a 93% error rate while the agency treated rising penalty revenue as a success indicator.
The pattern across cases is consistent: governance gaps, absent human oversight, unverified legal authority, and flawed data all led to increased challenges and inequality.

Self-assessment tool

These failures share a common root, which the AI Hub aims to help address. A key output from this session is the draft AI Readiness Self-Assessment Tool for Social Protection, developed by the AI Hub specifically for programme managers, digital leads, and senior officials who are not AI specialists. The tool is designed to help agencies answer one practical question: are we institutionally ready to introduce AI responsibly, and if not, what do we need to do first? 

It covers six key areas for identifying AI readiness: 

1. Strategy and political readiness: Does the institution have the expertise and vision for responsible AI adoption?  
2. Legal and regulatory readiness: Is there a legal basis and regulatory framework for the use of AI in social protection?  
3. Data readiness: Are there risks of bias, exclusion, or misrepresentation in the data used? 
4. Technology and infrastructure readiness: Do existing systems support effective AI deployment? 
5. Governance and capacity readiness: Are there clear responsibilities and internal structures for oversight? 
6. Accountability and ethics: Can AI-supported decisions be contested? Are affected communities protected?  

The AI Hub’s self-assessment tool is still evolving, as it is being developed collaboratively with practitioners and ultimately intended for public use. 

Country voices: peer-to-peer exchange

The session was designed to allow for peer-to-peer exchange, specifically bringing in the direct experience of practitioners from Canada and Uzbekistan. 

Raphael Duteau, from Employment and Social Development Canada (ESDC), shared how Canada has approached AI governance. He emphasised the importance of internal accountability structures and iterative human-in-the-loop design, even as AI use scales up. Previously, ESDC’s use of AI relied mainly on machine learning engineering. Models were trained internally using specific datasets for specific tasks, making risks and potential gaps easier to identify and manage because the training data was known and controlled. However, increased use revealed existing challenges, such as knowing what data was used to train these models, what biases they may contain, what forms of discrimination they may generate, and whether they are suitable for public service delivery. Through that experience, the ESDC concluded that AI readiness is both organisational and case specific. Mr. Duteau affirmed that institutions need processes to test different use cases, evaluate risks, and recognise that each case requires a different approach. To address this, ESDC created an internal register that assesses each AI use case across different categories, helping ensure that AI is used safely, ethically, and in accordance with internal standards and directives.

Dr. Amirbek Pulatov from the National Agency for Social Protection (NASP) in Uzbekistan offered a different context: a country rapidly digitising its social protection infrastructure, navigating the tensions between ambition and institutional maturity. The NASP approaches readiness through four key dimensions: (1) data readiness, (2) governance, (3) legal and ethical safeguards, and (4) organisational capacity. The NASP’s experience with datasets from call centres showed that data volume is not enough to account for data readiness; the data must also be of high quality, properly labelled and fit for training purposes. In the same manner, unclear data ownership/accountability and lack of standardised workflow undermined governance readiness, driving the NASP to work on strengthening coordination and internal processes.  

Finally, our understanding of capacity readiness must not be limited to engineers who need AI knowledge, but also managers and decision-makers who must understand the risks, limitations, and implications of AI systems. Continuous training, knowledge sharing, and collaboration are essential to ensuring responsible AI use. Dr Pulatov stressed that all four dimensions must work together for successful AI deployment. AI readiness is not simply about having the technology or technical expertise but about building a strong institutional foundation capable of delivering meaningful and responsible impact. 

Diving deeper: insights from the open discussion 

During the Clinic, participants encouraged the inclusion of civil society in the process of implementing AI in social protection by strengthening citizen participation and capacity (literacy, skills, and awareness) to participate. In the same light, participants in the Clinic emphasised the need for mechanisms for feedback loops which act as a source of empowerment for government officials to contest AI-generated decisions, protecting those directly affected by these decisions and those advocating for them.  

Asking participants, ‘where is the biggest readiness gap in your institution today?’ sparked interesting insights.  Out of multiple replies, several highlighted digital literacy and data handling; be it regulatory oversight, data integration, data structing/labelling, or data verification. This highlights the necessity of prioritizing investments in data and technical skills before making the decision to adopt AI.

Another important discussion arose when addressing this question: what should you avoid doing too early? Contributions from participants emphasised that AI cannot be adopted in social protection systems without prior due diligence on social impact, nor should it be adopted without clear rules and regulations. It is useful, especially for data management analysis, to invest in pilots, and the training of staff to steer and document learnings. Ignoring social impact and scaling without governance undermines the potential of AI in social protection.

Finally, participants agreed that institutions must not rush into adopting AI, or replacing in-person services with digital ones, too early without sufficient data readiness and clear use cases. Rushing without proper institutional and technical readiness can significantly compromise data sovereignty, thereby compromising the data of citizens and their right to receive social protection services.  

Moving towards implementation 

Perspectives from Duteau and Dr. Pulatov, next to the open discussion, reinforced what the framing input argued: context matters, but the questions institutions need to answer are the same. 

It is important for institutions and decision-makers to move from abstract ambitions to assess real-life constraints, gaps, and enablers for AI readiness. In doing so, they should ask concrete and tangible questions, guided by the six key areas for identifying AI readiness highlighted in the AI Hub self-assessment tool.  

The AI Hub for Social Protection provides advisory services to countries in the areas of AI policy, strategy, governance, and the technical development of AI models. Interested in working with us? Please e-mail us at contact@spdci.org.  

The AI Hub Clinic Series 

This Clinic was the first of six in the series. You can watch the full recording in English here, and translations are available in French and Spanish. 

The AI Hub Clinic Series is designed as a 6-part practical learning journey for social protection institutions that are exploring, planning, or already experimenting with artificial intelligence. Rather than starting with tools, the series begins with a more fundamental question: what needs to be in place for AI to be used responsibly and effectively? 

Our upcoming session on ‘Data foundations for AI: Weak data, weak decisions’ will take place on July 2, 2026, you can register to attend here. Information about the following sessions can be found on the AI Hub webpage. 

Author: Nour Barakat