Why Citizens Advice built a chatbot – then made sure citizens couldn’t use it
Avoiding the ‘chatbot loop of hell’
With more people than ever turning to Citizens Advice for help with issues like debt, housing and employment, the charity itself is feeling the strain. Demand is soaring, but funding, staffing and time aren’t keeping pace.
“Demand for our services is obviously extremely high, yet resource in meeting that demand is quite difficult as a charity,” says Stuart Pearson, head of innovation and business development at Citizens Advice Stockport, Oldham, Rochdale and Trafford (SORT).
Stuart’s team had dipped a toe into the ocean of AI and chatbots in 2019 to address these concerns, but quickly stepped back. The tech just wasn’t ready.
“Advice is really hard,” says Stuart, and the bots’ capabilities were “nowhere near” what vendors were claiming.
The release of ChatGPT in November 2022 changed all that. Generative AI was “like speaking to a person.”
Now, AI is helping (important word) Citizens Advice answer questions, saving time, and supporting inexperienced volunteers. So, how did it get there?
Human-centred design
The first thing the team (originally just two people, but now seven) decided was that any AI solution “cannot be client-facing.” While they could see opportunities there, it was important to implement AI "in a controlled, safe and responsible way.” Human-centred design has sat at the forefront of the bot’s development at every step.
“A lot of the people who approach us, particularly during the cost of living crisis, are vulnerable, upset, confused, angry. The last thing we wanted to do is put them in some kind of chatbot loop of hell where they didn't understand things.”
Without involving the people coming to Citizens Advice for help, the obvious use-case was internal. The team identified the flow of questions between advisors and supervisors as the biggest pain point in the organisation.
“Obviously we've got trainees, we've got volunteers. They don't know everything. What they would do is reach out to an experienced supervisor to say ‘I've got a client; this is their issue. Can I just check is this the right answer, or could you give me some guidance?’”
This wasn’t a problem when everyone worked in an office together, but the remote work trend has made getting timely answers a challenge, leaving clients waiting for answers and eating into their time with an advisor.
“We thought, ‘This is a pain point we can solve.’ We could inject a gen-AI powered chatbot into that support space that could act as a go-between between the advisor and the supervisor. That would be a safe space to deploy it where we could get the benefits and mitigate the risks.”
Meet Caddy
Before writing a single line of code, the team sat down with both supervisors and advisors to gather their input. After stressing that the project wasn’t about replacing anyone – most of whom are volunteers anyway – and getting important feedback on where and how the tool could be used, they got to work.
The result was Caddy: a prototype chatbot that sits between advisors and supervisors to help answer the former’s questions. It was developed and built in-house, though using tech from the private sector.
“The initial pilot was primarily using OpenAI's API,” says Stuart. “However, since we were working with the Cabinet Office we're using a lot of AWS technologies; we're using Anthropic’s Claude and Bedrock to power the LLM.”
Caddy uses retrieval-augmented generation (RAG), and can only pull data from trusted sources – in this case, the gov.uk and Citizens Advice websites.
“We wanted to reduce the risk of hallucinations. We'd already taken the temperature down to 0.3, which is the recommended temperature for legal and medical; but then [also] reduced the information sources it could go to.”
When advisors ask Caddy a question, it will go to its trusted sources and pull the result; but rather than bringing it straight back to the source, it delivers the answer to a supervisor for validation first.
“What we wanted to do is amplify the human experience,” says Stuart. “We wanted to make sure that the client and the advisor have more time together.”
This human-in-the-loop aspect was a core principle of Caddy’s development, and all AI-generated responses require human approval.
The results
Caddy has been demonstrated to Citizens Advice at a national level and adopted at six trial locations across England and Wales. The charity also looped the SORT team into the Cabinet Office’s Incubator for AI; they have been working together for about 12 months.
The Cabinet Office team is "basically work[ing] as our scale-up partner...[to] help us scale it and roll out nationally.” Stuart says the government is looking at Caddy as “a versatile tool with opportunities for replication across various government departments.”
The results from the trial period are encouraging, reducing the time for a supervisor to respond to a query from 10 minutes to four.
“If you extrapolate that across a national rollout, it means we'd be helping a lot more people because the interactions would be quicker.”
Of those answers, Caddy gets about 80% right – as in, requiring no additional information from a supervisor.
Of the 20% of other answers, they mostly “fell into two key buckets”: either that the initial prompt lacked information, or Caddy’s RAG knowledge base did.
To solve the first issue the team is now building a solution to help write better prompts, which is a form of agentic AI:
“Essentially another bot would be Caddy's supporter. So when they ask Caddy a question, this other bot will jump in and say ‘Could you ask these follow-up questions?’”
As for the RAG problem, the team is talking to stakeholders like the Child Poverty Action Group (CPAG) about extending Caddy’s permissions, “which would allow us to increase the accuracy rate from 80%.”
“Trust is a key part of the brand"
The SORT team had consulted advisors and supervisors while building Caddy, but one group of stakeholders had been left out: the people coming for advice in the first place. The team wanted to make sure they were happy with the use of AI in the service.
Citizens Advice SORT is based in Greater Manchester, and were lucky in that Manchester Metropolitan University and Manchester City Council were in the middle of a pilot project called the People's Panel for AI.
Stuart calls the panel “a kind of Dragon’s Den” for trained residents to assess and feed back on AI innovations. Luckily, it gave “a ringing endorsement,” as did Manchester Met itself.
The university was “really impressed” with the implementation of consequence scanning: essentially proving you’ve considered all the way things could go wrong. This was an essential step, because Citizens Advice is “extremely risk averse...[because] we can’t afford to get it wrong.”
“Trust is a key part of the brand. If we're misadvising people, if people think we're using AI to get answers without any kind of human expertise, then it can have a huge impact on us and the future of Citizen's Advice.”
Caddy V2 is now live, with a new pilot underway and preceding a national rollout later this year. Key improvements include a more conversational interface to enhance accuracy, and the integration of more trusted information sources like CPAG.
While there are risks and concerns around the use of AI in the non-profit space, it can also enable new opportunities if used in the right way. The SORT team’s firm guard rails and insistence on human-centred design have been key to Caddy’s successful trial and coming expansion.