• Home
  • News
  • Big Data & Analytics
  • DevOps
  • Security
  • GDPR
  • AI & ML
  • Women in Tech
  • Cloud & Infrastructure
  • CIO
  • Deskflix
  • Events
  • Whitepapers
  • Spotlights
  • IT Leaders 250
  • Research
  • Delta
  • Tech Marketing Hub
  • About Us
  • Newsletters
  • Sign in
  •  
      • Newsletters
      • Account details
      • Contact support
      • Sign out
     
     
    • You are currently accessing Computing via your Enterprise account.

      If you already have an account please use the link below to sign in.

      If you have any problems with your access or would like to request an individual access account please contact our customer service team.

      Phone: +44 (0) 1858 438800

      Email: customerservices@incisivemedia.com

      • Sign in
     
  • Follow us
    • Twitter
    • LinkedIn
    • Newsletters
    • Facebook
    • YouTube
  • Register
  • Events
    • Upcoming events
      event logo
      Where the buck stops: Why a shared responsibility model will help you own your cloud security flaws

      This webinar, and accompanying dedicated research, will reveal to what extent organisations are practicing a shared responsibility model for cloud security today and the degree to which IT leaders are aware of what they should be doing to ensure the secure use of their multi- and hybrid-cloud environments.

      • Date: 27 Jan 2021
      event logo
      Leveraging the Cloud to Defeat Data Disasters

      Join us and learn how your IT team can realize many of the powerful advantages of the cloud and solve the operational complexity behind managing data across hybrid and multi-cloud IT environments with centralized management, automation, end-to-end security, and lower TCO.

      • Date: 28 Jan 2021
      event logo
      Deskflix Hybrid and Multi Cloud

      One of the most powerful tools for breaking down silos and integrating resources is cloud computing. But multi-tenancy cloud is not the ideal environment for every application or every class of data and some will need to remain on-prem for the foreseeable future; nor are all clouds equal. Tune in to Deskflix season 1 to hear industry experts speak on the questions you need answered on hybrid and multi cloud.

      • Date: 10 Feb 2021
      event logo
      Deskflix Financial Services

      oin us for this episode of Deskflix to hear from industry experts and peers on their 2020 best practices, what they’ve learnt for 2021 and how they plan to overcome the next wave of disruption.

      • Date: 03 Mar 2021
      View all events
  • Whitepapers
    • LATEST WHITEPAPERS
      Darktrace 120x194
      Cyber AI Response: Threat Report 2019

      This white paper details 7 case studies of attacks that were intercepted and neutralised by Darktrace cyber defense AI, including a zero-day trojan in a manufacturing company's network. Learn how Darktrace Antigena AI Response modules fight back autonomously, no matter where a threat may emerge, extending to the Cloud, Email and SaaS.

      Download
      Darktrace 120x194
      Cyber AI & Darktrace Cloud

      This white paper explores how cloud is a security blind spot for many organisations who struggle with the limited visibility and control in this new environment, where their existing security tools are often not applicable.

      Download
      Find whitepapers
      Search by title or subject area
      View all whitepapers
  • Spotlights
    • Spotlights

      Welcome to Computing's Spotlight section, where we focus in on particularly important themes and topics of enterprise IT.

      Intel logo

       

      Endpoint Management and Security Hub

  • IT Leaders 250
  • Research
  • Delta
  • Tech Marketing Hub
  • About Us
Computing
Computing
  • Home
  • News
  • Big Data & Analytics
  • DevOps
  • Security
  • GDPR
  • AI & ML
  • Women in Tech
  • Cloud & Infrastructure
  • CIO
  • Deskflix
 
    • Newsletters
    • Account details
    • Contact support
    • Sign out
 
 
  • You are currently accessing Computing via your Enterprise account.

    If you already have an account please use the link below to sign in.

    If you have any problems with your access or would like to request an individual access account please contact our customer service team.

    Phone: +44 (0) 1858 438800

    Email: customerservices@incisivemedia.com

    • Sign in
 
  • Developer

Behind the scenes at the BBC's groundbreaking Own It app

App designed to safeguard children online was the result of two years' toil at AI's cutting edge, says product manager Jon Howard

  • John Leonard
  • John Leonard
  • @_JohnLeonard
  • 30 July 2020
  • Tweet  
  • Facebook  
  • LinkedIn  
  • Send to  
0 Comments

The BBC is in a unique position when it comes to developing AI, says Jon Howard, executive product manager children's future media at the broadcaster. 

Large enough to make the most of relationships with the tech giants working at the cutting edge of the craft like Apple and Google's TensorFlow engineers, its remit also enables the broadcaster to step into areas frequently neglected by VC-funded start-ups.

When looking at ways to tackle the epidemic of online bullying and childhood mental health problems related to living life on screen, Howard's team asked 50 parents and 50 children what they'd like to see in an app dedicated to children's' wellbeing. The parents universally wanted a monitoring app while the children universally did not, so that was that. In any case, there are already plenty of parental monitoring apps, said Howard, because that's where the money is, but there was nothing to help children to manage their own experiences on social media and the wider online world.

"We saw this as a key space where the BBC could help out," said Howard (pictured), speaking to Computing after the Deskflix Public Sector event where he delivered a keynote presentation. "Here's where our public service principles could play their part."

New worlds

Many kids receive their first smartphone between the ages of 8 and 12 and become active on social media, entering a world governed by a very different set of rules, norms and expectations. It's a time of life when they are exploring new horizons, and unlike in the physical world where a disapproving glance from an adult is enough to tell them they may be acting inappropriately, cyberspace offers little in the way of guardrails.

Children's charity NSPCC has registered a sharp rise in referrals to mental health services, some of which is related to the use of social media. The BBC's Own IT app seeks to replace the adult in the room with an AI in the phone, an advisor who can warn the child that the message they're about to send may be hurtful to the recipient, or that sending their phone number to a new contact may not be a good idea. 

Own It is a keyboard for iOS and Android plus a companion app that enables the user to record their feelings. Designed in consultation with child psychologists, the Turing Institute and Public Health England, the idea is to help children to better understand their behaviour online and its effect on others, and to navigate the new world safely. It adopts a ‘nudge' approach rather than being proscriptive and is focused purely on output - what children are saying rather than what they're seeing. In this way, it can work in conjunction with monitoring apps if parents insist.

You say it you own it

The product of two years' development work, the Own It keyboard deploys machine learning models to support autocorrect, autocomplete and next-word-prediction functions based on how children really communicate online. In this, it works in a similar way to Google's GBoard keyboard, but using kids' vocabulary. But there's also a proactive element: start typing a message with ‘I hate you…' and the guardrails kick in. The UI border turns red and a worried-looking emoticon pops up with a message "How would you feel if someone sent you that?" The keyboard does not stop the hateful message being sent because Own It is about encouraging children to be responsible for their own behaviour, offering guidance rather than policing. Instead it prompts the user to try a different set of words until a happy face and calm blue border appears.

"Kids generally want to be good, they generally want to be nice people," said Howard. "We won't stop them sending a message, we just provide a bit of friction."

Own It keyboard. Source: BBC
Own It keyboard

The companion app, meanwhile, allows the user to keep a private record of their thoughts and feelings, which has been shown to be an effective psychological approach to combating stress.

The app has won many plaudits, including a CogX award for best innovation in natural language processing (NLP), a UXUK gong for best design for education, and a Banff World Media award for best interactive content for young people, plus there's been international interest from other publishers and broadcasters.

"A lot of people are taking notice", Howard said.

Own it was developed by ten BBC product and project managers, technical architects and UX and editorial content producers who worked with five engineers from Swiss consultancy Privately to develop an SDK containing the machine learning, AI and business logic, and a similar number from Glasgow-based UX specialists Chunk Digital. Other specialists and consultants were brought in as the needs arose, including Apple and Google engineers - who were consulted to ensure the app would work cross-platform with only one development workflow - child psychologists and experts in AI ethics.

"As much as possible we tried to work as a single team, ensuring that communications were constant and dependencies were tightly managed," said Howard.

Keeping up with the yeets

The app's functionality is simple, intentionally so, but this simplicity is the result of a great deal of research, experimentation and the jettisoning of many a bell and a whistle.

Howard's team encountered a number of tricky challenges as they developed the MVP, the first of which was the lack of any coherent dataset on the way young people speak online with which to train the machine learning models. A core dictionary had to be painstakingly pieced together by analysing young peoples' messages on social media and comparing word frequencies and usage with adult equivalents.

The way new words arrive is absolutely fascinating

And of course, language doesn't stand still. Howard mentions yeet, a word meaning to forcefully throw away, which suddenly became popular after a Vine video of a kid dancing and then another featuring a girl hurling a can of soda while yelling ‘yeet!' went viral. New words can spring up on social media anywhere in the world and become part of the global lexicon within a few short months, sometimes with local variants. Incidentally, the Urban Dictionary shows that yeet peaked in 2019 and is now, presumably, being used ironically by the cool kids. The app's models need to be able to keep up with nuanced changes like these, or it will find itself yeeted.

"The way new words arrive is absolutely fascinating," Howard said. "I went off on a massive track of learning all about this stuff, building glossaries and then taking words that have just been introduced by children and refining the models."

Neutral territory

Then there was the perennial problem of data-driven bias in machine learning models. Own It deploys four models designed to recognise hate, toxicity, emotion sentiment and emotion, which it uses to decide on the appropriateness of a message. But during development the team uncovered a number of areas where it overreacted or made a mistake. For example, a message beginning ‘Men are..' would cause the concerned face and red border to immediately pop up before any qualifying words were added. To tackle this issue, the team replaced certain gender-, religion- and ethnicity-specific terms with ‘neutral' and were surprised at the results.

"That was a really simple trick and we thought this is never going to work, but it worked really well and it was way better than before," Howard said.  

The privacy conundrum

Trust is all-important with an app like this, and Own It adheres strictly to all the principles of Privacy by Design. No information that might identify the user or the device is collected or stored, and all messages are deleted from the keyboard as soon as they have been sent.

Great, but for a machine learning app this presents a problem: how to integrate feedback to improve the models?

For now, this is a semi-manual process using the feedback provided by users, for example a  complaint when a warning has been shown during casual friendly banter, to pinpoint areas for improvement. That information is then used to source better data to cover the problem areas. Differential privacy techniques will eventually solve these problems, but they are not ready for the primetime just yet.

bbc Own It companion app. Own It keyboard. Source: BBC
Own It companion app

3.5 billion into 15 million will go

In line with its privacy-preserving credentials, Own It is completely decentralised. Its functions are fully encapsulated on the device and it does not communicate with the models in the cloud. This design presented another major challenge: the 3.5GB machine learning models were far too large for a smartphone app. So, Howard's team got to work whittling it down to size. Using a variety of methods including FastText compression and shaving off a few model layers they succeeded in shrinking the 3.5GB to 40mb, and with further attention to the data reduced the final ML ensemble still further to a mere 15mb, with accuracy actually improved by making the machine learning less generic more specific.

The result is a package that can be easily updated over the air every few weeks, which is frequent enough even to keep up with the rapid flow of teen memes and text-speak.

We were at the cutting edge 18 months ago

What's next?

NLP is improving at an exponential rate. "We were at the cutting edge 18 months ago" quipped, Howard, adding that the team is constantly reviewing new developments to see how they might be incorporated, including differential privacy techniques - which would allow models to learn from anonymous user data without the danger of de-anonymisation - and federated learning, in which changes in weights and biases are synchronised between on-device and cloud models. The latter would ultimately allow some of the machine learning training to be performed on the smartphone rather than in the cloud, something Google is already doing with Gboard: "It's really interesting and Google are nailing it now," said Howard.

Another area Howard is keeping an eye on is synthetic data, deploying text-generating neural networks like OpenAI's GPT-2 and the latest GPT-3, which is currently making headlines for its ability to generate meaningful and contextually appropriate sentences with only very small example sets to work from. Models like these could be used to fill in gaps in the data, currently a manual process. "If you give these models some of our phrases they churn out terms which are like them. Experimentally we've found they're really good."

As well as rounding out the dataset, such models can also be used to check for their own biases too.

Future iterations will be informed by the results of ongoing efficacy tests by Manchester University which are expected to conclude in the next few months. The team is also developing an efficacy framework to gauge the effectiveness of Own It in helping children make the most of what the web has to offer while avoiding the pitfalls.

"It's about finding out which interventions are most effective and how best to present them," Howard said. 

Jon Howard will be speaking at Computing's IT Leaders Festival 2020 - register today!

  • Tweet  
  • Facebook  
  • LinkedIn  
  • Send to  
  • Topics
  • Developer
  • Software
  • Mobile
  • BBC
  • Apple
  • Google
  • NSPCC
  • Chunk Digital
  • ai
  • machine learning
  • Artificial Intelligence

More on Developer

Poor-quality software cost the US economy $2.1 trillion last year
Poor-quality software cost the US economy $2.1 trillion last year

Operational software failure due to unpatched bugs was the leading contributor to the wasted $2.1 trillion

  • Software
  • 11 January 2021
DevSecOps: the art of bringing security into the fold

QA and testers need to be first-class members of the team

  • DevOps
  • 05 January 2021
GitHub reinstates popular YouTube downloader three weeks after its takedown over copyright complaint
GitHub reinstates popular YouTube downloader three weeks after its takedown over copyright complaint

Following intervention by the Electronic Frontier Foundation, GitHub changes its mind about Youtube-dll

  • Software
  • 17 November 2020
DeepMind open-sources Lab2D to support creation of 2D environments for AI and machine learning
DeepMind open-sources Lab2D to support creation of 2D environments for AI and machine learning

The system aims to help researchers understand the influence of environments in multi-agent reinforcement learning

  • Software
  • 17 November 2020
New AI model listens to cough sounds to reveal if a person has coronavirus
New AI model can detect coronavirus by listening to a cough

The AI uses four biomarkers to decide if the person is infected

  • Software
  • 04 November 2020
blog comments powered by Disqus
Back to Top

Most read

Women in IT: don't get stuck in 'glue work' if you want to get on
Women in IT: don't get stuck in 'glue work' if you want to get on
Russian malware found on government-issued laptops for home schooling
Russian malware found on government-issued laptops for home schooling
New SolarWinds hack victims emerging every day, as Malwarebytes goes public on breach
New SolarWinds hack victims emerging every day, as Malwarebytes goes public on breach
Google locks out another top AI ethicist
Google locks out another top AI ethicist
BT faces possible £500m claim for overcharging landline-only customers
BT faces possible £500m claim for overcharging landline-only customers
  • Contact
  • Delta
  • Marketing solutions
  • Enterprise IT Events
  • Incisive Media
  • Terms & conditions
  • Policies
  • Careers
  • Twitter
  • LinkedIn
  • Newsletters
  • Facebook
  • YouTube

im_logo

© Incisive Business Media (IP) Limited, Published by Incisive Business Media Limited, New London House, 172 Drury Lane, London WC2B 5QR, registered in England and Wales with company registration numbers 09177174 & 09178013

Digital publisher of the year
Digital publisher of the year 2010, 2013, 2016 & 2017
Loading