Social media companies have been ordered to have emergency measures in place to stop illegal content going viral, as regulators battle to stop the type of misinformation spiral that circulated after the 2024 summer riots.
Sites such as X, formerly Twitter, and TikTok will have to have a “crisis protocol” in place to intervene when the sharing of dangerous content begins to rise.
Under the measures to be implemented by Ofcom, the UK’s tech regulator, online platforms will also need to reserve a dedicated line of communication channel through which the police can contact them in a crisis.
It follows concerns at the top of government over the speed with which misinformation spreads at pivotal moments. Ofcom’s announcement also follows the outbreak of rioting in Southampton over the police response to the fatal stabbing of Henry Nowak.
The idea of forcing social media platforms to have emergency measures ready to stop illegal viral content was suggested by the Commons science, innovation and technology committee.
It made the recommendation following an investigation it launched after the riots of summer 2024, when misinformation spread quickly after the murder of three girls at a dance class in Southport.
Its report found that “misleading and hateful messaging proliferated rapidly online, amplified by the recommendation algorithms of social media companies”.
Ofcom said it would define a crisis as an “extraordinary situation in which there is a serious threat to public safety in the United Kingdom”, which is highly likely to have “resulted from a significant increase in relevant content”.
“During a crisis, certain kinds of illegal content and/or content harmful to children can spread rapidly online,” it said. “In some cases, this can create a significant risk to the public safety within the United Kingdom.
“For example, evidence from previous crisis events illustrate how the perpetrators use online services to carry out illegal activity, such as inciting racial or religious hatred, making threats or inciting violence. This can lead to an increase in the amount of illegal content circulating online but also manifest in violence in the real world.
“Such crises are exceptional, and this means that online service providers’ usual content moderation systems and processes may not be sufficient in such circumstances.”
The bar for the implementation of such protocols is high, dealing with outright illegal content. The new measures will come into force after being approved by parliament.
The move is a sign of the urgency with which ministers and the regulator are keen to tackle the speed with which misinformation can travel in the often confusing climate of a crisis.
Experts examining the flow of dangerous viral content have also pointed to other measures as being successful in countering fast-moving misinformation. Analysis by the Centre for Emerging Technology and Security (Cetas) found that “debunking” efforts involving police forces, community leaders and local councillors could be effective in alerting communities to false information.
Its research found that a swift and coordinated response to an incident in Liverpool, in which a vehicle hit spectators at Liverpool FC’s league title parade in May last year, helped reduce the risk of misinformation running rampant online.
