Business models and opaque algorithms

According to the SITC, social media companies have “often enabled or even encouraged” the viral spread of misinformation – and may have profited from it – as a result of their advertising and engagement-based business models.

“The advertisement-based business models of most social media companies mean that they promote engaging content, often regardless of its safety or authenticity,” MPs wrote. “This spills out across the entire internet, via the opaque, under-regulated digital advertising market, incentivising the creation of content that will perform well on social media.”

They added that while major tech companies told the committee there are no incentives to allow harmful content on their platforms, as it can damage the brand and repel advertisers, “policymaking in this space has lacked a full evidence base” because the inner workings of social media recommendation algorithms are not disclosed by the firms.

“We asked several tech companies to provide high-level representations of their recommendation algorithms to the committee, but they did not,” they said, adding that this “shortfall in transparency” makes it difficult to establish clear causal links between specific recommendations and harms.

“The technology used by social media companies should be transparent, explainable and accessible to public authorities,” they said.

The SITC added that the government should create measures to compel social media platforms to embed tools in their systems that can identify and algorithmically deprioritise fact-checked misleading content, or content that cites unreliable sources, where it has the potential to cause significant harm.

“It is vital that these measures do not censor legal free expression, but apply justified and proportionate restrictions to the spread of information to protect national security, public safety or health, or prevent disorder or crime,” said MPs.

On tackling the underlying business models that incentivise misinformation, MPs said there is a regulatory gap around digital advertising, as the focus is currently on harmful advertising content rather than “the monetisation of harmful content through advertising”.

“The government should create a new arms-length body – not funded by industry – to regulate and scrutinise the process of digital advertising, covering the complex and opaque automated supply chain that allows for the monetisation of harmful and misleading content,” they added. “Or, at the least, the government should extend Ofcom’s powers to explicitly cover this form of harm, and regulate based on the principle of preventing the spread of harmful or misleading content through any digital means, rather than limiting itself to specific technologies or sectors.”

While generative artificial intelligence (GenAI) only played a marginal role in the spread of misinformation before the Southport riots, the SITC expressed concern about the role it could play in a “future, similar crisis”.

They said GenAI’s “low cost, wide availability and rapid advances means that large volumes of convincing deceptive content can increasingly be created at scale”.

It said the government should therefore pass legislation that covers GenAI platforms, in line with other online services that pose a high risk of producing or spreading illegal or harmful content.

“This legislation should require generative AI platforms to: provide risk assessments to Ofcom on the risks associated with different prompts and outputs, including how far they can create or spread illegal, harmful or misleading content; explain to Ofcom how the model curates content, responds to sensitive topics and what guardrails are in place to prevent content that is illegal or harmful to children; implement user safeguards such as feedback, complaints and output flagging; and prevent children from accessing inappropriate or harmful outputs.”

They added that all AI-generated content should be automatically labelled as such “with metadata and visible watermarks that cannot be removed”.