Overseas 'Content Farms' Using AI to Spread UK Political Deepfakes
Overseas "content farms" are employing Artificial Intelligence (AI) to generate social media posts concerning UK politics, according to expert warnings. Technology company Meta took action by removing several Facebook pages based in Vietnam following a BBC Wales investigation that revealed these pages were disseminating fake news.
The alert, issued by Prof Martin Innes of Cardiff University, coincides with the Electoral Commission's efforts to develop software aimed at detecting and countering deepfakes ahead of the Welsh and Scottish parliamentary elections scheduled for May.
BBC Wales also uncovered AI-generated videos shared by pages in Wales that falsely portrayed Welsh politicians in compromising scenarios, such as endorsing political rivals or engaging in inappropriate behavior like kissing colleagues.
Multiple Welsh politicians have reported their experiences as victims of deepfakes to the BBC.
"I don't think you'll find a politician who hasn't had this done to them... to say it out loud makes me feel quite sad," said Labour MP Alex Davies-Jones.
What is a Deepfake?
Deepfakes are digitally altered videos, images, or audio clips manipulated to make fabricated content appear authentic. While some deepfakes are harmless or humorous, others have the potential to damage reputations. The advent of text-to-image AI tools has made creating such content easier than ever.
BBC Wales identified numerous Facebook pages publishing fake news about UK politicians, often accompanied by AI-generated images. These pages typically had thousands of followers and frequently posted identical or very similar content. Many bore names suggesting they were UK-based news outlets; however, Facebook's transparency feature revealed that nearly all were operated from Vietnam.
Although some content included genuine news stories, a significant portion was demonstrably false. Meta removed some of these pages after being contacted by the BBC, but new pages continued to appear almost daily throughout the investigation.
The pages often depicted multiple politicians, including Nigel Farage, Boris Johnson, Rishi Sunak, and Zia Yusuf, in fabricated scenarios—for example, storming out of a BBC interview following heated arguments with Laura Kuenssberg.
There is no indication that these pages are affiliated with or operated on behalf of any political party.
Prof Martin Innes, director of Cardiff University's Crime and Security Research Institute, described these pages as "content farms" designed to go viral. While they may generate revenue through Facebook's monetisation program, it is unclear whether each page is monetised.
Facebook flagged some stories with warnings indicating they had been debunked by third-party fact-checkers, such as Full Fact's refutation of a claim that Reform UK leader Nigel Farage was hospitalized. However, similar stories without warnings were also found.
Many stories appeared intended to attract attention from Nigel Farage's supporters, though not all were flattering. Some falsely depicted him adopting dogs, donating personal wealth to charity, or welcoming a new baby, while others showed him being arrested, accompanied by AI-generated images of him in handcuffs.
The pages also targeted other politicians, including Sir Keir Starmer, with false claims that he had collapsed on stage, been sued over "election rigging," or removed as Prime Minister.
Not only politicians but various celebrities were also inserted into fabricated scenarios.
Innes stated that these pages are "driven by a profit imperative" and will "do anything where they think they can get eyeballs on their content if it will make them money."
Some posts received significant engagement, including likes, comments, and shares, while others had minimal interaction despite the pages' large follower counts.
It is uncertain whether all followers are genuine, as page operators often employ bots to "spoof the algorithm" and increase content visibility.
The comments sections revealed mixed reactions; some users expressed skepticism or frustration with the fake stories, while others appeared to accept them as true.
After the BBC presented these findings to Meta, the company removed several pages and reiterated its policy against "inauthentic" accounts or pages on its platforms.
Potential Impact of Deepfakes on the Welsh Election
Devolved elections are scheduled in Wales and Scotland on 7 May, coinciding with local elections in parts of England.
Warnings of a so-called 'deepfake election' in the UK have circulated previously, with varying opinions on whether such influence has materialized.
The Alan Turing Institute, a national center for data science and AI, reported "no evidence" that AI-enabled deepfakes or disinformation had a meaningful impact on the 2024 general election results.
However, nearly two years later, the "barriers to entry" for creating such content have decreased, according to Innes.
"Where previously doing this kind of image [or] video manipulation would have required quite a lot of computing power and technical knowledge, you don't need that anymore."
This development could lead to a "trickle-down effect" where manipulated videos begin to influence devolved political processes.
The risk is sufficiently significant for the Electoral Commission to intervene. The independent body responsible for overseeing UK elections is collaborating with the Home Office to develop software capable of identifying, tracking, and reporting deepfakes.
Its chief executive, Vijay Rangarajan, stated the initiative would "help voters identify misinformation during election campaigns and reduce activity which negatively impacts voters' confidence in the conduct of campaigners."
Following this announcement, the BBC identified several AI-generated fake videos of Welsh politicians created in recent weeks, collectively amassing over 200,000 views.
These included a fabricated video of Prime Minister Keir Starmer and Wales' First Minister Eluned Morgan kissing, and another showing Plaid Cymru leader Rhun ap Iorwerth shouting "I love Reform."
The Facebook page responsible for these videos described them as "obvious" satire, trusting their audience to apply "common sense," and indicated the videos were fake in their captions.
Another video, shared by an anti-Reform page, depicted Nigel Farage with a Welsh flag backdrop. In the clip, he states: "Britain is great; I don't see the point of having four different sporting teams and four different parliaments." There is no record of Farage making this statement, and the video exhibited distortions consistent with AI generation.
The page behind this video has been contacted for comment.
These examples were characterized by Innes as "shallowfakes" rather than "deepfakes," meaning they were less realistic and produced with less sophisticated software.
However, identifying fake content is becoming increasingly difficult, with Innes' team sometimes spending an entire day analyzing more advanced examples.
He noted that while the Electoral Commission's efforts might aid in understanding deepfake activity after elections, they would not prevent deepfakes from affecting the electoral process.
Davies-Jones, representing Pontypridd, Rhondda Cynon Taf, reported experiencing "horrific" deepfakes, including some depicting her in "crude sexual positions" wearing underwear.
She added speaking out about non-consensual sexual deepfakes could be "embarrassing" and lead to "increased targeting"—and their existence was "no good for any healthy democracy."
Llŷr Powell, Reform UK's candidate in the Caerphilly by-election, stated his team discovered AI-generated videos misrepresenting him and colleagues by attributing false statements about his policies.
Powell said he was "in favour of free speech" and freedom of expression but acknowledged a problem when AI use causes "some people believing that it was fact and not parody."
He added: "We don't want to sacrifice the freedoms we've got in this country, but at the same time the public have a right to know what information they're being fed and the tool is being misused here just like anything can be misused."
Janet Finch-Saunders, Conservative MS for Aberconwy in north Wales, expressed feeling "sick" upon seeing a photo of herself edited "in a very disgusting way" to create an explicit deepfake.
She said the fakes could be "very misleading" to people who were "less tech-savvy" and "it is quite worrying because there's far more capability now."
"When you're in politics... or whatever your line of work, you should not be attacked personally. And certainly not AI used to belittle you."
Baroness Carmen Smith, a Plaid Cymru peer, emphasized the threat deepfakes pose to democracy and individuals alike.
"Deepfakes seriously are a threat to democracy, but they are also a threat to many, many people whether that is your sister, your friend, your colleague.
"We have to keep up to date with what developments are being made with AI tools... it's something that we do need to address and prioritise now."
Rachel Millward, deputy leader of the Green Party of England and Wales, reported being targeted by a "misogynistic deepfake" and urged the UK government to regulate the AI sector to prevent such content from "undermining the legitimacy of future elections."
A Welsh Liberal Democrat spokesperson described deepfakes as a "threat to the safety of politicians across the political spectrum" and called for "cross-nation action."
The UK government's department for science, innovation and technology acknowledged the "potential for deepfakes to sow division, spread false information, and influence public opinion" as "well recognised."
It stated that social media platforms must "proactively tackle illegal fraudulent content" under the Online Safety Act or "face enforcement action."
BBC Wales Your Voice Live: Ask the Leader
BBC Wales Your Voice Live: Ask the Leader invites the public to attend and pose questions to the leaders of Wales' political parties.
to submit questions for the leaders of the Welsh Conservatives, Plaid Cymru, and the Welsh Liberal Democrats in Haverfordwest on 8 April.
to submit questions for the leaders of Welsh Labour, Reform UK, and the Wales Green Party in Llandudno on 15 April.
