Generative AI and The Explosion of Misinformation

Charting a Course for Information Integrity

Dec 01, 2023

From viral hoaxes to manipulated narratives, the explosion of misinformation has cast a shadow over the credibility of our digital landscape. Simultaneously, Generative AI—machines designed to autonomously create multimodal content, ranging from art to text— emerges as a disruptive force. It promises innovation while still being capable of deception. As the forces of rising misinformation and generative AI collide, a new concern emerges: Will Generative AI become an unwitting accomplice in the proliferation of misinformation? What effect will this have on already threatened industries like journalism? Zooming out even further, what consequences does it have on knowledge work?

Generative AI is uniquely positioned to disperse misinformation

As we explore the relationship between Generative AI and the proliferation of misinformation, it becomes imperative to delve deeper into how this technology — with its unique challenges and cost-efficient capabilities — can be harnessed as a potent tool for spreading deceptive narratives. Generative AI is uniquely positioned to contribute to the explosion of misinformation. Unlike conventional methods of spreading misinformation, Generative AI introduces a unique set of challenges.

In a paper titled ‘Combating Misinformation in the Era of Generative AI Models’ the authors (faculty from NUS) present three key ways in which generative AI is uniquely positioned to disperse misinformation.

First, Generative AI marks an evolution — it’s better disguised to spread misinformation. Malevolent actors can easily generate content across modalities (text, speech, video, images) that threatens the integrity of information we consume. What’s worse is that existing solutions to track this new form of misinformation are lacking.

“Merging of multiple modalities by generative models presents a challenge in combating fake information, making it difficult for both users and algorithms to differentiate between truth and fabrication”
— Danni Xu, Shaojing Fan & Mohan Kankanhalli

Second, generative AI can be used to generate hyper-personalized and persuasive misinformation. While this hyper-personalization has its benefits, other information integrity experts have also argued, this can have a siloing effect. Malicious actors can use generative AI to create and reinforce echo chambers — making it even more challenging to combat misinformation.

Finally, generative AI amplifies the scale at which misinformation can be dispersed. Generative AI tools are notoriously known for their hallucination problem — machines conjuring convincingly realistic yet entirely fabricated content. Compounding this is the low-cost nature of developing such misinformation and the user-friendly nature of most generative AI interfaces (like ChatGPT). The effect this has is lowered barriers of entry for malevolent actors.

The ramifications of misinformation and disinformation already represent a threat to fundamental institutions, but generative AI applications exacerbate this threat. Generative AI applications can be employed in a journalistic context to mass produce articles, deepfake interviews and essentially do whatever it takes to blur fact from fiction. For an industry with already-dwindling credibility, Generative AI poses an existential threat to journalism. Scientific communication faces a similar peril when the integrity of information is undermined by AI-generated research and discourse. Keeping in line in information integrity, Generative AI can be leveraged in election contexts — manipulating opinions and eroding trust in the foundations of democracy.

As consumers of information — be it news for day-to-day or voting purposes, or technical/scientific — it’s of utmost importance that it’s truthful. It shouldn’t take much convincing that a misinformed society is fundamentally a bad thing.

How do we Respond?

The intersection of Generative AI and misinformation necessitates a collective call to action, particularly for entrepreneurs. They have a unique chance to mitigate risks — build technology that fosters a more resilient and trustworthy information ecosystem. Experts, like Nina Schick (who specializes in generative AI) have already outlined a few suggestions. Since flagging artifacts generated by AI (images, deep fakes, text etc) is so challenging, she suggests the need for better information verification tools. Emphasis was placed on tools that establish content provenance and transparency. This is just a small step towards stronger information integrity.

Another suggestion was to improve cooperation between tech companies, researchers and policy-makers to create legal frameworks that govern and appropriately punish misinformation. This is a heavy collective responsibility. It begins with readers staying informed, being advocates (or better, practitioners) of responsible AI development and actively supporting initiatives that address these challenges.

Innovation with AI can be aligned with societal well-being but that demands that we embrace this collective responsibility. Ultimately, algorithms are supposed to serve society but when algorithms begin to weaponize data and narrative we need to proactively guard against the erosion of truth and trust in our information landscape.

Entrepreneurship in the Age of AI

Discussion about this post

Ready for more?