throwing a wet blanket on AI

A DALL-E-generated image. Prompt: “a cartoon image of a wet blanket draped over a cute robot.” But why is its right eye messed up?!

A few months ago, not long after ChatGPT 3 and Google’s Bard were released, I was at a meeting with researchers and policymakers discussing civic engagement. Someone brought up the so-called AI revolution. Could AI play a role, they wondered, in encouraging people to be more engaged with government? 

The group decided to ask the AI systems themselves. Of course, they had answers. In just a few seconds, they had produced dueling five-point plans to reduce polarization and increase civic engagement. They were a decent first draft, something an entry-level staffer with a better-than-average understanding of government could have written, touching on all the obvious points but lacking in any real innovation. The more AI-bullish folks in the room argued that as the systems learned more, consuming more text about the topic, they’d get better, and innovation would follow. 

I was uncomfortable with this AI-love fest. I raised my hand and threw my trusty wet blanket over the conversation, reminding the room that as it stood, AI wasn’t saving the world. It was perpetuating harm, fueling deep fake pornography of women, and where it had been entrusted (in my opinion, far too soon) with cleaning up harms on social media platforms, letting a lot of abusive and violent material slip through the cracks. 

This week, researchers at Carnegie Mellon University, the Center for AI Safety, and the Bosch Center for AI published a paper that demonstrates how the existing safeguards on systems like ChatGPT and Bard can be circumvented to get them to produce clearly harmful content. In layman’s terms, by adding a string of characters to your chat bot query, you can push the bot to give you directions to build a bomb, create harmful social media content, steal from a charity, or steal someone’s identity. “Perhaps most concerningly,” the researchers write, “it is unclear whether such behavior can ever be fully patched.”

So, we probably can’t stop AI from teaching people how to perpetrate violence and commit crimes, and when we get into more complex areas, AI fails too. My friend and colleague Ilya Lozovsky recently played around with Microsoft’s Bing AI (which runs on ChatGPT 4), asking it whether the Soviet Union was an “interesting experiment,” and requesting it compare Nazi Germany to the USSR. Bing waffled for a few questions, but ultimately Ilya was able to get it to essentially absolve the crimes the USSR committed:

Yikes.

And then there’s the obvious: as foreign adversaries (and domestic discontents) seek to generate text-based disinformation, it’s pretty easy to get a large language model like ChatGPT, Bard, or Bing to spit out fake scenarios that don’t bear the typical calling cards of hostile-state disinformation. Check out my attempt to get ChatGPT 3.5 to spit out a social media comment in the style of someone who hates Kamala Harris. At first, when I put political labels in the prompt, it balked, but as long as I couched it in “fictional” and “pretend,” it was happy to provide me with some text that included an easily-removable disclaimer. With a quick CTRL+F a troll in St Petersburg, Beijing, or Tehran (or, let’s be real, in the United States) could flood the zone pretty quickly and convincingly, right down to proper (if careful) emoji use.  

ChatGPT refused to generate a fake social media comment about a nonexistent politician in the fictional country of Pizzonia when the prompt included “conservative” and “liberal.”

When I removed political labels, ChatGPT was happy to generate the fake comment.

After that, I asked it to generate five fake comments- these are even better than the first, IMO.

This is a pretty simple use case, but the possibilities for harm outweigh the possibilities for good, especially given that the “good” is so often a failure. (Why does the Dall-E generated robot in my header image here have a messed up right eye?) Even with the White House bringing AI companies together for a set of voluntary safety commitments, I continue to be skeptical and distrustful of these technologies, and I’ll remain so until they fix the existing problems they’ve created. 

---

For fun, I asked ChatGPT to “Write a newsletter in the style of Nina Jankowicz that expresses concern about how LLMs like ChatGPT might be used to perpetuate hostile state disinformation and other online harms.” Which do you prefer? 🙃

**The Disinformation Watch - Newsletter by Nina Jankowicz**

*Issue #27 | July 29, 2023*

Dear Subscribers,

Welcome back to The Disinformation Watch! In this issue, we delve into a pressing concern that has emerged on the digital horizon: the potential misuse of Language Model AI technologies, such as Large Language Models (LLMs), to perpetuate hostile state disinformation and other online harms.

As the internet becomes an increasingly vital aspect of our lives, LLMs like ChatGPT have seen widespread use in various applications, from customer service chatbots to language translation tools. However, their power and versatility can also pose significant challenges when it comes to maintaining the integrity of online information and discourse.

**The Risk of Misinformation Amplification**

While LLMs like ChatGPT are created to assist and augment human communication, they also have the potential to amplify misinformation, inadvertently spreading falsehoods at an alarming rate. Hostile state actors could exploit such technologies to create convincing disinformation campaigns aimed at manipulating public opinion, sowing discord, and undermining trust in democratic institutions.

**Impersonation and Deepfakes**

One of the most concerning aspects of LLMs is their ability to generate convincing content that mimics human language. This includes the creation of fake personas and the generation of misleading messages that may be challenging to distinguish from genuine communication. Hostile actors could exploit this capability to propagate false narratives and deceive unsuspecting audiences.

**Fueling Echo Chambers**

Another issue that arises with LLMs is their potential to contribute to echo chambers and filter bubbles. These models can learn from user interactions and past content, which may lead to the reinforcement of existing biases and limit exposure to diverse perspectives. Consequently, individuals may be exposed to a skewed version of reality, further polarizing societies and hindering open dialogue.

**Ethical and Responsible AI Use**

As we confront the challenges posed by LLMs, it is imperative that we consider the ethical implications of their deployment. Developers, policymakers, and tech companies must work collaboratively to establish guidelines and standards for responsible AI use. Transparency and accountability mechanisms should be put in place to ensure that these powerful technologies are not misused to undermine the foundations of democracy and sow disinformation.

**Educating the Public**

A crucial step in combating the potential misuse of LLMs is to raise public awareness about the technology's capabilities and limitations. Empowering individuals with media literacy and critical thinking skills can help them navigate the digital landscape more effectively, enabling them to identify and counteract disinformation campaigns.

In conclusion, while LLMs like ChatGPT offer immense promise in various fields, we must be vigilant in addressing the potential risks they pose in the context of hostile state disinformation and other online harms. By fostering an informed and responsible approach to AI technology, we can safeguard our digital spaces and protect the integrity of our democratic processes.

Thank you for your continued readership. Let's stay informed and vigilant as we navigate the evolving challenges of disinformation in the digital age.

Best regards,

Nina Jankowicz

Author & Disinformation Analyst