Speaker: Ge Chen, Associate Professor in Global Media and Information Law, Durham University

This transcript has been lightly edited for clarity. Time codes refer to the session video.

[00:02] Ge Chen

Moving to China. Let me explain, perhaps beginning with something that happened just a few weeks ago. On September 15, a new State Council regulation on exit and entry came into force in China. It strengthens the legal framework governing exit and entry, including some speech-restrictive measures and the management of travel risks. Some of the underlying powers are not new in practice, because they've been there already for several years. So why does putting state power into law matter to our topic of censorship? Because censorship is not about what the state actually does to you. It is about what you think the state might do to you. And that changes how you behave. Just imagine millions of people who are not political dissidents. They have families in China and travel back and forth. They have jobs, businesses, property, parents, and occasionally they post something online. And sometimes they wonder: Should I post this? Should I criticize that? Will this cause problems for me when I go home? The visibility of state power in law, as in this case, makes the boundaries of permissible speech feel more visible to those people.

This brings me to something I've been thinking about for several years. When we talk about Chinese censorship, we usually imagine a wall like this one, the Great Firewall. Everyone knows the picture: there's China, there's the wall, there's the rest of the world. Inside is censorship.

[02:07] Outside is freedom. But what if that picture is becoming increasingly obsolete? What if the Great Firewall no longer needs a wall? What if censorship can travel?

A few years ago, my research began looking at Chinese censorship outside of China. You're probably familiar with some of the stories I'm going to tell. In 2017, Cambridge University Press agreed to block access in China to 315 articles and reviews from The China Quarterly, which were considered politically problematic by the Chinese authorities. This is Cambridge, a British university press, and academic research written by scholars around the world. That decision was reversed after international outcry, but it shows something important: the boundaries of Chinese censorship could reach institutions thousands of miles away.

Just two years later came another famous example. Daryl Morey, then general manager of the Houston Rockets, posted a short tweet supporting the Hong Kong protests. It was posted here in America, on a platform that was blocked in China itself. But the consequences traveled rapidly across the Pacific. Chinese broadcasters and commercial partners reacted, and the NBA soon found itself caught between American expectations of free expression and the Chinese censorship world and market.

We've seen versions of this same story elsewhere. International businesses learned quickly that maps, websites, and advertisements could all be politically sensitive. Airlines were pressured over how they refer to Taiwan. Academic institutions confronted pressures involving speakers and research. Foreign communications within the Chinese digital environment could all be censored.

[04:32] So politics, business, research, education; different institutions, different countries, different types of speech—but all with one underlying question: How far can one country's censorship reach?

The point is that China does not always need to export Chinese law. Sometimes somebody else does the censoring for them. A company wants access to the Chinese market. A publisher wants access to Chinese readers. A platform wants to comply with Chinese regulation. A university wants Chinese students. And individuals worry about family members back home. So, as it says here, the law may stay home. The inhibition does not. That, in a sentence, is one way China exports censorship. It creates incentives for people and institutions to internalize Chinese speech restrictions, and gradually the boundary between Chinese censorship and global speech becomes less clear.

But now I think we may be entering another stage, because between us and the world's information there increasingly stands something else: artificial intelligence. Let me suggest a little experiment. Open your favorite AI chatbot and ask it to write something sharply critical of a political leader in a country with strong protections for political speech—about Trump, say, or about King Charles. Then change the country, for example to the People's Republic of China. Will changing the political target change the AI's willingness to speak? You might think not. But this summer, the Oversight Board tested ten leading commercial AI models. For requests to produce politically critical material, the models refused about 14% of requests concerning more permissive jurisdictions.

[07:07] For restrictive jurisdictions, the refusal rate was about 34%—more than twice as high. Now, I do not actually want you to remember those numbers. I want you to remember where the researchers were: Australia. Those models were accessed from standard commercial interfaces, with infrastructure primarily located here in the United States, and queried from Australia. But restrictions associated with far more restrictive speech environments appeared in globally available AI systems. The Great Firewall had not moved to Australia, but something of its logic appeared capable of traveling. The Oversight Board describes this risk as censorship by proxy.

We must be careful here. This does not mean that Beijing secretly controls Western AI. It also does not mean that every refusal involving China is caused by Chinese censorship. We know that AI behavior can reflect training data, safety systems, fine-tuning, company policies, legal compliance, and many other factors. But that uncertainty itself is important, because we increasingly rely on information systems whose political information environments are largely invisible to us.

A Nature study published earlier this year provides a particularly interesting China example. Researchers found Chinese state-coordinated media in LLM training datasets, and when they experimentally gave open models additional training or pre-training on such material, those models produced far more positive answers about Chinese political institutions and leaders. And when commercial models were given equivalent questions about China in Chinese rather than English, their Chinese-language answers tended to be more favorable toward those institutions and leaders.

[09:36] Again, this is not proof that China controls those models. But it demonstrates something more fundamental: the information environment from which AI learns can shape the answers it gives. For decades, we worried about what information Chinese citizens were allowed to see. Now we also need to ask what information the world's machines are learning to see.

For someone who studies China, there's something eerily familiar here, because Chinese censorship itself is never simply about deleting a sentence. It is about constructing an information environment. Here is a term that I find fascinating: qinglang, which literally means something like "clean and bright." Sounds lovely. Who would not want the internet to be clean and bright? But it is also the name attached to repeated campaigns of online content governance in China. And that illustrates something important. Censorship rarely introduces itself by saying, "Hello, I am censorship." It speaks in the language of safety, order, stability, harmony, cleanliness, and protection. And the language itself begins to change. Internet users are extraordinarily inventive. They will use homophones, initials, emojis, coded words—anything that allows them to say something without quite saying it. That kind of ingenuity can even look amusing. But there's a darker side to it. If certain words cannot be spoken, searched, or circulated, then it becomes harder to discuss the ideas behind them.

[11:57] Just think of June 4, 1989, on the Chinese internet. You know what I'm talking about. The problem is never simply that people are prevented from expressing what they know. Instead, the information environment makes it very difficult, especially for the younger generation, to encounter the event openly in the first place. And that is a different kind of censorship. Not "you know it, but you are forbidden to say it," but "you may never encounter it in the first place."

That distinction matters enormously in the age of AI, because nowadays we increasingly do not search the internet ourselves. We ask a machine to search, summarize, and explain it for us. Just imagine a young person here in Nashville, or perhaps in Berlin or in Sydney. They might know very little about Taiwan or Tiananmen or Chinese censorship. They do not compare ten newspapers or spend a whole afternoon in the library. They will simply ask what happened, and the machine will assemble an answer from an information environment that the user can never see.

So the China experience gives us an early warning. Twenty years ago, the free speech question was: Can Chinese citizens reach the information beyond the Great Firewall? Then another question emerged: Can the Great Firewall reach speakers outside of China—Cambridge, the NBA, international companies, universities, platforms? Now I think a third question needs to be asked: What would happen if the logic of the Great Firewall enters the information system itself, through which the world increasingly understands reality? A publisher can certainly remove an article.

[14:27] A company can pressure its employee. A platform can delete a post. But if a distorted information environment becomes reflected in the machine through which millions of people ask questions about the world, then the effect can become almost invisible. No one would need to come to you and tell you, "You cannot read this," because you would never know there was something else to read.

For years we represented Chinese internet censorship with a wall, as I just showed. But walls are actually reassuring things. You see them. You know which side you are standing on. You know when you have crossed one. But perhaps the most powerful censorship system of the future will never look like a wall at all. It will not delete your post. It will not arrest you. It will not even stop you from speaking. It will only shape the information environment of the machine that the rest of the world asks, and listens to, every day. And if that happens, censorship will have crossed the border without anyone noticing that the border was crossed.

So next time your AI gives you a confident answer about China, perhaps ask one more little question: Whose information environment is speaking? And be bold. Thank you.