Achieving a more transparent and less manipulative online media may well be the defining political battle of the 21st century.
There was a time when the internet was seen as an unequivocal force for social good. It propelled progressive social movements from Black Lives Matter to the Arab Spring; it set information free and flew the flag of democracy worldwide. But today, democracy is in retreat and the internet’s role as driver is palpably clear. From fake news bots to misinformation to conspiracy theories, social media has commandeered mindsets, evoking the sense of a dark force that must be countered by authoritarian, top-down controls.
This paradox — that the internet is both savior and executioner of democracy — can be understood through the lenses of classical economics and cognitive science. In traditional markets, firms manufacture goods, such as cars or toasters, that satisfy consumers’ preferences. Markets on social media and the internet are radically different because the platforms exist to sell information about their users to advertisers, thus serving the needs of advertisers rather than consumers. On social media and parts of the internet, users “pay” for free services by relinquishing their data to unknown third parties who then expose them to ads targeting their preferences and personal attributes. In what Harvard social psychologist Shoshana Zuboff calls “surveillance capitalism,” the platforms are incentivized to align their interests with advertisers, often at the expense of users’ interests or even their well-being.
This economic model has driven online and social media platforms (however unwittingly) to exploit the cognitive limitations and vulnerabilities of their users. For instance, human attention has adapted to focus on cues that signal emotion or surprise. Paying attention to emotionally charged or surprising information makes sense in most social and uncertain environments and was critical within the close-knit groups in which early humans lived. In this way, information about the surrounding world and social partners could be quickly updated and acted on.
But when the interests of the platform do not align with the interests of the user, these strategies become maladaptive. Platforms know how to capitalize on this: To maximize advertising revenue, they present users with content that captures their attention and keeps them engaged. For example, YouTube’s recommendations amplify increasingly sensational content with the goal of keeping people’s eyes on the screen. A study by Mozilla researchers confirms that YouTube not only hosts but actively recommends videos that violate its own policies concerning political and medical misinformation, hate speech, and inappropriate content.
In the same vein, our attention online is more effectively captured by news that is either predominantly negative or awe inspiring. Misinformation is particularly likely to provoke outrage, and fake news headlines are designed to be substantially more negative than real news headlines. In pursuit of our attention, digital platforms have become paved with misinformation, particularly the kind that feeds outrage and anger. Following recent revelations by a whistle-blower, we now know that Facebook’s newsfeed curation algorithm gave content eliciting anger five times as much weight as content evoking happiness. (Presumably because of the revelations, the algorithm was changed.) We also know that political parties in Europe began running more negative ads because they were favored by Facebook’s algorithm.
Besides selecting information on the basis of its personalized relevance, algorithms can also filter out information considered harmful or illegal, for instance by automatically removing hate speech and violent content. But until recently, these algorithms went only so far. As Evelyn Douek, a senior research fellow at the Knight First Amendment Institute at Columbia University, points out, before the pandemic, most platforms (including Facebook, Google, and Twitter) erred on the side of protecting free speech and rejected a role, as Mark Zuckerberg put it in a personal Facebook post, of being “arbiters of truth.” But during the pandemic, these same platforms took a more interventionist approach to false information and vowed to remove or limit Covid-19 misinformation and conspiracy theories. Here, too, the platforms relied on automated tools to remove content without human review.
Even though the majority of content decisions are done by algorithms, humans still design the rules the tools rely upon, and humans have to manage their ambiguities: Should algorithms remove false information about climate change, for instance, or just about Covid-19? This kind of content moderation inevitably means that human decision makers are weighing values. It requires balancing a defense of free speech and individual rights with safeguarding other interests of society, something social media companies have neither the mandate nor the competence to achieve.
None of this is transparent to consumers, because internet and social media platforms lack the basic signals that characterize conventional commercial transactions. When people buy a car, they know they are buying a car. If that car fails to meet their expectations, consumers have a clear signal of the damage done because they no longer have money in their pocket. When people use social media, by contrast, they are not always aware of being the passive subjects of commercial transactions between the platform and advertisers involving their own personal data. And if users experience has adverse consequences — such as increased stress or declining mental health — it is difficult to link those consequences to social media use. The link becomes even more difficult to establish when social media facilitates political extremism or polarization.
Users are also often unaware of how their news feed on social media is curated. Estimates of the share of users who do not know that algorithms shape their newsfeed range from 27% to 62%. Even people who are aware of algorithmic curation tend not to have an accurate understanding of what that involves. A Pew Research paper published in 2019 found that 74% of Americans did not know that Facebook maintained data about their interests and traits. At the same time, people tend to object to collection of sensitive information and data for the purposes of personalization and do not approve of personalized political campaigning.
They are often unaware that the information they consume and produce is curated by algorithms. And hardly anyone understands that algorithms will present them with information that is curated to provoke outrage or anger, attributes that fit hand in glove with political misinformation.
People cannot be held responsible for their lack of awareness. They were neither consulted on the design of online architectures nor considered as partners in the construction of the rules of online governance.
What can be done to shift this balance of power and to make the online world a better place?
Google executives have referred to the internet and its applications as “the world’s largest ungoverned space,” unbound by terrestrial laws. This view is no longer tenable. Most democratic governments now recognize the need to protect their citizens and democratic institutions online.
Protecting citizens from manipulation and misinformation, and protecting democracy itself, requires a redesign of the current online “attention economy” that has misaligned the interests of platforms and consumers. The redesign must restore the signals that are available to consumers and the public in conventional markets: users need to know what platforms do and what they know, and society must have the tools to judge whether platforms act fairly and in the public interest. Where necessary, regulation must ensure fairness.
Four basic steps are required:
Several legislative proposals in Europe suggest a way forward, but it remains to be seen whether any of these laws will be passed. There is considerable public and political skepticism about regulations in general and about governments stepping in to regulate social media content in particular. This skepticism is at least partially justified because paternalistic interventions may, if done improperly, result in censorship. The Chinese government’s censorship of internet content is a case in point. During the pandemic, some authoritarian states, such as Egypt, introduced “fake news laws” to justify repressive policies, stifling opposition and further infringing on freedom of the press. In March 2022, the Russian parliament approved jail terms of up to 15 years for sharing “fake” (as in contradicting official government position) information about the war against Ukraine, causing many foreign and local journalists and news organizations to limit their coverage of the invasion or to withdraw from the country entirely.
In liberal democracies, regulations must not only be proportionate to the threat of harmful misinformation but also respectful of fundamental human rights. Fears of authoritarian government control must be weighed against the dangers of the status quo. It may feel paternalistic for a government to mandate that platform algorithms must not radicalize people into bubbles of extremism. But it’s also paternalistic for Facebook to weight anger-evoking content five times more than content that makes people happy, and it is far more paternalistic to do so in secret.
The best solution lies in shifting control of social media from unaccountable corporations to democratic agencies that operate openly, under public oversight. There’s no shortage of proposals for how this might work. For example, complaints from the public could be investigated. Settings could preserve user privacy instead of waiving it as the default.
In addition to guiding regulation, tools from the behavioral and cognitive sciences can help balance freedom and safety for the public good. One approach is to research the design of digital architectures that more effectively promote both accuracy and civility of online conversation. Another is to develop a digital literacy tool kit aimed at boosting users’ awareness and competence in navigating the challenges of online environments.
Achieving a more transparent and less manipulative media may well be the defining political battle of the 21st century.
By Stephan Lewandowsky and Anastasia Kozyreva
Stephan Lewandowsky is a cognitive scientist at the University of Bristol in the U.K. Anastasia Kozyreva is a philosopher and a cognitive scientist working on cognitive and ethical implications of digital technologies and artificial intelligence on society. at the Max Planck Institute for Human Development in Berlin. This piece was originally published by OpenMind magazine and is being republished under a Creative Commons license.