When to Play Games During a Pandemic

At the beginning of the pandemic, we at Digital Public Square (DPS) saw an opportunity to be helpful - by taking what we have learned about misinformation and gamification in other divisive contexts, and applying it right here in Canada. As the in-person world slowed to a stand-still, and people became reliant on online spaces, misinformation skyrocketed.

Seeing the risk in this, we partnered with the Public Health Agency of Canada (PHAC) to create and distribute an online rapid learning tool, to get good information through to Canadians and help them make sense of a rapidly evolving pandemic.

■ Try 'Know it or Not'

In a modern age where information is flying at us at a rate never before seen, our minds have not yet learned to cope. Our brains continue to lag evolutionarily way behind modern technology: we’re still cognitively better suited for the stone age than for getting our news from a screen.

In our stone age brains, once an opinion is formed, it is very hard to change - which has the unfortunate effect of making otherwise sensible people irrational. Add a layer of personal biases and psychological vulnerabilities - and what results is what makes all of us so susceptible to misinformation.

And while misinformation is nothing new, the rate at which our brains are exposed to it today is faster than what it was yesterday, and the day before that. 

In a space of just nine months between March and November 2021, COVID-19 misinformation cost at least 2,800 lives and $300 million in hospital costs in Canada. A report released by the Council for Canadian Academies (CCA) stresses that these are conservative estimates. Misinformation undermines an individual’s right to be informed, is an existential threat to social cohesion, and can be deadly. So what is to be done about it? How can facts win out in an ever more crowded haystack of misinformation? 

At Digital Public Square (DPS), we see the harms that technology can pose to communities around the world. But we also see the flip side - and recognize that technology has a tremendous power to redress the problems faced by modern society. Based in Toronto, DPS is a not-for-profit organization with a mission to help build healthy communities using good technology. We have worked to solve problems all over the world, partnering with like minded organizations who are also working to peel back the layers of misinformation that surround our time’s most complex issues. 

Spread of COVID-19 and its Misinformation

In March 16th 2020, the Canadian government shared with the country measures to halt the spread of COVID-19, saying that “they are exceptional circumstances calling for exceptional measures.” While we had been tracking COVID-19 misinformation since February, March 2020 saw an explosion of it as online spaces took on an even greater role in our lives. 

Together with our new partner - at first Heritage Canada, and then The Public Health Agency of Canada (PHAC), we saw the need for a space online, outside of misinformation echo chambers, where accurate information on the pandemic could be seen, interacted with, and shared. Thankfully, we weren’t starting from scratch -  we at DPS had previously created an online tool that had proven effective at using gamification to revise misconceptions, and increase knowledge on a given topic. We called the game: It’s Contagious.  

Beginning with research, we amassed 3.6 million pieces of Canadian-specific COVID-19 misinformation content. We then drew on this to determine seven key misinformation typologies and leading narratives - everything from “Bill Gates started it!” to “Vitamin C cures it!.” We then packaged these into It’s Contagious. 

It’s Contagious provided players the opportunity to test their knowledge by answering a series of True or False questions about the pandemic. After each question, players were provided with reinforcing or corrective information. If a player got a question wrong, they could recover their rank by giving us their opinion on a topical public policy question, such as mask mandates, or lockdowns. This served the double purpose of encouraging playthrough of the entire game, and collecting data on people’s emotions and experience during the pandemic so far. We gave players the option to provide demographic information about themselves.

To make sure players were comfortable we made it clear that we would not be collecting personally identifiable information.

We saw over 180,000 people engage with the tool, translating into over 2.11 million corrective statements. We also tested knowledge retention, as players were given a second chance later in the game to answer a question they had initially got wrong. A statistically significant 86.7% of players, across several demographic groups, got the question right on their second try. 

But to truly test the game’s effectiveness, we needed to put it up against a control group. We wanted to know whether the massive engagement was a result of our gamified approach, or just due to the topical nature of the informational content. So, we undertook a comparison study with nearly 20,000 Canadians who engaged with a non-gamified version of the same content. 

What we found was that It’s Contagious was more associated with positive emotions, which in turn were linked to support for public health measures and intention to take a COVID-19 vaccine once available. In contrast, those who experienced the non-gamified version were more likely to display anger and scepticism concerning the vaccine and public health measures. 

Together, these findings corroborated our understanding of gamification as an effective tool to address misinformation and improve knowledge. 

■ Try 'Know it or Not'

The Shift and How to Adapt

Come early 2021 however, the misinformation landscape shifted. Vaccines for COVID-19 were no longer theoretical. As a key measure, intent to take the vaccine was replaced with actual vaccine uptake, which was then replaced by booster uptake. With every step came new floods of misinformation and misconceptions. A notable divide in trust became increasingly apparent across the country. Differences in public messaging from varying levels of government and arguments between them created confusion. But despite this, the vast majority of Canadians rolled up their sleeves and followed public health recommendations to get vaccinated - 48% getting two jabs by July 2021. As time went on, more vaccines became available and public health guidance changed. Some interpreted changing recommendations from leaders as a cause for doubt. Groups that were already suspicious of the government felt their fears had been confirmed.

Amidst all this, we decided to take our game a step further. As the pandemic changed, we recognized that the question of COVID-19 misinformation required better than a one-size-fits-all answer. We thought that if we made our game adapt to the many types of vaccine hesitancy we were seeing, and used empathy to appeal to those hardest to reach, then our approach would be even more effective. Additionally, we committed to a rolling review of the content - safeguarding the content’s accuracy and topical nature. 

We also decided the game needed a new name. In the current iteration of the tool, Know it or Not, players are asked to indicate their feelings about booster vaccines by indicating whether they are either Confident, Neutral, Skeptical, or Angry about them. Their answer sends them down one of four possible streams, each equipped with specific information we have determined to be most relevant to their vaccine-hesitant typology. 

Questions for each stream were chosen from an ever growing pool of COVID-19 topics that we would update on a rolling basis based on new scientific knowledge and public health guidance. This was done based on the desired outcome for each stream. For example, for the stream of players who answered Confident, our goal was for them to share content found in the game with their friends and family, as social endorsement of health information can be a powerful motivator. So that is where we would place our more shareable content. For those who answered Neutral our intent was to encourage booster uptake - so here is where we would place content that focused to that end. 

This has a marked effect on one of our most important metrics - game completion. In earlier versions of the game, players who were unconfident in the vaccine, and therefore least likely to engage positively with the content, completed the game 28.75% of the time. The adaptive version caused this to jump to almost 50% - nearly double. 

What the adaptive version has taught us is that a one size fits all approach does not work as well as one that is tailored to each specific group. There are many different reasons why people might not want to get vaccinated - and we found that addressing those reasons in a targeted manner was a vital driver for the success of our new adaptive game.

Where We Are Now

In December 2022, we conducted a randomized control trial (RCT). Here, a nationally representative sample of 4,071 Canadian adults were asked to interact either with our game Know it or Not, a non-gamified blog offering the same information, or third, no-information group was used as another control. For all three, a baseline was taken to best assess the game’s potential impact. In this RCT,  the gamified version resulted in statistically significant improvements to vaccine knowledge, confidence and intent to get vaccinated. 

Specifically, in terms of vaccine knowledge, the gamified group improved by 129% compared to their baseline taken before the game and by 13% more than those who interacted with the blog. The gamified approach outperformed the blog on other related key outcomes too: participants trusted it more, were more willing to share it, and crucially - reported that it made them more flexible in their thinking on the vaccine. 

This was also true among the unboosted participants (those with 2 or less COVID-19 vaccine doses, which was 37% of our sample), with whom we saw statistically significant improvements to vaccine knowledge, intent to get vaccinated in the next 6 months, and flexible thinking about vaccines. 

We also compared DPS ads promoting the Know it or Not game with traditional health promotion ads, sponsored by the Canadian government. While the majority of the sample ranked the government ads as their top choice, DPS’s game ads were considered the top choice among people who, at baseline, reported lower on the following four metrics: vaccine knowledge, vaccine confidence, intent to get vaccinated in the next 6 months, and number of vaccine doses they have received; or in other words, among the most vulnerable segment of the population in need of additional educational support.

Nearly 150,000 Canadians have played the Know it or Not game to date - they have learned a lot from it. We have learned a lot from them as a group as well. Generally, Canadians got vaccinated when they were asked to. The unvaccinated are few, but remain loud - and as the years in this pandemic have dragged on, have likely become even more entrenched in their views. Apathy and burnout, even among those initially enthusiastic about vaccines, are on the rise. What our games, It’s Contagious and then Know It or Not, have shown is that while it is difficult to get through to those on the opposite side of an issue as contentious as the pandemic: it is not impossible, and it is worth it. Gamification has been proven as an effective tool against misinformation, and thus has an important role to play both within the context of the COVID-19 pandemic, and in misinformation crises to come. 

The national mood has shifted many times since March 2020. From fear of the unknown, to hope held by many in the vaccine, to the anger seen in protests, to the apathy felt towards the pandemic now. Our game has shifted with it, and adapted to get the accurate messaging Canadians need to hear through to those that need it most. 

■ Try 'Know it or Not'

If you would like a distinct link to share with your networks and collect aggregate, non-identifiable insights about your audiences, get in touch with us at hello@digitalpublicsquare.org.