Misinformation Exposure Beyond Traditional Feeds: Evidence from WhatsApp Deactivation Experiments in the Global South

.title[
# <span style="font-size: 70%;"> Misinformation Exposure Beyond Traditional Feeds: Evidence from WhatsApp Deactivation Experiments in the Global South</span>
]
.author[
### Tiago Ventura<br> <br><span style="font-size: 80%;">Georgetown University </span>
]
.date[
### <span style="font-size: 80%;"> <br>IDB - Governance & Development Seminar - Oct 3rd, 2024 </span>
]

---

<div class="my-footer"><span>Tiago Ventura (Georgetown University) &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp &nbsp IDB - Governance & Development Seminar</span></div>

---
class:middle

## Presentation Today

---
class:middle, inverse

# Motivation

---
class:middle
### Social Media, Misinformation and Politics

- **From Liberation to Turmoil**: In the last 10 years, the general public, journalist accounts and the academic research on the effects of social media in society has been dominated by the issue of misinformation (.midgrey[Tucker et. al., 2018])

- **A decade after**: gap between public discourse and scientific research on this issue .midgrey[(Budak et. al., 2024)]. Scientific research shows:

--
   - People aren’t actually exposed to a lot of misinformation – but exposure is heavily concentrated .midgrey[(Grinberg et. al. 2019, Guess et. al. 2019)]

- Nulls effects from in-the-field studies for exposure to misinformation on attitudes .midgrey[(Eady et al. 2023, and Meta papers)]
  
--
  
  - Mixed evidence from many deactivation studies on Facebook .midgrey[(Asimovic et. al. 2022, Alcott et. al. 2020, Arcenaux and Ladd 2023)]

---
class:middle
### Social Media, Misinformation and Politics

--
   - Over 80% of existing studies on interventions to correct misinformation focus on Global North countries .midgrey[(Blair et. al. 2023,  Badrinathan and Chauchard, 2023a)]

- Spreading, production and exposure to misinformation differs greatly across the world

--
   - In the Global South:
   
      - Informal networks for information sharing matter more ~ easier to mix true and false information
      - The real-world effects of misinformation exposure may be more pronounced in these countries, fewer fact-checking agencies, for example,
      - Social media messaging apps are more prevalent vs feed-based platforms

---
class:middle

---

.content-box-gray[Identify the causal effects of **.red[WhatsApp usage]** on **.red[exposure to online rumors]** and its downstream effects on **.red[belief accuracy]** and **.red[political attitudes]**]

---
class:middle

#### <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504 256C504 119 393 8 256 8S8 119 8 256s111 248 248 248 248-111 248-248zm-448 0c0-110.5 89.5-200 200-200s200 89.5 200 200-89.5 200-200 200S56 366.5 56 256zm72 20v-40c0-6.6 5.4-12 12-12h116v-67c0-10.7 12.9-16 20.5-8.5l99 99c4.7 4.7 4.7 12.3 0 17l-99 99c-7.6 7.6-20.5 2.2-20.5-8.5v-67H140c-6.6 0-12-5.4-12-12z"></path></svg> .red[Misinformation Exposure Beyond Traditional Feeds: Evidence from a WhatsApp Deactivation Experiment in Brazil] (With Rajeshwari Majumdar, Jonathan Nagler, and Joshua A. Tucker)

#### <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504 256C504 119 393 8 256 8S8 119 8 256s111 248 248 248 248-111 248-248zm-448 0c0-110.5 89.5-200 200-200s200 89.5 200 200-89.5 200-200 200S56 366.5 56 256zm72 20v-40c0-6.6 5.4-12 12-12h116v-67c0-10.7 12.9-16 20.5-8.5l99 99c4.7 4.7 4.7 12.3 0 17l-99 99c-7.6 7.6-20.5 2.2-20.5-8.5v-67H140c-6.6 0-12-5.4-12-12z"></path></svg> .palegrey[Reducing WhatsApp Usage to Mitigate Misinformation Exposure During Elections: Evidence from a Multi-Country Experiment (With Rajeshwari Majumdar, Shelley Liu, Carolina Torreblanca, and Joshua A. Tucker)]

#### <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504 256C504 119 393 8 256 8S8 119 8 256s111 248 248 248 248-111 248-248zm-448 0c0-110.5 89.5-200 200-200s200 89.5 200 200-89.5 200-200 200S56 366.5 56 256zm72 20v-40c0-6.6 5.4-12 12-12h116v-67c0-10.7 12.9-16 20.5-8.5l99 99c4.7 4.7 4.7 12.3 0 17l-99 99c-7.6 7.6-20.5 2.2-20.5-8.5v-67H140c-6.6 0-12-5.4-12-12z"></path></svg> .palegrey[WhatsApp Links Explorer: Data Donation Pipeline for WhatsApp Data (With Jonathan Nagler, and any others interested in collecting WhatsApp data)]

---
class:middle

## Deactivation Studies: A design to measure causal effects of social media usage

.pull-left[
.center[Facebook Deactivation in the US [(Alcott et al., 2020)](https://www.aeaweb.org/articles?id=10.1257/aer.20190658)]
]

.center[Facebook Deactivation in Bosnia and Herzegovina [(Asimovic et. al, 2022)](https://www.pnas.org/doi/10.1073/pnas.2022819118)]
]

---
class: middle

### Multimedia Deactivation Experiment in Brazil, 2022

- Disable their automatic download of media on WhatsApp.
- Do not purposefully click and watch any media on WhatsApp.

- **Start:** Three weeks before general election in Brazil.
- **End:** three days after the voting day.

---
class:middle, center, inverse

# Overview of the Experiment

---
class:middle

---
class:middle
## Recruitment

**Method:** Facebook Ads targeting adults Brazil

**Started Recruitment:** September 8th

**Number of respondents:** 1,947

**Screening variables:**

+ Willingness to join: .red[1,8861 participants]
  
  + More 5min WhatsApp daily

+ Only WhatsApp mobile
  
  + More than 2 min to complete recruitment + other quality controls

---
class:middle
## Treatment Assignment

**Start**: September 15.

**Treatment**:

- Turn off the automatic download.
- Three .red[weeks] without consuming any multimedia.
  
**Control**: Three .red[days] without consuming any multimedia.

**Invited**: 1,135 respondents

**Enrolled**: 773 respondents

---

## Deactivation

---
### Treatment Media Intervention & Compliance

#### Compliance: Monitoring Behavior

]]

---
class:middle
## Compliance

**Compliance Tasks:**

- Screenshots of WhatsApp storage intormation

- Every week + final survey

- After the  first compliance check, the control group was re-activated

---
class:middle

## Outcomes

**Self-Reported Exposure and Accuracy Judgment**

- False Rumors published on fact-checking webpages during the month of the experiment + True News stories headlines from mainstream media.

**Polarization**

- Affective polarization (Feeling Thermometer).
  - Social Polarization (Willingness to engage with outgroup voter).
  - Issue Polarization (Abortion, gun control, etc..).
  - Perceptions about ideological polarization ('Where do I observe the two main candidates?').

**Subjective Well-being**

- How did you feel for the past three weeks? (Happy, Anxious, etc...).

---
class: center, middle, inverse

# Results

---

### First-Stage Results

---
### Effects on Exposure to Misinformation Rumors (H1)

---
### Reduction in exposure is large for false news (H1)

---
### Effects on Belief Accuracy (H2)

---
### Effects on Polarization & Subjective Well-Being (H3 & H4)

---
class: middle, inverse, center

## Additional Analyses  (Non Pre-Registered)

---
### Exposure and Partisanship

---
### Belief Accuracy Conditional on WhatsApp usage for Politics

---
### Polarization Outcomes

---

## Discussion

---
class:middle

#### .palegrey[Misinformation Exposure Beyond Traditional Feeds: Evidence from a WhatsApp Deactivation Experiment in Brazil (With Rajeshwari Majumdar, Jonathan Nagler, and Joshua A. Tucker)]

#### <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504 256C504 119 393 8 256 8S8 119 8 256s111 248 248 248 248-111 248-248zm-448 0c0-110.5 89.5-200 200-200s200 89.5 200 200-89.5 200-200 200S56 366.5 56 256zm72 20v-40c0-6.6 5.4-12 12-12h116v-67c0-10.7 12.9-16 20.5-8.5l99 99c4.7 4.7 4.7 12.3 0 17l-99 99c-7.6 7.6-20.5 2.2-20.5-8.5v-67H140c-6.6 0-12-5.4-12-12z"></path></svg> .red[Reducing WhatsApp Usage to Mitigate Misinformation Exposure During Elections: Evidence from a Multi-Country Experiment] (With Rajeshwari Majumdar, Shelley Liu, Carolina Torreblanca, and Joshua A. Tucker)

#### .palegrey[WhatsApp Links Explorer: Data Donation Pipeline for WhatsApp Data (With Jonathan Nagler, and any others interested in collecting WhatsApp data)]

---
class: middle
### Extension: Multi-Country Deactivation Experiment

- **Problem**: Single country studies have limited power to speak of contextual effects (and in our case, limited statistical power)

- **Summary**: Multicountry extension across three major Global South countries; Brazil (In the field), India and South Africa

- **Stronger Treatment**: Replicate Multimedia Deactivation + Time Reduction

- **More statistical power**: ~ 2500 participants across the three countries.

---
class: middle
## Experimental Conditions

- **Types of partial deactivation**
   -  Time: Limit daily WhatsApp usage to 10 minutes 
   -  Multimedia: Do not consume multimedia content received via WhatsApp

- **Treatment and control**
 
 - Treatment: Change settings and  behavior for .red[4 week (one month)] leading up to elections in South Africa and India
 - Control: Change only behavior for .red[3 days] a month before election

- **2x2 design**

|Treatment|Control|
|----------------|--------------|
| Time Treatment | Time Control |
| Media Treatment | Media Control|

---
class:middle

### Treatment Time Intervention & Compliance

.pull-left[
.center[
#### Intervention: Adding Friction
<img src="time-limite.png" width="55%" />
]
]
.pull-right[
.center[
#### Compliance: Monitoring Behavior

<img src="ss_ttime_actualsubject_lightmode.png" width="60%" />
]
]

---
class:middle 
### Treatment Media Intervention & Compliance

.pull-left[
.center[
#### Intervention: Adding Friction
<img src="ss_tmedia_examplereal_dlpre.png" width="60%" />
]
]

#### Compliance: Monitoring Behavior

]]
---
class:middle

## Implementation

- Used Meta Advertisements to recruit WhatsApp users in all the three countries
- All participants could earn a total of $15 for completing surveys and uploading weekly screenshots
- Participants in treatment conditions who successfully reduced usage or multimedia consumption received an additional $45

- Started with 1,498 respondents in SA and IN countries;  1,394 (93\%) completed the endline survey. Currently have 930 participants enrolled in Brazil.

- Timeline SA and IN:

---
class: inverse, middle, center

# (Very) Preliminary Results

---
class: middle, center
#### Limiting WhatsApp usage reduces exposure to misinformation & news

---
class: middle, center

#### Reduction in exposure to misinformation & news has null effects on beleif accuracy

---
class: middle, center

#### Mostly nulls in polarization, except for identity-based prejudice in India

---
class:middle

## Next Steps

- Aggregate results from the Brazil new experiment.

- More HTEs and secondary outcomes to understand divergences in results across countries

-  Analyze compliance data  and estimate complier effects

-  2-week follow-up survey: effects of ``returning'' to WhatsApp?

---
class:middle

#### <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504 256C504 119 393 8 256 8S8 119 8 256s111 248 248 248 248-111 248-248zm-448 0c0-110.5 89.5-200 200-200s200 89.5 200 200-89.5 200-200 200S56 366.5 56 256zm72 20v-40c0-6.6 5.4-12 12-12h116v-67c0-10.7 12.9-16 20.5-8.5l99 99c4.7 4.7 4.7 12.3 0 17l-99 99c-7.6 7.6-20.5 2.2-20.5-8.5v-67H140c-6.6 0-12-5.4-12-12z"></path></svg> .palegrey[Misinformation Exposure Beyond Traditional Feeds: Evidence from a WhatsApp Deactivation Experiment in Brazil (With Rajeshwari Majumdar, Jonathan Nagler, and Joshua A. Tucker)]

#### <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504 256C504 119 393 8 256 8S8 119 8 256s111 248 248 248 248-111 248-248zm-448 0c0-110.5 89.5-200 200-200s200 89.5 200 200-89.5 200-200 200S56 366.5 56 256zm72 20v-40c0-6.6 5.4-12 12-12h116v-67c0-10.7 12.9-16 20.5-8.5l99 99c4.7 4.7 4.7 12.3 0 17l-99 99c-7.6 7.6-20.5 2.2-20.5-8.5v-67H140c-6.6 0-12-5.4-12-12z"></path></svg> .palegrey[Reducing WhatsApp Usage to Mitigate Misinformation Exposure During Elections: Evidence from a Multi-Country Experiment (With Rajeshwari Majumdar, Shelley Liu, Carolina Torreblanca, and Joshua A. Tucker)]

#### <svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M504 256C504 119 393 8 256 8S8 119 8 256s111 248 248 248 248-111 248-248zm-448 0c0-110.5 89.5-200 200-200s200 89.5 200 200-89.5 200-200 200S56 366.5 56 256zm72 20v-40c0-6.6 5.4-12 12-12h116v-67c0-10.7 12.9-16 20.5-8.5l99 99c4.7 4.7 4.7 12.3 0 17l-99 99c-7.6 7.6-20.5 2.2-20.5-8.5v-67H140c-6.6 0-12-5.4-12-12z"></path></svg> .red[WhatsApp Links Explorer: Data Donation Pipeline for WhatsApp Data] (With Jonathan Nagler, and any others interested in collecting WhatsApp data)

---
class:middle
### WhatsApp Link Explore: A Data Donation Pipeline

- WhatsApp is end-to-end encrypted.

- But, users can easily export their data. This makes Data Donation the easiest (possible only way to collect WhatsApp Data)

- Develop a pipeline using [WhatsApp Web](https://github.com/pedroslopez/whatsapp-web.js); A WhatsApp API client that connects through the WhatsApp Web browser app
---
class:middle
## Data Donation Pipeline:

- Recruit participants via surveys
  
- Ask them to authenticate in the web app (no need to download anything; but need two devices)
  
- Participants can choose which groups to donate the data. Only allow groups >5 participants.
  
- Collect: 
   - users' metadata, 
   - groups' metadata,
   - messages from donated groups.
   - only collect messages that contains URLs, avoid collecting risky and private content (images, messages, videos.)

---
## Examples of the Data Pipeline

]

- Planned Data Collections: CSIP-Latinos in the US; online sample in Brazil.

---
class:center, inverse, middle

# Thank you