Class 8: Diff-in-diff I

# In-person session 8

**March 6, 2025**

]

---

# Plan for today

.box-6.medium.sp-after-half[Matching and IPW]

.box-5.medium.sp-after-half[Models vs. designs]

.box-3.medium.sp-after-half[Interactions and regression]

.box-1.medium.sp-after-half[Simple diff-in-diff]

---

layout: false
name: matching-ipw
class: center middle section-title section-title-6 animated fadeIn

# Matching and IPW

---

---

.box-6.large[Can you talk more about propensity scores and "weirdness" weights?]

.center.small[[Lecture slide](https://evalsp25.classes.andrewheiss.com/slides/07-slides.html#128)]

---

.center[
<figure>
 <img src="img/07-class/fig-propensity-treated-untreated-1.png" alt="IPW weight histogram" title="IPW weight histogram" width="75%">
</figure>
]

---

.center[
<figure>
 <img src="img/07-class/fig-propensity-weird-highlight-1.png" alt="IPW weight histogram" title="IPW weight histogram" width="75%">
</figure>
]

---

.center[
<figure>
 <img src="img/07-class/fig-propensity-weighted-ate-1.png" alt="IPW weight histogram" title="IPW weight histogram" width="75%">
</figure>
]

---

.box-6.large[Why not just control for confounders instead of doing the whole matching/IPW dance?]

---

.box-6.large[Do you have to use logistic regression + OLS for IPW?]

---

.box-6.large[Which should we use? Matching or IPW?]

---

.box-6.large[Can you walk through an example of IPW and matching in class?]

---

layout: false
name: models-designs
class: center middle section-title section-title-5 animated fadeIn

# Models vs. designs

---

---

.center[
<figure>
 <img src="img/08-class/2021-nobel-winners.jpg" alt="2021 econ Nobel winners" title="2021 econ Nobel winners" width="55%">
</figure>
]

???

- Card (and Krueger): NJ/PA minimum wage + the beginning of this whole credibility revolution thing
- Angrist: MHE and MM and making causal inference accessible
- Imbens: A ton of CI stuff + attempting to bridge DAG world with situation-based world

- https://twitter.com/NobelPrize/status/1447502627114205187 - PA/NJ
- https://twitter.com/MaxCRoser/status/1447505582450151431
- https://twitter.com/Stanford/status/1447549033539637248

---

.center[
<figure>
 <img src="img/08-class/alan-krueger.jpg" alt="Alan Krueger" title="Alan Krueger" width="80%">
</figure>
]

???

Alan Krueger died by suicide in 2019

---

.center[
<figure>
 <img src="img/08-class/pa-nj-nobel.jpg" alt="Nobel PA/NJ" title="Nobel PA/NJ" width="57%">
</figure>
]

---

---

.box-5.large[Design-based vs. model-based inference]

---

.box-5.medium[How would you know when it is appropriate to use a quasi-experiment over an RCT?]

---

---

# Identification strategies

.box-inv-5.small.sp-after[The goal of *all* these methods is to isolate (or **identify**) the arrow between treatment → outcome]

.box-inv-5.less-medium[Model-based identification]

.float-left.center[.box-5[DAGs] .box-5[Matching] .box-5[Inverse probability weighting]]

.box-inv-5.less-medium.sp-before[Design-based identification]

.float-left.center[.box-5[Randomized controlled trials] .box-5[Difference-in-differences]]

.float-left.center[.box-5[Regression discontinuity] .box-5[Instrumental variables]]

---

# Model-based identification

.pull-left[
<figure>
 <img src="04-slides_files/figure-html/edu-earn-adjust-1.png" alt="Education earnings DAG" title="Education earnings DAG" width="100%">
</figure>
]

.box-inv-5.small[Everything that needs to be adjusted is measurable; no unobserved confounding]

.box-inv-5.small[**Big assumption!**]

.box-inv-5.tiny[This is why lots of people don't like DAG-based adjustment]
]

---

.center[
<figure>
 <img src="img/08-class/charles-ozzy.png" alt="King Charles and Ozzy Osbourne" title="King Charles and Ozzy Osbourne" width="50%">
</figure>
]

---

---

# Design-based identification

.box-inv-5.small[Use randomization to remove confounding]

.center[
<figure>
 <img src="05-slides_files/figure-html/experimental-dag-1.png" alt="RCT DAG" title="RCT DAG" width="60%">
</figure>
]
]

.box-inv-5.small[Use before/after & treatment/control differences to remove confounding]

.center[
<figure>
 <img src="08-slides_files/figure-html/min-wage-dag-1.png" alt="Diff-in-diff DAG" title="Diff-in-diff DAG" width="90%">
</figure>
]
]

---

---

.box-5.large[Which is better or more credible? RCTs, quasi experiments, or DAG-based models?]

---

.center[
<figure>
 <img src="img/08-class/causality-continuum.png" alt="The (wrong!) causality continuum" title="The (wrong!) causality continuum" width="90%">
</figure>
]

---

.box-5.huge[There's no hierarchy!]

---

layout: false
name: interactions
class: center middle section-title section-title-3 animated fadeIn

# Interactions and regression

---

.box-3.large[Can we talk more about interaction terms and how to interpret them?]

---

.box-3.large[Regression is just fancy averages!]

---

layout: false
name: diff-in-diff
class: center middle section-title section-title-1 animated fadeIn

# Simple diff-in-diff

---

.center[
<figure>
 <img src="img/08-class/lambeth-southwark-vauxhall.jpg" alt="Lambeth and Southwark-Vauxhall" title="Lambeth and Southwark-Vauxhall" width="70%">
</figure>
]

---

]

---

.center[
<figure>
 <img src="img/08-class/bedtime-math.png" alt="Bedtime math" title="Bedtime math" width="45%">
</figure>
]

---

.center[
<figure>
 <img src="img/08-class/bedtime-math-diff-diff.png" alt="Bedtime math diff-in-diff" title="Bedtime math diff-in-diff" width="100%">
</figure>
]

---

---

.box-1.medium[When doing your subtracting to get your differences in the matrix, is it better to do the vertical or horizontal subtractions?]

.box-1.medium[Are there situations where one is preferable to the other?]

---

.box-1.medium[Why are we learning two ways to do diff-in-diff? (2x2 matrix vs. `lm()`)]

---

.box-1.large[What happened to confounding??]

.box-1.medium[Now we're only looking at just two "confounders"?]

.box-1.medium[Should we still control for things?]

???

The parallel trends assumption takes care of that

---

.box-1.large[Can you walk through an example of diff-in-diff in class?]