Class 4: Logic models, measurement, and DAGs

# In-person<br>session 4

**February 6, 2025**

]

---

# Plan for today

.box-1.medium[Logic models and evaluation]

.box-6.medium[More regression things]

.box-2.medium[Measuring outcomes]

.box-5.medium[DAGs]

---

layout: false
name: exam
class: center middle section-title section-title-4 animated fadeIn

# Exam 1 details

---

layout: false
name: exam
class: center middle section-title section-title-4 animated fadeIn

# Final project details

---

layout: false
name: exam
class: center middle section-title section-title-7 animated fadeIn

# Extra bonus things

---

layout: false
name: logic-models
class: center middle section-title section-title-1 animated fadeIn

# Logic models<br>and evaluation

---

---

.box-1.medium[Do people really have the<br>job title "program evaluator"?]

.box-1.medium[How much does this evaluation stuff cost?]

.box-1.medium[Can you do scaled-down versions<br>of these evaluations?]

???

<https://www.eval.org/>

---

.box-1.medium[Isn't it best to always<br>have an articulated theory?]

.box-1.medium[Should implicit theory and articulated theory<br>be the same thing in most cases?]

---

.box-1.medium[What if a program exists already<br>and doesn't have a logic model?]

---

.box-1.medium[Why would a program aim for final outcomes<br>that can't be measured?]

---

.box-1.medium[What should you do if you find that your<br>theory of change (or logic model in general)<br>is wrong in the middle of the program?<br>Is it ethical to stop or readjust?]

---

.box-1.medium[How does regression<br>relate to impact evaluation?]

---

layout: false
name: regression-faqs
class: center middle section-title section-title-6 animated fadeIn

# More regression things

---

---

.box-6.medium[Do we care about the actual coefficients<br>or just whether or not they're significant?]

.box-6.medium[How does significance relate to causation?]

.box-6.medium[If we can't use statistics to assert causation<br>how are we going to use this information<br>in program evaluation?]

---

.box-6.large[What counts as a "good" R²?]

---

.center[
<figure>
  <img src="img/02-class/nice-plot-1.png" alt="Euler diagram" title="Euler diagram" width="45%">
</figure>
]

---

.center[
<figure>
  <img src="img/02-class/plot-diagram-prediction-1.png" alt="R2 prediction" title="R2 prediction" width="75%">
</figure>
]

---

.center[
<figure>
  <img src="img/02-class/plot-diagram-estimation-1.png" alt="R2 estimation" title="R2 estimation" width="75%">
</figure>
]

---

.center[
<figure>
  <img src="img/02-class/prediction-vs-estimation.jpg" alt="R2 estimation vs prediction" title="R2 estimation vs prediction" width="100%">
</figure>
]

---

.box-6.large[Side-by-side regression tables]

---

<div id="asxqktjote" style="padding-left:0px;padding-right:0px;padding-top:10px;padding-bottom:10px;overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>#asxqktjote table {
  font-family: system-ui, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol', 'Noto Color Emoji';
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

#asxqktjote thead, #asxqktjote tbody, #asxqktjote tfoot, #asxqktjote tr, #asxqktjote td, #asxqktjote th {
  border-style: none;
}

#asxqktjote p {
  margin: 0;
  padding: 0;
}

#asxqktjote .gt_table {
  display: table;
  border-collapse: collapse;
  line-height: normal;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#asxqktjote .gt_caption {
  padding-top: 4px;
  padding-bottom: 4px;
}

#asxqktjote .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#asxqktjote .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 3px;
  padding-bottom: 5px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#asxqktjote .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#asxqktjote .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#asxqktjote .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#asxqktjote .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#asxqktjote .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#asxqktjote .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#asxqktjote .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#asxqktjote .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#asxqktjote .gt_spanner_row {
  border-bottom-style: hidden;
}

#asxqktjote .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  text-align: left;
}

#asxqktjote .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#asxqktjote .gt_from_md > :first-child {
  margin-top: 0;
}

#asxqktjote .gt_from_md > :last-child {
  margin-bottom: 0;
}

#asxqktjote .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#asxqktjote .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#asxqktjote .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#asxqktjote .gt_row_group_first td {
  border-top-width: 2px;
}

#asxqktjote .gt_row_group_first th {
  border-top-width: 2px;
}

#asxqktjote .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#asxqktjote .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#asxqktjote .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#asxqktjote .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#asxqktjote .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#asxqktjote .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#asxqktjote .gt_last_grand_summary_row_top {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: double;
  border-bottom-width: 6px;
  border-bottom-color: #D3D3D3;
}

#asxqktjote .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#asxqktjote .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#asxqktjote .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#asxqktjote .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#asxqktjote .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#asxqktjote .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#asxqktjote .gt_left {
  text-align: left;
}

#asxqktjote .gt_center {
  text-align: center;
}

#asxqktjote .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#asxqktjote .gt_font_normal {
  font-weight: normal;
}

#asxqktjote .gt_font_bold {
  font-weight: bold;
}

#asxqktjote .gt_font_italic {
  font-style: italic;
}

#asxqktjote .gt_super {
  font-size: 65%;
}

#asxqktjote .gt_footnote_marks {
  font-size: 75%;
  vertical-align: 0.4em;
  position: initial;
}

#asxqktjote .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#asxqktjote .gt_indent_1 {
  text-indent: 5px;
}

#asxqktjote .gt_indent_2 {
  text-indent: 10px;
}

#asxqktjote .gt_indent_3 {
  text-indent: 15px;
}

#asxqktjote .gt_indent_4 {
  text-indent: 20px;
}

#asxqktjote .gt_indent_5 {
  text-indent: 25px;
}

#asxqktjote .katex-display {
  display: inline-flex !important;
  margin-bottom: 0.75em !important;
}

#asxqktjote div.Reactable > div.rt-table > div.rt-thead > div.rt-tr.rt-tr-group-header > div.rt-th-group:after {
  height: 0px !important;
}
</style>
<table class="gt_table" data-quarto-disable-processing="false" data-quarto-bootstrap="false">
  <thead>
    <tr class="gt_col_headings">
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" scope="col" id="a-"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" scope="col" id="a(1)">(1)</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" scope="col" id="a(2)">(2)</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" scope="col" id="a(3)">(3)</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" scope="col" id="a(4)">(4)</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td headers=" " class="gt_row gt_left">(Intercept)</td>
<td headers="(1)" class="gt_row gt_center">362.307</td>
<td headers="(2)" class="gt_row gt_center">-5780.831***</td>
<td headers="(3)" class="gt_row gt_center">-5736.897***</td>
<td headers="(4)" class="gt_row gt_center">-5433.534***</td></tr>
    <tr><td headers=" " class="gt_row gt_left"></td>
<td headers="(1)" class="gt_row gt_center">(283.345)</td>
<td headers="(2)" class="gt_row gt_center">(305.815)</td>
<td headers="(3)" class="gt_row gt_center">(307.959)</td>
<td headers="(4)" class="gt_row gt_center">(286.558)</td></tr>
    <tr><td headers=" " class="gt_row gt_left">bill_length_mm</td>
<td headers="(1)" class="gt_row gt_center">87.415***</td>
<td headers="(2)" class="gt_row gt_center"></td>
<td headers="(3)" class="gt_row gt_center">6.047</td>
<td headers="(4)" class="gt_row gt_center">-5.201</td></tr>
    <tr><td headers=" " class="gt_row gt_left"></td>
<td headers="(1)" class="gt_row gt_center">(6.402)</td>
<td headers="(2)" class="gt_row gt_center"></td>
<td headers="(3)" class="gt_row gt_center">(5.180)</td>
<td headers="(4)" class="gt_row gt_center">(4.860)</td></tr>
    <tr><td headers=" " class="gt_row gt_left">flipper_length_mm</td>
<td headers="(1)" class="gt_row gt_center"></td>
<td headers="(2)" class="gt_row gt_center">49.686***</td>
<td headers="(3)" class="gt_row gt_center">48.145***</td>
<td headers="(4)" class="gt_row gt_center">48.209***</td></tr>
    <tr><td headers=" " class="gt_row gt_left"></td>
<td headers="(1)" class="gt_row gt_center"></td>
<td headers="(2)" class="gt_row gt_center">(1.518)</td>
<td headers="(3)" class="gt_row gt_center">(2.011)</td>
<td headers="(4)" class="gt_row gt_center">(1.841)</td></tr>
    <tr><td headers=" " class="gt_row gt_left">sexmale</td>
<td headers="(1)" class="gt_row gt_center"></td>
<td headers="(2)" class="gt_row gt_center"></td>
<td headers="(3)" class="gt_row gt_center"></td>
<td headers="(4)" class="gt_row gt_center">358.631***</td></tr>
    <tr><td headers=" " class="gt_row gt_left" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;"></td>
<td headers="(1)" class="gt_row gt_center" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;"></td>
<td headers="(2)" class="gt_row gt_center" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;"></td>
<td headers="(3)" class="gt_row gt_center" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;"></td>
<td headers="(4)" class="gt_row gt_center" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;">(41.572)</td></tr>
    <tr><td headers=" " class="gt_row gt_left">Num.Obs.</td>
<td headers="(1)" class="gt_row gt_center">342</td>
<td headers="(2)" class="gt_row gt_center">342</td>
<td headers="(3)" class="gt_row gt_center">342</td>
<td headers="(4)" class="gt_row gt_center">333</td></tr>
    <tr><td headers=" " class="gt_row gt_left">R2</td>
<td headers="(1)" class="gt_row gt_center">0.354</td>
<td headers="(2)" class="gt_row gt_center">0.759</td>
<td headers="(3)" class="gt_row gt_center">0.760</td>
<td headers="(4)" class="gt_row gt_center">0.807</td></tr>
    <tr><td headers=" " class="gt_row gt_left">R2 Adj.</td>
<td headers="(1)" class="gt_row gt_center">0.352</td>
<td headers="(2)" class="gt_row gt_center">0.758</td>
<td headers="(3)" class="gt_row gt_center">0.759</td>
<td headers="(4)" class="gt_row gt_center">0.805</td></tr>
    <tr><td headers=" " class="gt_row gt_left">AIC</td>
<td headers="(1)" class="gt_row gt_center">5400.0</td>
<td headers="(2)" class="gt_row gt_center">5062.9</td>
<td headers="(3)" class="gt_row gt_center">5063.5</td>
<td headers="(4)" class="gt_row gt_center">4863.3</td></tr>
    <tr><td headers=" " class="gt_row gt_left">BIC</td>
<td headers="(1)" class="gt_row gt_center">5411.5</td>
<td headers="(2)" class="gt_row gt_center">5074.4</td>
<td headers="(3)" class="gt_row gt_center">5078.8</td>
<td headers="(4)" class="gt_row gt_center">4882.4</td></tr>
    <tr><td headers=" " class="gt_row gt_left">Log.Lik.</td>
<td headers="(1)" class="gt_row gt_center">-2696.987</td>
<td headers="(2)" class="gt_row gt_center">-2528.427</td>
<td headers="(3)" class="gt_row gt_center">-2527.741</td>
<td headers="(4)" class="gt_row gt_center">-2426.664</td></tr>
    <tr><td headers=" " class="gt_row gt_left">F</td>
<td headers="(1)" class="gt_row gt_center">186.443</td>
<td headers="(2)" class="gt_row gt_center">1070.745</td>
<td headers="(3)" class="gt_row gt_center">536.626</td>
<td headers="(4)" class="gt_row gt_center">457.118</td></tr>
    <tr><td headers=" " class="gt_row gt_left">RMSE</td>
<td headers="(1)" class="gt_row gt_center">643.54</td>
<td headers="(2)" class="gt_row gt_center">393.12</td>
<td headers="(3)" class="gt_row gt_center">392.34</td>
<td headers="(4)" class="gt_row gt_center">353.66</td></tr>
  </tbody>
  <tfoot class="gt_sourcenotes">
    <tr>
      <td class="gt_sourcenote" colspan="5">+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001</td>
    </tr>
  </tfoot>
  
</table>
</div>

---

.box-6.large[See full documentation and<br>examples for `modelsummary()` [here](https://vincentarelbundock.github.io/modelsummary/articles/modelsummary.html)]

---

.box-6.large[Make nicer tables [with {tinytable}](https://vincentarelbundock.github.io/tinytable/)]

---

layout: false
name: outcomes
class: center middle section-title section-title-2 animated fadeIn

# Measuring<br>outcomes

---

---

# The paradox of evaluation

--
.box-2.medium[Evaluation is good, but expensive]

.box-inv-2.sp-after.SMALL["Evaluation thinking"]

.box-2.medium[Too much evaluation is bad]

.box-inv-2.sp-after-half.SMALL[Taming programs]

---

# Outcomes and programs

.box-2.medium[Outcome variable]

.box-inv-2.sp-after-half.SMALL[Thing you're measuring]

.box-2.medium[Outcome change]

.box-inv-2.sp-after-half.SMALL[∆ in thing you're measuring over time]

.box-2.medium[Program effect]

.box-inv-2.SMALL[∆ in thing you're measuring over time *because of* the program]

---

# Outcomes and programs

---

.box-2.large[Abstraction]

---

layout: false
name: dags
class: center middle section-title section-title-5 animated fadeIn

# DAGs

---

---

.box-5.large[Causal thinking is necessary—<br>even for descriptive work!]

---

]

???

Necessity of causal thinking: Mention the McElreath tweet on birth certificate introduction and death ages: <https://twitter.com/rlmcelreath/status/1427564280744976384>

<https://www.biorxiv.org/content/10.1101/704080v2>

---

.box-5.less-medium["Every time I get a haircut, I become more mature!"]

---

.box-5.less-medium["Every time I get a haircut, I become more mature!"]

---

.box-5.less-medium[Getting older opens a backdoor path]

---

.box-5.medium[Ice cream causes crime]

.center[
<figure>
  <img src="img/05-class/eat-ice-cream-do-crime.jpg" alt="Eat ice cream, do crime" title="Eat ice cream, do crime" width="100%">
</figure>
]

???

- <https://twitter.com/EpiEllie/status/1080680304581451778>
- <https://www.redbubble.com/i/sticker/Eat-Ice-Cream-Do-Crime-by-epiellie/150075475.EJUG5>

---

.box-5.medium[But what does that mean,<br>"opening a backdoor path"?]

.box-5.medium[How does statistical association<br>get passed through paths?]

---

.box-5.less-medium.sp-after[How do I know which of these is which?]

.center[
<figure>
  <img src="img/05-class/dag-associations.png" alt="DAG associations" title="DAG associations" width="100%">
</figure>
]

---

.pull-left[
<figure>
  <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%">
</figure>
]

.pull-right[
<img src="04-class_files/figure-html/confounding-dag-alone-1.png" width="100%" style="display: block; margin: auto;" />
]

---

.pull-left[
<figure>
  <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%">
</figure>
]

.pull-right[
<img src="04-class_files/figure-html/mediating-dag-alone-1.png" width="100%" style="display: block; margin: auto;" />
]

---

.pull-left[
<figure>
  <img src="img/04-class/slider-switch-plain-80.jpg" alt="Switch and slider" title="Switch and slider" width="100%">
</figure>
]

.pull-right[
<img src="04-class_files/figure-html/colliding-dag-alone-1.png" width="100%" style="display: block; margin: auto;" />
]

---

.center[
<video controls loop>
  <source src="img/05-class/video-confounding-unblocked.mp4" type="video/mp4">
</video>
]

---

.center[
<video controls loop>
  <source src="img/05-class/video-confounding-blocked.mp4" type="video/mp4">
</video>
]

---

.center[
<video controls loop>
  <source src="img/05-class/video-mediating.mp4" type="video/mp4">
</video>
]

---

---

.box-5.medium[d-separation]

.box-inv-5[Except for the one arrow between X and Y,<br>no statistical association can flow between X and Y]

.box-inv-5[This is **identification**—<br>all alternative stories are ruled out<br>and the relationship is isolated]

---

.box-5.large[How exactly do colliders<br>mess up your results?]

.box-5.medium[It looks like you can<br>still get the effect of X on Y]

---

---

.center[
<figure>
  <img src="img/04-class/facebook.png" alt="Facebook collider" title="Facebook collider" width="55%">
</figure>
]

???

<https://www.businessinsider.com/facebook-sent-incomplete-misinformation-data-flawed-researchers-2021-9>

<https://www.nytimes.com/live/2020/2020-election-misinformation-distortions#facebook-sent-flawed-data-to-misinformation-researchers>

---

# Does niceness improve appearance?

]

]

---

# Collider distorts the true effect!

]

]

---

.box-5.large[Effect of race on police use of force<br>using administrative data]

---

.box-5.medium[Effect of race on police use of force<br>using administrative data]

.pull-left[
<figure>
  <img src="img/05-class/klm-dag.png" alt="Use of force" title="Use of force" width="100%">
</figure>
]

.pull-right[
<figure>
  <img src="img/05-class/klm.png" alt="Use of force" title="Use of force" width="100%">
</figure>
]