Class 3: {dplyr} and logic models

# In-person<br>session 3

**January 30, 2025**

]

---

# Plan for today

.box-4.medium[{ggplot2} and {dplyr}]

.box-1.medium[Logic models and evaluation]

.box-5.medium[Regression FAQs]

---

layout: false
name: regression
class: center middle section-title section-title-4 animated fadeIn

# {ggplot2} and {dplyr}

---

layout: false
name: logic-models
class: center middle section-title section-title-1 animated fadeIn

# Logic models<br>and evaluation

---

---

.box-1.large[Logic models as managerial tools]

---

.box-1.large[Inputs vs. Activities vs.<br>Outputs vs. Outcomes]

---

.box-1.large[Impact theory vs. logic model]

---

# Impact theory

.center[
<figure>
  <img src="img/03/psd-impact-theory-arrow.png" alt="PSD impact theory" title="PSD impact theory" width="100%">
</figure>
]

---

.center[
<figure>
  <img src="img/01/psd-logic-model.png" alt="PSD logic model" title="PSD logic model" width="70%">
</figure>
]

---

# MPA/MPP at GSU

.pull-left[
<figure>
  <img src="img/03/mpp-mission@4x.png" alt="MPP mission" title="MPP mission" width="130%">
</figure>
]

.pull-right[
<figure>
  <img src="img/03/mpa-mission@4x.png" alt="MPA mission" title="MPA mission" width="100%">
</figure>
]

???

Inputs:

- Students
- Curriculum
- Money (grants, tuition)
- AYSPS itself
- Faculty, staff
- Technology (iCollege, PAWS)
- Infrastructure
- Transportation
- State regulations
- National accreditation standards
- Faculty research

Activities:

- Classes
- Group work
- Exams
- Internships
- Job fairs
- Commencement
- Studying
- Tutoring
- Office hours
- Advising
- Attendance

Outputs:

- Assignments and projects
- Grades
- Degree
- Jobs
- Awards
- Network
- Debt

Outcomes:

- Leadership
- AYS national ranking
- Public service motivation
- Critical thinking
- Efficient workforce
- Income
- Better society

---

---

.box-1.medium[Do people really have the<br>job title "program evaluator"?]

.box-1.medium[How much does this evaluation stuff cost?]

.box-1.medium[Can you do scaled-down versions<br>of these evaluations?]

???

<https://www.eval.org/>

---

.box-1.medium[Isn't it best to always<br>have an articulated theory?]

.box-1.medium[Should implicit theory and articulated theory<br>be the same thing in most cases?]

---

.box-1.medium[What if a program exists already<br>and doesn't have a logic model?]

---

.box-1.medium[Why would a program aim for final outcomes<br>that can't be measured?]

---

.box-1.medium[What should you do if you find that your<br>theory of change (or logic model in general)<br>is wrong in the middle of the program?<br>Is it ethical to stop or readjust?]

---

.box-1.medium[How does regression<br>relate to impact evaluation?]

---

layout: false
name: regression-faqs
class: center middle section-title section-title-5 animated fadeIn

# Regression FAQs

---

---

.box-5.medium[Do we care about the actual coefficients<br>or just whether or not they're significant?]

.box-5.medium[How does significance relate to causation?]

.box-5.medium[If we can't use statistics to assert causation<br>how are we going to use this information<br>in program evaluation?]

---

.box-5.large[What counts as a "good" R²?]

---

.center[
<figure>
  <img src="img/02-class/nice-plot-1.png" alt="Euler diagram" title="Euler diagram" width="45%">
</figure>
]

---

.center[
<figure>
  <img src="img/02-class/plot-diagram-prediction-1.png" alt="R2 prediction" title="R2 prediction" width="75%">
</figure>
]

---

.center[
<figure>
  <img src="img/02-class/plot-diagram-estimation-1.png" alt="R2 estimation" title="R2 estimation" width="75%">
</figure>
]

---

.center[
<figure>
  <img src="img/02-class/prediction-vs-estimation.jpg" alt="R2 estimation vs prediction" title="R2 estimation vs prediction" width="100%">
</figure>
]

---

.box-5.large[Side-by-side regression tables]

---

<div id="mckilmuzha" style="padding-left:0px;padding-right:0px;padding-top:10px;padding-bottom:10px;overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>#mckilmuzha table {
  font-family: system-ui, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol', 'Noto Color Emoji';
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

#mckilmuzha thead, #mckilmuzha tbody, #mckilmuzha tfoot, #mckilmuzha tr, #mckilmuzha td, #mckilmuzha th {
  border-style: none;
}

#mckilmuzha p {
  margin: 0;
  padding: 0;
}

#mckilmuzha .gt_table {
  display: table;
  border-collapse: collapse;
  line-height: normal;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#mckilmuzha .gt_caption {
  padding-top: 4px;
  padding-bottom: 4px;
}

#mckilmuzha .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#mckilmuzha .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 3px;
  padding-bottom: 5px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#mckilmuzha .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#mckilmuzha .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#mckilmuzha .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#mckilmuzha .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#mckilmuzha .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#mckilmuzha .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#mckilmuzha .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#mckilmuzha .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#mckilmuzha .gt_spanner_row {
  border-bottom-style: hidden;
}

#mckilmuzha .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  text-align: left;
}

#mckilmuzha .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#mckilmuzha .gt_from_md > :first-child {
  margin-top: 0;
}

#mckilmuzha .gt_from_md > :last-child {
  margin-bottom: 0;
}

#mckilmuzha .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#mckilmuzha .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#mckilmuzha .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#mckilmuzha .gt_row_group_first td {
  border-top-width: 2px;
}

#mckilmuzha .gt_row_group_first th {
  border-top-width: 2px;
}

#mckilmuzha .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#mckilmuzha .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#mckilmuzha .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#mckilmuzha .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#mckilmuzha .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#mckilmuzha .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#mckilmuzha .gt_last_grand_summary_row_top {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: double;
  border-bottom-width: 6px;
  border-bottom-color: #D3D3D3;
}

#mckilmuzha .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#mckilmuzha .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#mckilmuzha .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#mckilmuzha .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#mckilmuzha .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#mckilmuzha .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#mckilmuzha .gt_left {
  text-align: left;
}

#mckilmuzha .gt_center {
  text-align: center;
}

#mckilmuzha .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#mckilmuzha .gt_font_normal {
  font-weight: normal;
}

#mckilmuzha .gt_font_bold {
  font-weight: bold;
}

#mckilmuzha .gt_font_italic {
  font-style: italic;
}

#mckilmuzha .gt_super {
  font-size: 65%;
}

#mckilmuzha .gt_footnote_marks {
  font-size: 75%;
  vertical-align: 0.4em;
  position: initial;
}

#mckilmuzha .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#mckilmuzha .gt_indent_1 {
  text-indent: 5px;
}

#mckilmuzha .gt_indent_2 {
  text-indent: 10px;
}

#mckilmuzha .gt_indent_3 {
  text-indent: 15px;
}

#mckilmuzha .gt_indent_4 {
  text-indent: 20px;
}

#mckilmuzha .gt_indent_5 {
  text-indent: 25px;
}

#mckilmuzha .katex-display {
  display: inline-flex !important;
  margin-bottom: 0.75em !important;
}

#mckilmuzha div.Reactable > div.rt-table > div.rt-thead > div.rt-tr.rt-tr-group-header > div.rt-th-group:after {
  height: 0px !important;
}
</style>
<table class="gt_table" data-quarto-disable-processing="false" data-quarto-bootstrap="false">
  <thead>
    <tr class="gt_col_headings">
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" scope="col" id="a-"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" scope="col" id="a(1)">(1)</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" scope="col" id="a(2)">(2)</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" scope="col" id="a(3)">(3)</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" scope="col" id="a(4)">(4)</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td headers=" " class="gt_row gt_left">(Intercept)</td>
<td headers="(1)" class="gt_row gt_center">362.307</td>
<td headers="(2)" class="gt_row gt_center">-5780.831***</td>
<td headers="(3)" class="gt_row gt_center">-5736.897***</td>
<td headers="(4)" class="gt_row gt_center">-5433.534***</td></tr>
    <tr><td headers=" " class="gt_row gt_left"></td>
<td headers="(1)" class="gt_row gt_center">(283.345)</td>
<td headers="(2)" class="gt_row gt_center">(305.815)</td>
<td headers="(3)" class="gt_row gt_center">(307.959)</td>
<td headers="(4)" class="gt_row gt_center">(286.558)</td></tr>
    <tr><td headers=" " class="gt_row gt_left">bill_length_mm</td>
<td headers="(1)" class="gt_row gt_center">87.415***</td>
<td headers="(2)" class="gt_row gt_center"></td>
<td headers="(3)" class="gt_row gt_center">6.047</td>
<td headers="(4)" class="gt_row gt_center">-5.201</td></tr>
    <tr><td headers=" " class="gt_row gt_left"></td>
<td headers="(1)" class="gt_row gt_center">(6.402)</td>
<td headers="(2)" class="gt_row gt_center"></td>
<td headers="(3)" class="gt_row gt_center">(5.180)</td>
<td headers="(4)" class="gt_row gt_center">(4.860)</td></tr>
    <tr><td headers=" " class="gt_row gt_left">flipper_length_mm</td>
<td headers="(1)" class="gt_row gt_center"></td>
<td headers="(2)" class="gt_row gt_center">49.686***</td>
<td headers="(3)" class="gt_row gt_center">48.145***</td>
<td headers="(4)" class="gt_row gt_center">48.209***</td></tr>
    <tr><td headers=" " class="gt_row gt_left"></td>
<td headers="(1)" class="gt_row gt_center"></td>
<td headers="(2)" class="gt_row gt_center">(1.518)</td>
<td headers="(3)" class="gt_row gt_center">(2.011)</td>
<td headers="(4)" class="gt_row gt_center">(1.841)</td></tr>
    <tr><td headers=" " class="gt_row gt_left">sexmale</td>
<td headers="(1)" class="gt_row gt_center"></td>
<td headers="(2)" class="gt_row gt_center"></td>
<td headers="(3)" class="gt_row gt_center"></td>
<td headers="(4)" class="gt_row gt_center">358.631***</td></tr>
    <tr><td headers=" " class="gt_row gt_left" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;"></td>
<td headers="(1)" class="gt_row gt_center" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;"></td>
<td headers="(2)" class="gt_row gt_center" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;"></td>
<td headers="(3)" class="gt_row gt_center" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;"></td>
<td headers="(4)" class="gt_row gt_center" style="border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: #000000;">(41.572)</td></tr>
    <tr><td headers=" " class="gt_row gt_left">Num.Obs.</td>
<td headers="(1)" class="gt_row gt_center">342</td>
<td headers="(2)" class="gt_row gt_center">342</td>
<td headers="(3)" class="gt_row gt_center">342</td>
<td headers="(4)" class="gt_row gt_center">333</td></tr>
    <tr><td headers=" " class="gt_row gt_left">R2</td>
<td headers="(1)" class="gt_row gt_center">0.354</td>
<td headers="(2)" class="gt_row gt_center">0.759</td>
<td headers="(3)" class="gt_row gt_center">0.760</td>
<td headers="(4)" class="gt_row gt_center">0.807</td></tr>
    <tr><td headers=" " class="gt_row gt_left">R2 Adj.</td>
<td headers="(1)" class="gt_row gt_center">0.352</td>
<td headers="(2)" class="gt_row gt_center">0.758</td>
<td headers="(3)" class="gt_row gt_center">0.759</td>
<td headers="(4)" class="gt_row gt_center">0.805</td></tr>
    <tr><td headers=" " class="gt_row gt_left">AIC</td>
<td headers="(1)" class="gt_row gt_center">5400.0</td>
<td headers="(2)" class="gt_row gt_center">5062.9</td>
<td headers="(3)" class="gt_row gt_center">5063.5</td>
<td headers="(4)" class="gt_row gt_center">4863.3</td></tr>
    <tr><td headers=" " class="gt_row gt_left">BIC</td>
<td headers="(1)" class="gt_row gt_center">5411.5</td>
<td headers="(2)" class="gt_row gt_center">5074.4</td>
<td headers="(3)" class="gt_row gt_center">5078.8</td>
<td headers="(4)" class="gt_row gt_center">4882.4</td></tr>
    <tr><td headers=" " class="gt_row gt_left">Log.Lik.</td>
<td headers="(1)" class="gt_row gt_center">-2696.987</td>
<td headers="(2)" class="gt_row gt_center">-2528.427</td>
<td headers="(3)" class="gt_row gt_center">-2527.741</td>
<td headers="(4)" class="gt_row gt_center">-2426.664</td></tr>
    <tr><td headers=" " class="gt_row gt_left">F</td>
<td headers="(1)" class="gt_row gt_center">186.443</td>
<td headers="(2)" class="gt_row gt_center">1070.745</td>
<td headers="(3)" class="gt_row gt_center">536.626</td>
<td headers="(4)" class="gt_row gt_center">457.118</td></tr>
    <tr><td headers=" " class="gt_row gt_left">RMSE</td>
<td headers="(1)" class="gt_row gt_center">643.54</td>
<td headers="(2)" class="gt_row gt_center">393.12</td>
<td headers="(3)" class="gt_row gt_center">392.34</td>
<td headers="(4)" class="gt_row gt_center">353.66</td></tr>
  </tbody>
  <tfoot class="gt_sourcenotes">
    <tr>
      <td class="gt_sourcenote" colspan="5">+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001</td>
    </tr>
  </tfoot>
  
</table>
</div>

---

.box-5.large[See full documentation and<br>examples for `modelsummary()` [here](https://vincentarelbundock.github.io/modelsummary/articles/modelsummary.html)]

---

.box-5.large[Make nicer tables [with {tinytable}](https://vincentarelbundock.github.io/tinytable/)]