Group sign-up

Please add your names to this Google Sheet. Include the names of your group members, the article name, a link to the article, and indicate whether you are open to others joining you on your assignment. Be sure to use the correct tab in the Google Sheet: “AQM Elective” or “AQM Pol behavior”.

Introduction

In this assignment you will apply the skills that you acquire in the course to replicate or reproduce an existing academic study. You can choose to do this in one of two ways:

  1. Re-analyze data from an existing study by:
    • Reproducing the original results
    • Providing an extension of the original study
  2. Reproducing the results of a published study with new data.
    • Apply the same analysis in an article to a new context with different data

The study you replicate or reproduce must have a quasi-experimental research design that uses at least one of the following methods:

  1. Instrumental variables
  2. Difference-in-differences
  3. Regression discontinuity

You can also replicate a sufficiently advanced quantitative article that does not use one of these methods. However, the bar for what I consider “sufficiently advanced” is high, so you must ask me first if you want to replicate/reproduce an article that does not use one of the methods above.

Selecting an article to replicate or reproduce

To find a relevant article, I strongly suggest that you look through recent issues of political science’s “top three” general interest journals:

  1. American Political Science Review (APSR)
  2. American Journal of Political Science (AJPS)
  3. Journal of Politics (JOP).

These journals now require authors to submit replication data so that anyone—like yourselves—can replicate their study. The link to the replication data for articles in these journals is usually printed on the first page of the article (either below the abstract or in the article’s first footnote). i.e. you should not have to search around to find the original data.

You can use a study from a different journal. However, the three journals above are considered the best in the field; will often use research designs whose aim is causal inference; and will have replication data readily available. Many other journals do not require replication data, in which case you would be out of luck.

Occasionally, the data or code that you might expect to be in a replication archive will not be there. If you believe that something is truly missing, please email or talk with me first. Do not contact the authors of the study without talking to me first (people are busy!).

If you selected a “Short article” or “Letter”, and it does not appear like sufficient work is required to replicate it, please talk to me.

Pre-selected articles

Below are a few articles that I have pre-selected for you to replicate, if you have trouble/don’t want to look for an article:

Violence and Voting in the United States: How School Shootings Affect Elections
Difference-in-differences
American Political Science Review, 2022
García-Montoya, Laura, Ana Arjona, and Matthew Lacombe

How Political Parties Shape Public Opinion in the Real World
Difference-in-differences
American Journal of Political Science, 2021
Rune Slothuus and Martin Bisgaard

When Time is of the Essence: A Natural Experiment on How Time Constraints Influence Elections
Regression discontinuity
Journal of Politics, 2020
Jerome Schafer and John B. Holbein

How Partisan Is Local Law Enforcement? Evidence from Sheriff Cooperation with Immigration Authorities
Regression discontinuity
American Political Science Review, 2020
Daniel M. Thompson

Does Exposure to the Refugee Crisis Make Natives More Hostile?
Instrumental variables
American Political Science Review, 2019
Dominik Hangartner, Elias Dinas, Moritz Marbach, Konsstantinos Matakos, and Dimitrios Xefteris

Example assignments

Instructions for each assignment type are further below. However, to give you a sense of what a higher-quality assignment might look like, I have posted a few examples of assignments from students from the 2020 AQM cohort (students gave permission to post their assignments).

Formalities

All of the below requirements are standard in basically all working papers in political science and for submission to journals, so please stick to the following conventions:

  • 12-point font
  • Double-spaced (i.e. not one or one-half spacing)
  • 1 inch margins

If you write your assignment in LaTeX (e.g. Overleaf), figuring this out is pretty easy.

General expectations

Please note that although the technical side of your assignment should be well-executed, you should also focus your energy on the clarity of your writing. The reason that students often do not achieve the grade that they might hope for is because the written part of their assignment is unclear. So please be sure to spend sufficient time working on the structure of the assignment and ensuring that you clearly write the goal of the study, the research design, the results, and the discussion and conclusion.

Option 1: Re-analysis of existing data

For this type of replication, you are expected to reproduce and extend the core results of a published study by using the original data, which are often released alongside the published article. The instructions are as follows:

  1. Briefly introduce the research question; it’s importance; and a description of the theory and hypotheses.
    • Do not have a paragraph describing the structure of the paper. i.e. Do not have paragraph that reads something like: “First, we do this. Second, we do that. etc.”
  2. Reproduce and present all results in the paper without copying and pasting or referring to the authors’ code. This means that every figure and table in the main text of the article (i.e. not the appendix) should be replicated and inserted into your assignment. You do not need to present the figures and tables in exactly the same way as the original study if you want, i.e. you are free to choose the format that you think best communicates the central findings. You do not need to replicate anything in the appendix.
    • You do not need to replicate a figure or table if it does not use data (e.g. a stylized example of a relationship between two variables, a timeline, a flowchart of a survey)
    • You do not need to replicate any maps if you do not want to
    • Note: Most replication files will not come with a codebook. Thus, it may not always be clear from an article and the dataset alone which variable is which, or what subset of data the authors use. Thus, you can look at the authors’ code to do a bit of detective work to figure out which variables, subsets, and regression specifications the authors use.
    • Please note that it is extremely obvious when students copy and paste code from the replication code from a study, even if it is a few lines. So make sure to talk to me if you run into any issues with your own code.

  3. Extend the original analysis in a theoretically and methodologically informed manner. This can be done in a variety of ways, e.g. by estimating uncertainty differently; by seeing whether the analysis of certain sub-groups would change the original conclusions; by providing better visualization that display the results more clearly (for theoretical reasons, not aesthetically). Try to provide one meaningful extension, or two smaller ones (e.g. a different visualization is typically what I would call a small extension). When you write up your extension, make it clear why it is useful to do the extension that you are doing, whether the reason is theoretical or empirical. If part of your extension is a better visualization of the results or data, make sure to explain why the author(s)’ version was insufficient (show the original so I can see it for comparison).

  4. Present a substantive discussion of the study’s limitations, and how they matter for the inferences we can possibly draw from it. Suggest changes to the design that you believe would improve the inferences.

Option 2: Reproduction with new data

This option requires that you reproduce the core results of a published study by using a different dataset. You are not expected to collect new data yourselves. Instead, you should use an existing dataset to re-investigate a previously published study. The instructions are as follows:

  1. Reproduce and present the results (i.e. all tables and figures) from a published article of your choice with a different dataset. Explain how this dataset captures the concepts that the original study was interested in, and insofar as the exact procedure cannot be replicated with the new dataset, explain why the replication nevertheless informs us about the original study’s object of inference.

  2. Discuss the extent to which your results are similar or dissimilar from those in the original study.

  3. Discuss the study’s limitations, and how they matter for the inferences that we can draw from it. Suggest potential and realistic changes to the design that would allow for more credible inferences.

Submission instructions

  1. Remember these requirements:

    • 12-point font
      • In Overleaf: \documentclass[12pt, letterpaper]{article}
    • Double-spaced
      • In Overleaf: \doublespacing after your \begin{document}
    • 1 inch margins
      • In Overleaf, \usepackage[margin = 1in]{geometry}

  2. Submit your (1) assignment, (2) the article itself, and (3) your R code separately. Please send your assignment as a PDF (not a .docx!), and use an equivalent file name across them all:

    • Assignment_2.pdf
    • Assignment_2_Article.pdf
    • Assignment_2.R

  3. None of your R code should be in the PDF. It only needs to be in the .R file itself.

  4. Print the number of characters on the front page of your assignment. If you use LaTeX, use the Monterey Language Services to calculate the number of words in your assignment, with the assumption of 6 characters per word.

  5. Your code should be fully commented so that it is clear to me that you know what each piece of code does. You don’t need to be excessive here, but comment code in a way you think is reasonable.

  6. If you are in a group, only one of you needs to submit the assignment. Just ensure that all of your student IDs are on the title page.

  7. Submit all of this through Absalon under “Assignment 2”.