Part 1: Definitions and Basic Concepts

1.   Define the concept of Parallel Trends in the context of difference-in-difference (DD) and describe a possible empirical test to check whether this assumption holds in a      model.

Solution: The parallel trends assumption is the core assumption of the DD model.      This assumption states that, although the treatment and the control group may look     different in terms of their observable characteristics at baseline (and by construction,  may be different in terms of their unobservable characteristics at baseline), the gap     between the treatment and the control group will remain constant over time as long as there is no treatment delivered to the treatment group.

An empirical test for parallel trends focuses on showing pre-treatment levels of the outcome of interest for both the treatment and control groups in one or more time   periods --i.e., at time t- 1, t-2, t-3, t-4,…etc. where t= time of the intervention.

2.   In your own words, explain why the difference-in-difference methodology is said to perform well in short-term evaluations but may not perform as well in long-term      evaluations?

Solution: The DD is ideal for short term evaluations but not so much for long-term    evaluations because of the parallel trend assumption. It is hard to argue that the           treatment and control group evolve in parallel trends for a long time since the              treatment was delivered, as many things can change over time and could differentially affect treatment and control groups.

3.   Define what is meant by a “natural experiment” and how it differs from a “randomized experiment” .

Solution: A natural experiment is a case in which individuals or units of study (e.g., families, villages, schools, etc.) are exposed to naturally occurring conditions outside the control of the researcher. One example of a natural experiment is variation in       environmental conditions – e.g., forest fires, hurricanes, COVID- 19, etc.

A natural experiment differs from a randomized experiment in the sense that the randomized experiment is 100% designed and executed by the researcher, so all

variables in the study are under the control of the researcher (e.g., the type of              treatment, the dosage of the treatment, when to deliver the treatment, when to stop the treatment, how to deliver the treatment, etc.).

4.   Explain what the difference between repeated cross-sectional data and longitudinal data is. Which of these datasets can be used to estimate a DD model?

Solution: A cross-sectional dataset is a dataset that samples individuals or other units  of study (e.g., families, villages, schools, etc.) at one point in time. Usually, the unit of study is not identified (for instance, the data does not include the names of the              individuals who are sampled nor any other identifiers), however, these type of             datasets include variables such as individuals’ sex, age, education, income, place of     residence, etc. An example of a cross-sectional dataset is the publicly available            version of the Australian Census for a given year.

Repeated cross-sectional datasets are a group of two (or more) individual cross-     sectional datasets collected in two (or more) time periods. Although the same          individual can be sampled in both datasets, because the data does not provide          individual identifiers you cannot identify the same individual across datasets.  An   example of repeated cross-sectional data is multiple waves of the publicly available version of the Australian Census, e.g., waves 2011 and 2016.

Longitudinal data is the type of data that follows the SAME individuals over time. Some examples of longitudinal (or panel) data are the Australian Household, Income and Labour Dynamics (HILDA), the U.S. Panel Study of Income Dynamics (PSID), or the German Socioeconomic Panel (SOEP).

BOTH repeated cross-sections and longitudinal datasets can be used to estimate DD models.

Part 2: Study using a DD model

Micronutrient deficiencies plague the developing world. The World Health Organization       estimates that over a quarter of the world’s population suffers from iron deficiency, which     leads to impaired cognitive development in children and reduced work capacity in adults (de Benoist et al. 2008). Renewed interest in combating micronutrient deficiencies in developing countries stems from the potentially large impact of health interventions on productivity and  quality of life. For instance, the Copenhagen Consensus of 2008 recommends iron and iodine fortification as a highly cost-effective development intervention.

The discovery of vitamins and minerals during the early 20th century intensified the public health profession’s interest in the nutritional status of Americans. A number of diet surveys and blood serum case studies during the 1930s showed a widespread prevalence of             deficiencies in iron. Low iron consumption was found in all socioeconomic classes, but the prevalence varied across geographic areas. The United States in the 1930s had rates of iron deficiency similar to those currently found in Turkey or Brazil (de Benoist et al. 2008).

To reduce iron deficiency in the working age and young population during World War II, the U.S. government introduced the fortification of bread with iron in 1943. Soon after the          policy, places with relatively low iron consumption levels before the program’s                      implementation experienced relatively large gains in labour market and schooling outcomes  after the program’s implementation.

A group of researchers were asked to evaluate whether the effects of this sweeping change in federal nutrition policy could be interpreted as a causal relationship. To do so, the group of    researchers exploited three main sources of variation. First, the fact that there were significant pre-existing differences in iron consumption levels across states. Figure 5 shows the large      variation in the proportion of the population that consumed less than the household-specific   recommended iron daily allowance across U.S. states.

Second, the timing of the federal mandate was determined by wartime and technological  constraints in the production of micronutrients. In this sense, the timing was exogenous to other potential confounders related to individual outcomes.

Finally, iron consumption has a nonlinear effect on health. Therefore, a program that            increases iron consumption across the entire population is likely to have disproportionate     effects on the health of those who were previously iron deficient (Hass and Brownlee 2001).

Using several waves of the U.S. population Census linked to historical data on iron           consumption across states (Figure 5), the group of policy makers estimate a difference-in- difference model on the effects of the policy on individuals’ outcomes. Table 5 shows the results from the model on men’s labour market performance. Each coefficient in the Table comes from a different specification that includes different controls and subsamples.

Questions:

1)  While the researchers did not report a summary statistics table that shows the           characteristics of states and individuals in areas with low vs. high levels of iron        consumption before the program’s implementation, explain what kind of differences would you expect to find across these two groups and why.

Solution: A possible version of a summary statistics table would likely show that       areas with low vs. high levels of iron consumption looked different across many         dimensions before the program’s implementation. For instance, it is possible that        areas with high levels of iron consumption at baseline could have had higher levels of income per capita, higher levels of education, better health outcomes, higher health     care spending, better public infrastructure, etc…  than areas with low initial iron         consumption. We expect these differences in observable characteristics because it is   likely that iron consumption before the program’s implementation was likely              associated with better economic and social characteristics of these areas or would       likely be another “indicator” of their economic/technological development.

2)  Explain how you would use the identification strategy proposed by the group of         policy makers to investigate the effects of the implementation of the nutritional policy on adults’ labour market performance. Describe any control variables you would        include in the model.

In particular, include the regression equation (or otherwise) you would estimate: Yist = alpha0 +

where Yist is the outcome of interest (i.e., employment, earnings) for individual i, in state s, in year t,...

Discuss the assumptions that are needed to interpret the results as a causal effect of  the policy and any tests or support that you would provide for these assumptions.      Please also discuss whether your results would be informative to policy makers in a developing country that is currently considering expanding its nutritional program to low-income adults.

Solution: A possible version of the model would be,

Yist = alpha0  + β1 (IRONs  * POSTt) + Xi  + δs + δt + εist

where Yist is the outcome of interest (i.e., employment, earnings, etc.) for individual i, in state s, and in year t. alpha0 represents the intercept. The variable IRONs denotes   the pre-intervention average iron consumption in state s. The variable POSTt denotes an indicator equal to one if year t is after the intervention date of 1943 and 0 o.w.       Interacting the two variables gives the variable of interest. Xi represents a matrix of   individual level characteristics such as whether individual i is non-white, married,     veteran, etc. (see characteristics included in Table 5). The terms δs and δt are fixed     effects (or dummy variables) at the state and year levels, respectively. The state fixed effects help absorb time-invariant differences at the state level (e.g., the fact that a     state is persistently poor or has had a low provision of public goods) while the time

fixed effects absorb factors that are common to all states but vary over time. For        example, δt may account for the fact that the U.S. experienced the massive WWII      shock from 1939 to 1945. ε is the error term and errors are clustered at the state-level to account for within-state serial correlation in the observations.  The coefficient of   interest is β1 .

To arguably provide causal evidence on the effects of iron fortification on individual  outcomes, the DD model relies on the assumptions that: First, treatment (states with   low iron consumption before program implementation) and control (states with high   iron consumption before program implementation) groups evolve in parallel trends.    Second, pre-existing geographic differences in iron consumption across states are not correlated with state heterogeneity in omitted characteristics that induce changes in    the outcome and so, AFTER conditioning the model on state and temporal fixed         effects, any variation in the interaction term (IRONs  * POSTt) is exogenous or random (recall the CIA assumption) and so the coefficient β1 captures the effect of the policy  on the outcome of interest.

The results would be informative to settings with similar pre-treatment characteristics that are considering the implementation of the iron-fortification program. Researchers sometimes say that the United States prior to the 1970s looked like a developing         country now, so, the results from this intervention could be partially informative to     developing nations today. However, it’s important to be cautious in how/if to

extrapolate the findings to other contexts as there may other potential                    conditions/shocks/trends that may vary and influence the program’s success (e.g., culture).

3)  Using Table 5:

a.   Explain why the coefficient of interest is negative?

b.   Describe the results obtained on men’s labour market outcomes.

Solution:

a.   We expect the coefficient of interest to be negative because areas with higher levels of iron consumption before the program’s implementation would         experience smaller health benefits from fortification and therefore smaller     gains in labour market outcomes.

b.   Table 5 presents point estimates for β 1, the coefficient on (IRONs  * POSTt).   The point estimate suggests that iron fortification led to statistically and         economically significant relative gains in income for men in areas with lower iron consumption. The base estimate for men in column 4 suggests a one        milligram increase in the average iron consumption is associated with a 1       percent difference in income growth. This estimate is only statistically            significant at the 90 percent level. The results on income are broadly              consistent with relative changes in weeks worked correlated with low pre-      intervention iron consumption, although the estimates are noisy as shown in   columns 8- 10. Little effect is observed on labour supply (columns 5-7), likely due to the fact that labour supply may be already very high with for men.