Movin’ on up? A Survey Experiment on Mobility Enhancing Policies

We use a nationwide survey experiment in the United States to measure whether information on intergenerational economic mobility or policy-specific arguments influence support for six pro-mobility policies advocated by political entrepreneurs. We find the information treatments do not affect support, but the argument treatments significantly increase support for three of the policies. We also include a behavioral measure by allowing respondents the opportunity to write their U.S. Senators. We find argument treatments significantly increase the likelihood that letters address economic mobility and significantly promote advocacy for that policy in the letter, but no increase in advocacy from the information treatments.


Introduction
The United States has experienced an increase in income inequality over the last four decades (Piketty et al., 2018).Despite models that predict higher support for income redistribution the higher the level of income inequality (Meltzer and Richard 1981), there is and has been relatively little support for income redistribution in the U.S., especially compared to other developed countries (Ashok et al., 2015;Niehues 2014).While part of the low support for redistribution may be due to misperceptions of the income distribution--Americans tend to underestimate income inequality (Gimpelson and Treisman 2018)-providing them the correct information does not much affect redistributive preferences (Kuziemko et al., 2015). 1  An alternative explanation for low support for redistributive policies is that Americans have either experienced or believe in the "prospect of upward mobility."If today's middle and upper class have experienced upward mobility in the past (Piketty 1995), or today's poor expect to exceed the status of their birth (Benabou and Ok 2001), both experience and anticipation may lower support for redistribution today.There is evidence that intergenerational mobility leads to tolerance of inequality.Shariff et al. (2016) find in cross-country regressions that there is less concern regarding income inequality in countries that have higher father-son intergenerational income mobility.When they experimentally induce beliefs that intergenerational economic mobility is higher among U.S. residents, they also find that those treated find U.S. income inequality less objectionable.Day and Fiske (2017) find perceptions of mobility among subjects in the U.S. are positively correlated with support for the status quo political and economic system.Alesina et al. (2018) examine the relationship between people's views of intergenerational mobility and their support for policies to enhance equality of opportunity or of outcome across several countries.They find lowering people's perception of mobility increases support for the general category of "equality of opportunity policies" only among liberal respondents (converting those who already believe, as it were), and has no effect on views of equality of outcome policies (such as progressive taxation or safety net policies), though their treatment does increase subjects' belief that unequal opportunity is a serious domestic problem.
Their treatment to cause reduced perceptions of mobility is, however, intentionally vague (to move perceptions in one direction), and also does not distinguish between relative mobility (where does one end up in the distribution compared to one's parents) and absolute mobility (is one better off than one's parents).This latter point is important, as one's policy views may be more motivated by the ability to do better for oneself irrespective of one's position (Gilbert 2016) or by positional concerns (Solnick and Hemenway 1998).It is possible that what matters for many voters is not the ability of the poor to move up the distribution of their peers, but to move up relative to their origin.
We examine how information on current levels of intergenerational mobility-both relative and absolute-for mobility-influencing policies.Using an online survey experiment, we measure how information about intergenerational mobility affects people's preferences for redistribution and other activities.Specifically, we test whether the information on relative and on absolute income mobility increases support for these policies.
In separate treatments, we also expose subjects to condensed arguments for one of these same policies framed in terms of their effect on social mobility.These condensed arguments draw on the key points made by policy proponents who link these policies to mobility in op-eds, briefs, or Congressional testimony.We do this for two reasons.First, as mentioned above, neither Kuziemko et al. (2015) nor Alesina et al. (2018) find evidence that information on inequality or mobility (or shifted perceptions of it) much affect support for any policies that may affect the poor.Kuziemko et al. do find that their narrative information on the estate tax increases support for the estate tax.Their study, however, was not designed to answer whether arguments affect people's policy views and it is unclear whether this effect was due to arguments or just the particular policy. 2Ours is the first study, to our knowledge, to systematically investigate the role of arguments and information on both individuals' policy views and their costly effort to write a letter to senator to express their views.And second, as McCloskey (1983) advocates, there is the idea in economics that rhetoric-argument-matters. Thus, we examine the effect of expressly advocating for a policy as well as information alone.
Measuring the change in respondents' support for the six mobility-linked policies after they are exposed to the treatment argument relative to a placebo argument on an unrelated topic (recycling), we find that several arguments significantly increase respondents' support for these policies.Specifically, arguments for additional cash assistance to the poor, for housing vouchers for low-income families to move to middle class neighborhoods, and for universal pre-kindergarten programs all increase support for those policies significantly more than that of the respondents in the control who only view the placebo argument.Evidence shows that these effective arguments are also those with more solid economic research support. 3In contrast, the information treatments do not statistically significantly change policy positions relative to the control.
In addition to examining changes in views, we also provide respondents an opportunity to demonstrate behavior related to those views.The last question of the survey is an open-ended prompt for respondents to write a letter to their U.S. Senators.Kuziemko et al. (2015) provided sample letters for respondents, but only ask them whether they would write their elected officials.Our subjects, in contrast, choose whether to write their Senators by doing so in the experiment itself.Unlike our previous questions, which respondents must answer in order to receive payment for the survey, they may freely skip this question.Yet roughly one-third of subjects chose to write their senators, and while there are no differences in the proportion of letters written across treatments, there is considerable difference in the content of these letters.
All but one argument treatment (lowering immigration 4 ) resulted in a higher proportion of letters mentioning economic mobility, and all argument treatments caused a higher proportion of letters to comment on the specific policy in that argument.Respondents in all treatments are between 3.5 percent (increasing marriage tax credit) and 11.3 percent (increasing the minimum wage) more likely to mention the policy they were exposed to compared to their counterparts in the control.Additionally, except for two policies (the marriage tax credit and reducing immigration), more than three quarters of these mentions advocate for the policy in question.The information treatments, in contrast, did not increase the likelihood of mentioning economic mobility at all, and were only statistically significantly related to the mention of one policy: subjects in the relative mobility treatment were less likely to discuss immigration policy than the control group.Arguments, in short, matter.
Our paper proceeds as follows: in section 2, we describe our experiment and hypotheses.In section 3, we discuss our data, and relate our pre-treatment policy views to the larger literature on support and opposition to redistribution.Section 4 presents the analysis 2 Di Tella et al. (2012) find that priming information where they let the participants read some government propaganda statements (like arguments) can overwhelm the effect of first-hand experience on changing people's privatization beliefs.While related, their paper was not trying to study the role of arguments regards a policy on changing people's beliefs or behaviors.
3 Our selection of arguments is based on policy proponents who link these policies to mobility.Half of these policies have more solid research support while the other half do not.We discussed this further under Hypothesis 2. 4 Recent work by Martinangeli and Windsteiger (2019) shows that highlighting the number of immigrants in Germany has no effect on demand for redistribution among Germans, though this is due to low-income (middle-income) earners increasing (decreasing) support for more progressive taxes (but reversing these roles in terms of higher education spending) when given information on the level of immigration.
of our experiment; we discuss our results in section 5.

The survey experiment
We conducted the survey in April 2019.All treatments had the following structure: 1) introduction and consent, 2) background demographic and socio-economic questions, including two questions on their charitable behavior and their view the roles of luck and effort in people's success, 3) their initial views of six mobility-related policies, 4) one of nine randomly-assigned conditions, 5) their posterior views of the same policies, and 6) an opportunity to write a letter to their U.S. Senators.The detailed survey is in Appendix B.

Data collection
We posted the survey on Amazon Mechanical Turk (mTurk) with a description stating that the survey would take roughly 10 min, and respondents would receive $1.50 for completing the survey (a $9.00 hourly wage, approximately).Respondents were free to drop out at any time or take up to 24 h to work on all the questions.In fact, except for roughly 40 respondents who started the survey toward the end of our sampling period and saw we had reached our posted quota, all subjects who started the survey chose to finish it. 5The median (mean) response time was 6.01 (7.17) minutes.
Several steps were taken to ensure the validity of the results.First, we restricted our sample to those workers Amazon has verified as U.S. residents.We did not, however, choose to work with only "masters qualified" mTurk workers to avoid frequent (if otherwise reliable) survey experiment participants in favor of less experienced respondents. 6Second, we launched our survey during U.S. business hours to discourage ineligible international respondents.Third, respondents only receive payment contingent upon completing the survey and are required to provide a unique password that is only visible at completion.Finally, aside from the voluntarily answered question on whether to write a letter to their U.S. Senator, we required subjects to answer all questions, and popup windows reminded them to complete all questions in each section before continuing. 7As shown below, our general population sample is demographically similar to the U.S. public, with some anticipatable differences given the nature of the mTurk platform.

Treatments
Our experimental design includes eight treatments (two "information" treatments and six "argument" treatments) and a control condition.In all of these, respondents begin by indicating whether they favor or oppose six policies that one or more political actors (elected official or advocacy groups) have linked to improving economic mobility.These six polices are: 1) raising the minimum wage, 2) increasing cash assistance to the poor, 3) providing housing vouchers to move poor people into middle class neighborhoods, 4) universal pre-kindergarten, 5) marriage tax credits to encourage two-parent families for children, and 6) reducing immigration (both legal and illegal).These are not necessarily policies developed by economists nor found to be effective to promote intergenerational economic mobility, but they have been framed in terms of current or intergenerational mobility by one or more political entrepreneurs. 8Following these questions, respondents are randomly assigned into one of the nine conditions (see Table 1 at the end of section 2.2).

Information treatments
The information treatments-Relative and Absolute-use a short interactive task to elicit respondents' beliefs about either relative mobility or absolute mobility, respectively, though they do not elicit any information on the respondents themselves.The Relative task asks respondents to indicate what fraction of children born in the bottom 20 percent of the income distribution in the 1980s end up in each income quintile as an adult today, and also what fraction of children born in the top 20 percent end up in each income quintile today.The Absolute task, in contrast, asks respondents to estimate what fraction of children born in each income quintile in the 1980s earn more today than their parents earned when the children were born.That is, the absolute task compares between children's 5 Unfortunately, we cannot do a meaningful analysis of attrition by treatment, as only four of these subjects reached the treatment questions (and thus were assigned to a treatment) before leaving the survey. 6Stewart et al. (2015) estimates the turnover rate on mTurk at roughly 26 percent per quarter, suggesting that workers with the masters qualification are more attentive, but also considerably more experienced, than the typical subject available on the platform. 7This last choice is not uncontroversial, as forcing responses could lead to attrition if respondents would prefer not to answer a question (e.g., on a private or sensitive topic), or to lower data quality.For this reason, we opened with questions on demographic and personal economic characteristics (such as income) that may make some respondents uncomfortable, so that any attrition we do experience may affect the external validity of our data, but not the internal validity of the experiment.As for data quality of the remaining subjects, Albaum et al. (2010) experiment on the use of both forced-answer and the inclusion of "prefer not to answer" options for individual questions.They find the forced-answer treatment has the highest completion rate, and that data quality is most negatively affected not by forced-answer, but by the option to not respond.Furthermore, Smyth et al. (2006) find that forced-choice questions provide higher quality data than "choose all that apply" questions (which allow subjects to choose none of the options), suggesting that forcing a response may lead respondents to consider their answers more carefully.absolute earnings and that of their parents, while the relative task compares between the rank of children's earnings and the rank of their parents' earnings.Following the respondents' guesses, we show them how their estimates compare to the actual data from Chetty et al. (2014) for relative mobility and Chetty et al. (2017) for absolute mobility, with the data presented both textually and graphically. 9We provide these treatments in the appendix (Figure A.1).
We have treatments on relative and absolute mobility because the two measures provide different viewpoints on intergenerational mobility today, and as such may indicate different levels of concern for improving mobility through redistributive policies.The information on relative mobility indicates that few children from the bottom quintile make it to the middle class or better, and few children from the top quintile fail to stay in the top half of the distribution, indicating little mobility by a relative standard.But examining the same people from an absolute perspective allows for a different narrative: it shows that there is more absolute upward mobility for poor children-most children of the bottom quintile out-earn their parents, while most children of the top do not.

Argument treatments
In addition to the two information treatments, we have six argument treatments.Each treatment is an explicit argument for one of the six policies styled similarly to how proponents of that policy tie it to economic mobility or poverty reduction in policy briefs, opeds, or speeches, alongside a graphical presentation of evidence for the policy drawn from policy entrepreneurs' arguments.Note that we are not claiming these policies will have the effect of increasing social mobility or decreasing income inequality.Rather, there are proponents of these policies who have framed arguments for each policy in these terms, and we have adapted their arguments and evidence into six treatments.Each of these treatments is accompanied by a graph and the argument is always framed in terms of increasing social mobility.All treatments are standardized for length (between 159 and 162 words, about the length of a typical abstract); we provide the text and accompanying graphic for all arguments in the appendix (Figure A.2).

Treatment design
To ensure that it is the content of each treatment, and not the type of activity (interactive guessing or reading arguments) or the length of time that drives the result, we pair each information treatment (Relative and Absolute) with a placebo argument and each argument treatment with a placebo interactive information task.Following Nickerson (2008) use of recycling as a "placebo" (as opposed to an uncontacted control group), we also employ an argument for recycling as a placebo argument for the two information treatments.Similarly, we use an interactive information task about recycling as a placebo for the six argument treatments (specifically, subjects guess what fraction of various products, like lead batteries or newsprint, are recycled).We chose recycling as a relatively "neutral" issue, both based on Nickerson (2008) and on its overwhelming popularity across partisan lines (Pew Research Center 2009), though other survey evidence shows a partisan divide on recycling consistent with differences in the parties toward environmental issues generally (Coffey and Joseph 2013).To ensure consistency in terms of the order of activities, each treatment begins with an interactive task and ends with an argument. 10 Our Control condition, therefore, is the pairing of both the placebo information task and the placebo argument.Absolute and Relative treatments each start with their respective treatment information tasks followed by the placebo argument.As a result, any comparison between Absolute (Relative) and Control identifies the effect of that information treatment alone on responders' policy views.Similarly, all argument treatments start with the placebo information task, and then follow with one policy argument.Thus, a comparison Note: An example of one treatment (minimum wage) can be found in the link below.https://bryant.qualtrics.com/jfe/form/SV_agHhC97GV0WhMJo.
9 Early papers on economic mobility, both relative and absolute (e.g., Pew (2012)), use data from the Panel Study of Income Dynamics.We chose to use the Chetty et al. (2014Chetty et al. ( , 2017) ) data for comparison for both absolute (2017) and relative mobility (2014) as these use population-level administrative income data rather than self-reported income data from a sample, and due to the consistency between their main results and their sensitivity analyses. 10As such, respondents in the information treatments perform the mobility-related information task first and then read the placebo argument, while those in the argument treatments first perform the placebo task and then read the mobility-related argument.While it is possible that there is some effect due to the order in which the placebo and mobility content are encountered, the treatment information or the treatment argument effect are always compared to the Control, which also preserves the task-argument order.
between any Argument treatment and the Control identifies the effect of that policy argument on respondents' policy views.Table 1 summarizes the contents of our treatments.Finally, one additional concern with our design is that of experimenter demand effects (de Quindt et al., 2018;Zizzo 2010).Unlike even the "weak demand effect" treatment in de Quindt et al., 2018, our language in the instructions is neutral and gives no indication of what we hope to observe, and unlike concerns over social demand effects in Zizzo (2010), we hold considerably less authority over our subjects than does the typical faculty-experimenter in an in-person laboratory setting.Our subjects are anonymous mTurk workers that we do not meet and whom we pay provided they complete the survey (irrespective of how they do so).We are not physically present with them and thus give them no cues by our verbal or nonverbal communication what we hope to occur.And while the style of the treatments may suggest a direction for their opinions to move, the desire for self-consistency in answering the policy question likely biases us toward finding null effects (Saris and Sniderman 2004).Thus, while important, we do not believe experimenter demand effects play a large role in our results.We revisit the topic of demand effects in the analysis of respondents' letters to their U.S. Senators.

Hypotheses
We test whether information on intergeneration mobility affects support for each of the six aforementioned policies.Our naïve hypothesis is that the Relative treatment increases support for mobility-linked policies relative to the Control.As mentioned above, Cruces et al. (2013) and Karadja et al. (2017) find that sharing information on individuals' position in the income distribution affects their support for redistribution in the abstract (i.e., not specific policies), though Alesina et al. (2018) do not.The Absolute treatment may also increase support for redistributive policies, as most children of the 1980s fail to out-earn their parents.
Hypothesis 1. Information treatments increase support for mobility-linked policies relative to the Control.
Despite our naïve hypothesis, there are several reasons why these two treatments may fail to move respondents' views on redistribution.First, as noted above, the Absolute information shows the least well-off children being the most likely to rise above their parents.Despite the fairly obvious interpretation of regression to the mean (which we do not introduce to our subjects), learning this pattern of absolute mobility may temper, rather than bolster, respondents' enthusiasm for redistribution.11Second, Day and Fiske (2019) argue that beliefs about economic mobility (which our treatments aim to influence) affect abstract concerns regarding inequality more than they change support for specific policies.
Finally, this information should matter to respondents only if it makes them believe mobility is lower than they previously thought.Some recent research indicates that United States residents tend to underestimate the level of intergenerational economic mobility in the U.S.12 (Cheng and Wen 2019;Chambers et al., 2015).If that holds true in our sample, and respondents link their estimate of mobility to their support for mobility-enhancing policies, then updating respondents' views of the level of economic mobility may lead to reductions in their support for mobility-enhancing policies.Furthermore, when respondents hold accurate beliefs on mobility, then this information will not affect their support for these policies.
Hypothesis 2. Argument treatments increase support for the mentioned policy compared to the control.
As one might expect from including only arguments for the policies, we expect that an argument for a policy raises support for that policy.The only treatment that consistently affected respondents' views in Kuziemko et al. (2015) was their narrative information about the estate tax.Recent work by Alesina et al. (2019) suggests that salience and narratives shape people's views more deeply than facts alone.They find that anecdotal evidence about a hard-working immigrant has a greater effect on people's preferences for redistribution than does factual information about immigrants.Additionally, while views on public policy are often correlated across issues, there is no reason ex ante to expect an argument to support universal pre-kindergarten to influence one's views on immigration.Thus, we do not expect "spillover" across policies.
That said, while we chose six policies whose proponents have linked to intergenerational economic mobility and standardized them as best we could, all arguments are not created equal.In particular, the National Academies of Sciences ( 2019) finds good evidence of the poverty-reducing effect of the minimum wage, housing vouchers, and cash assistance, though it makes no claims about the effect on mobility.There is evidence that cash assistance improves the next generation's economic outcomes (Aizer et al., 2016) or outcomes that subsequently affect economic mobility (Akee et al., 2010;Cooper andStewart 2013, 2017). 13 Chetty et al. (2016) find strong evidence of the mobility-enhancing effect of housing vouchers.And results on universal pre-kindergarten indicate both the impact of such programs on early educational outcomes that can affect later learning (Cascio 2021) and the importance of pre-kindergarten for future earnings (Bartik 2014, ch. 2, for a review).
In contrast, Zimmerman (2011) finds little evidence supporting a link between the minimum wage and economic mobility.Although Chetty et al. (2014) find evidence that U.S. neighborhoods with higher proportions of two-parent families exhibit higher intergenerational mobility, there is little direct evidence linking marriage to mobility, and the National Academy of Sciences' (2019) review of the evidence finds little to support a link between promoting marriage and poverty reduction.And while Zimmerman (2008) finds that on balance, immigration to the U.S. reduces the wages of the lowest-skilled Americans, there is little to tie levels of immigration to economic mobility. 14t is at least possible that respondents are either aware of the differential quality of evidence across arguments or are sufficiently discerning to pick up on the differential quality of the evidence we provide.In either case, this would mean that we only see an impact of our treatments on "well-supported" policies (namely, housing vouchers, universal pre-kindergarten, and to a lesser extent, cash assistance and the minimum wage), and not on our "less-supported" policies (tax credits to support two-parent families and reducing immigration).

Data
We collected 2442 responses.As noted above, 43 respondents chose not to complete the survey, and only four of these chose to leave the survey after being assigned to a treatment.Thus, we restrict our analysis to the remaining 2399 observations.Table 2 presents the socioeconomic and demographic characteristics of our sample, both overall and by treatment.In this section, we examine our data for the purposes of both internal and external validity.We first compare our sample to the U.S. adult population where possible, and then discuss our checks for balanced covariates across treatments.Finally, we examine the relationship between our subjects' support for each policy as a function of covariates and compare these with findings in the literature.

Sample characteristics compared to the U.S. Population
The first column of Table 2 presents data on the U.S. adult population from a combination of the 2018 wave of the American Community Survey and from a wave of the Gallup Poll conducted the same week as our survey.
Our survey sample is similar to the U.S. adult population, with some anticipatable departures consistent with previous research (see Berinsky et al., 2012;Boas et al., 2020;Heen et al., 2014).The sample has more males and is considerably younger than the U.S. adult population (though our average age is near the overall U.S. median age of 38.2).Our respondents also have a lower income than the national average household income. 15The sample is about as likely to be married as the national average, though consistent with their youth, they are much more (less) likely to have never married (be widowed or divorced) than the overall population.
Also consistent with previous samples from mTurk, our sample is whiter than American adults today (with fewer Hispanics and African Americans in particular), and considerably more educated.Less than ten percent of our sample has less than a high school education, while over a third of American adults have not completed high school.Also consistent with the age of our sample, our respondents are much more likely to be employed and much less likely to be retired or out of the labor force.Finally, while we have roughly the correct proportion of Democrats, we have relatively too few Republicans compared to the distribution nationally.Thus, our data on average levels of support for these policies are likely biased toward support, though our focus is on the change in respondents' views and not the level of support.

Covariates on predicting the treatment status
As the rightmost nine columns of Table 2 demonstrate, there are some differences in covariates across treatments.We check the balance of our treatments in two ways.First, we estimate eight separate regressions of how the covariates listed in Table 2 predict assignment to each information or argument treatment relative to the Control.That is, we estimate Treatment i = X i β + ε i , where Treatment i = 0 if the respondent is assigned to the Control and Treatment i = 1 if the respondent is assigned to the particular treatment.
Table A.1 (see appendix) shows the results of these regressions.Covariates are generally balanced between each treatment and the control, save for the proportion of divorced subjects (which is higher in the control than in mostly any other treatment).Most importantly, none of the variables on respondents' views (political party, charitable giving, and the role of effort and luck in success) are related to assignment to treatment.We also run a multinomial logit of assignment to treatment as a function of covariates (see appendix, Table A.2), and find similar results-aside from divorced subjects, treatments appear to be well-balanced on observable characteristics.That said, we present our results in section 4 below both with and without covariates.

Pre-treatment support for redistributive policies
Table 3 shows respondents' average policy views in each treatment prior to exposure to the treatment.Each row represents a treatment, and each column a particular policy; Panel A of Table 3 reports the average of the five-point Likert scale, and Panel B reports the average of a dummy variable equal to one if the respondent "strongly favors" or "favors" the policy, and zero for the remaining three categories ("neutral", "oppose", and "strongly oppose").In Panel B, we see that respondents favor all policies on average, save for Full time is total the employed percentage less the part-time percentage.We divide "not in labor force" from the Bureau of Labor Statistics between "not in labor force" and "retired" based on proportion of "not in labor force" that was retired in Hipple (2015).
reducing immigration, with sizeable majorities favoring increasing the minimum wage, more cash assistance for poor households, and universal pre-kindergarten.These numbers are likely higher than they would be if we had fewer independent and more Republican respondents, but not excessively so (e.g., an NPR/PBS Newshour poll from July 2019 finds that 56 percent of Americans favored increasing the national minimum wage to $15 an hour, and minimum wage referenda in both Missouri and Arkansas-Republicanleaning states-passed with more than 60 percent of the vote in 2018).
We regress respondents' policy views for each policy as a function of respondents' covariates from Table 2, where views are the five-point scale from 1 (strongly oppose) to 5 (strongly favor).We present and briefly discuss the results in Appendix Table A.3.Our results are largely consistent with findings from others who examine correlates of preferences favoring redistribution using nationally representative U.S. samples (Alesina and Giuliano, 2011;Alesina and La Ferrara 2005) or examining populations elsewhere (e.g., Ravallion and Lokshin 2000).
In short, it appears that our subjects' demographics, their views, and the relationship between these are similar to the U.S. adult population, and that we successfully randomized respondents across treatments.Additionally, other researchers have replicated previously published results using mTurk samples across a variety of topics from framing effects (Berinsky et al., 2012) to decision making biases (Goodman et al., 2013).Clifford et al. (2015) compares the mTurk sample to two benchmark national samples and finds substantively similar results with only minor variations in effect size.Thus, our experimental data is not without internal and external validity.That said, while we have attempted to demonstrate our sample's similarity to the U.S. population and others have demonstrated its external validity in political research, this is a study of U.S. individuals' preferences over redistributive policies, not a nationally representative sample or citizens voting in an election.With that in mind, we turn now to our results.

Results
We first examine the effect of our treatments on respondents' policy preferences, and then analyze their choice to write a letter to their U.S. Senators.

Policy preferences
Table 4 presents the differences in respondents' policy views before and after the treatment, by treatment (rows) and policies (columns).Recall that policy views are a 5-point scale from strongly oppose (1) to strongly favor (5), so positive numbers indicate an increase in support, while negative numbers indicate a decrease in support for a policy.We measure statistical significance here as a pairwise t-test between the pre-and post-view on each policy.
To correct the family-wise error rate for multiple comparisons, we apply Sankoh et al. (1997) correction to all p-values in this and subsequent tables, unless otherwise noted. 16When outcomes are perfectly uncorrelated, this adjustment is equivalent to the Bonferroni adjustment.This is a somewhat more restrictive correction than other multiple comparison corrections, though it only takes into account the number of outcomes and not the number of treatments.
We see limited effects from both information treatments.Aside from the relative treatment leading respondents to become less supportive of reducing immigration by roughly a fifth of a point on the scale (roughly one-sixth of a standard deviation, p < 0.01), there are no statistically significant changes in respondents' views from either the absolute or relative mobility information treatments.
Two arguments have statistically significant effects (i.e., p < 0.05) in the expected direction.Arguments for increasing cash assistance to poor families and for housing vouchers lead to statistically significant increases in support post-treatment relative to respondents' pre-treatment views; these correspond to roughly a tenth and a sixth of a standard deviation change in policy views relative to pre-treatment views (from Table 3), respectively.The argument for universal pre-kindergarten also makes respondents more supportive by about a tenth of a standard deviation, though this effect is weakly statistically significant (p = 0.066).
The remaining arguments either fail to influence views on that topic, or, contrary to our expectations regarding policy "spillover", lead respondents to become less supportive of other policies.Specifically, arguments for the minimum wage and for universal prekindergarten cause respondents' support for reducing immigration to fall, while the argument to reduce immigration leads subjects in that treatment to become less supportive of housing vouchers.
While we do not observe statistically significant changes in views for any policy among members of the control group, their views do not literally remain unchanged.It is for precisely this reason-that respondents' views may change following brief consideration of a topic-that we included an information-based control condition in our design.For our subsequent analysis, we estimate the following regression: where PolicyChange is the difference in support at the individual level as measured in Table 4, Treatment i is a set of binary variables for the two information and six argument treatments, and X i contains the covariates described in Table 2 as well as state-of-residence fixed effects.Unlike the "raw" results above, then, our regressions capture the movement in respondents' views due to the treatment compared to the movement in views of respondents in the control (who, as explained above, perform an estimating task and read an argument regarding recycling).
Table 5 presents the effects of our treatments on each policy.The top panel excludes the covariates, while the bottom panel includes them.The results between the two panels are essentially unchanged by the inclusion of covariates.Information on both absolute and relative mobility fails to move respondents' views relative to the control on any policy.Consistent with the raw results in Table 4, arguments for cash assistance, housing vouchers, and universal pre-kindergarten all cause respondents to become more supportive of those policies.The argument for housing vouchers moves respondents by roughly a quarter of a standard deviation of pre-treatment views on the policy. 17Only one treatment has any statistically significant effect on support for another policy: the argument for cash assistance weakly significantly increases support for the minimum wage.All other arguments did not lead to statistically significant revisions in positions for our respondents relative to those in the control condition.We turn now to an analysis of respondents' messages to their U.S. Senators.

Letters to respondents' senators
At the end of our survey, for all treatments, we asked respondents whether they would like to write a letter to their U.S. Senators. 16Specifically, we adjust all p-values using the formula p adjust = 1 − (1 − p i ) gi , where p i is the unadjusted p-value for the ith policy, g i = K 1− r− i , K is the total number of outcomes (in this case, six dependent variables), and r − i is the average correlation of all outcomes (dependent variables) other than i. 17 We obtain qualitatively similar results if we use change in support pre to post with support measured as a binary variable as in panel B of Table 3; we present these results in Table A.4. of the appendix.Changes in views range between four and eight percentage points relative to the change in the control.None of these differences are statistically significant after correcting for multiple comparisons, however, as they do not capture movement within non-supporters or supporters (e.g., from strongly oppose to oppose, or favor to strongly favor).
We made it clear that this was entirely optional.Roughly 62 percent of respondents chose to write nothing and end the survey.921 respondents, however, chose to write something.We describe how we analyze these messages below.

Letter content coding
We first created a binary variable for whether the respondents wrote anything at all, however short or devoid of content (any letter).Of the 921 respondents who wrote something, 283 of them wrote messages that were self-evidently not to their U.S. Senators.Some of these were a response to the prompt (e.g., "no thank you") or to us as researchers (e.g., "i like the survey"), or were fundamentally without content (e.g., "Rjiddoseoidjdjdjid") but were clearly not directed to their elected representatives.Without knowing what treatment these came from, we created an additional binary variable equal to one if the respondent wrote a "substantial" message.
(real message) and zero otherwise, and then we separated the 283 empty messages (about 27 percent of respondents) from the remaining 638 messages. 18 Following our initial sorting of the messages, we hired two research assistants to code the content of the remaining 638 messages.These research assistants did not meet with one another, and only knew the prompt for the letter, not the content of the survey nor our hypotheses.The research assistants coded each of the 638 messages on whether the message: 1. Referenced intergeneration economic mobility, income inequality, or poverty? 2. Advocated for any specific policy at all? 3. Advocated against any specific policy at all?Under the first question (any mobility), the research assistants coded the message as either referencing or not referencing mobility, inequality, or poverty.For each of the next two questions, the assistants were provided a list of nine choices.These choices allowed the research assistants to indicate that the respondent did not advocate for (or against) any policy, or that the respondent advocated for (or against) any combination of the six policies, for (or against) "environment-related policies such as recycling", and an "other" category.In the "other" category, we asked our assistants to describe the policy the respondent discussed in a few words. 19We instructed them to choose codes for all policies that apply for each message.
We present the percentage agreement between coders as well as two measures of inter-rater agreement - Cohen's (1960), Kappa and Gwet's (2008) AC 1 as implemented in Stata by Klein (2018) in appendix Table A.5.While there is some disagreement between sources on the relevant cutoff for acceptable agreement measured by either statistic, most authors view values greater than 0.60 as a reasonable level of agreement (see Wongpakaran et al., 2013;Table A.5).All our topics-from the catch-all "any mention of economic mobility" to the specific mentions for or against each policy-display very high levels of agreement, and reasonable levels of inter-rater   18 To be clear, we only separated out messages that (a) indicated a disinterest in contacting the Senator, (b) clearly communicated with us, or (c) contained non-word characters.Thus, "real" messages may still be cut-and-paste from Wikipedia, or logically incoherent, but they are not obviously attempting to decline the prompt or to communicate with the experimenters. 19A review of the handful of "other" codes used indicates respondents often conflate state or local and federal government.The set of all messages and our assistants' codes is available upon request.
agreement according to the AC 1 . 20Based on these high levels of inter-rater agreement, we use the average code of our raters for each variable we define below. 21 Before turning to our analysis of the assistants' coding of the messages, we also examine the issue of demand effects in the letterwriting outcome.Specifically, we measure the degree to which the messages from our respondents to their Senators simply replicate the arguments they were exposed to. 22We performed a latent semantic analysis (LSA) of all letters within each treatment and Notes: The participants report their views on a 1-5 scale (1 is strongly oppose, and 5 is strongly favor).The covariant included in the regression are presented in Table 2 and state-of-residence fixed effects.All N = 2399.Robust standard errors in parentheses.***p < 0.01, **p < 0.05, *p < 0.1, corrected using Sankoh et al. (1997). 20In one instance (against environmental policies), our Kappa statistic is undefined because of the high levels of inter-rater agreement combined with the way in which our assistant-raters disagreed.This paradox of Cohen's Kappa is documented by Zec et al. (2017)-and is why we also present Gwen's less frequently used, but also less biased measure of inter-rater reliability. 21Our results are considerably strengthened if we code a topic as "present" if any assistant coded it as present, and qualitatively similar (though generally not always statistically significant) if we code a topic as "present" only if both assistants code it as present.We provide these replications of Tables 6 through 8 using each coding method in Tables A.7 through A.11 in Appendix A. 22 One possibility we can likely rule out is that respondents copied and pasted the arguments to respond to the letter-writing prompt.Once respondents completed a section of the survey, they could not go back.Also, they were not aware of the letter prompt while they were reading the arguments.
examined the average cosine similarity (a measure of the similarity of two documents, for details, see Schwarz (2019)).The cosine similarity between two documents can vary from − 1 to 1, with 1 representing identical documents and − 1 representing completely unrelated texts.The average cosine similarity between the argument in a treatment and all letters written in that treatment ranged from 0.07 for respondents exposed to the recycling argument to 0.15 for those exposed to the cash assistance argument.While there is not a critical value for claiming cosine similarity, these are lower values than typically used to indicate two documents are similar (Evangelopoulos et al., 2012).Furthermore, while all arguments trigger more messages written on that topic as we show below, both the arguments for marriage tax credits and for reduced immigration lead to more opposing than supporting messages.Because of the low levels of similarity from the LSA and the fact that several treatments generate messages against the policies argued for, we believe that these are genuine messages from respondents, not relics of demand effects.

Analysis of coded letter content
We regress the binary variables for any writing, writing directed to respondents' Senators, and writing on mobility, inequality, or poverty in Table 6.The regressions take the same form as those in Table 5, save for the change to the dependent variable.As the first two columns demonstrate, the decision to write anything at all or any "real" message does not differ between the treatments and the control.The third column, however, shows that respondents in most of the argument treatments are much more likely to write on mobility, inequality, or poverty than those in the control.Respondents receiving the marriage tax credit and universal prekindergarten arguments are about 7 percentage points (6.6 and 6.9, respectively) more likely to write a mobility-related message than control respondents.Respondents in the minimum wage treatment are about 11 percentage points more likely to write a mobilityrelated message, and those that read the cash assistance and housing voucher arguments are 14 percentage points (13.8 and 14, respectively) more likely than control to send mobility-related messages.
Tables 7 and 8 present regressions on whether the messages include the specific redistributive policies we discuss in the argument treatments and whether the message is in support of the specific policy, respectively. 23 As shown in the first two rows of Table 7, other than the effect of the relative mobility treatment-it reduces mentions of restricting immigration (relative to the control) by 3.8 percentage points-the information treatments did not influence the content of respondents' messages.In contrast, respondents in each argument treatment were much more likely to mention the policy advocated by that argument.The effect of the argument on mentioning that policy ranges from 3.5 percentage points more likely than the control (marriage tax credits) to 11.3 percentage points more likely (raising the minimum wage).Additionally, there are a handful of "spillover" effects of one argument on mentions for other arguments: respondents in the cash assistance treatment are also more likely to mention housing vouchers in their messages, while respondents receiving arguments for the minimum wage and for universal prekindergarten are less likely than the control to discuss reducing immigration (though the latter effect is weakly statistically significant).
Given our framework for coding messages, mentions of a policy are the sum of messages supporting and messages opposing each policy.In Table 8, we see that most of the effect of the experimental treatments on respondents' writing a message about a policy comes from messages in support of that policy.Aside from the effect of the minimum wage on reducing immigration (respondents receiving the minimum wage argument are 3.7 percentage points less likely to favor immigration in their messages than respondents in the control), messages on a policy tend to be in favor of that policy.Almost the entire increase in messages on the minimum wage in the minimum wage treatment (10.6 out of 11.3 percentage points) are respondents in favor of the minimum wage.A supermajority of the increase in messages on cash assistance (7.1 of 9.1 percentage points), housing vouchers (6 of 7.8 percentage points), and universal pre-kindergarten (6.4 of 6.9 percentage points) relative to the control group are also supportive of these programs.Three messages also generated substantial messages in opposition (as a percentage of total messages): the marriage tax credit, reducing immigration, and housing vouchers, though this last difference is weakly statistically significant at best (see Table A6.).
Thus, our argument treatments influenced both respondents' policy views and the content of behavior, if not the quantity of it.Arguments moved respondents toward supporting positions already held by a supermajority of our sample.The arguments also generated significantly more messages related to mobility, but not more total messages.Arguments on each policy led to more messages on these policies, and the bulk of these messages were supportive (though half also generated some opposition).In contrast, our information treatments did not influence respondents' views or their behavior much at all.We examine the robustness of these results, before finally considering possible explanations for the difference in efficacy of our treatments.

Robustness checks
In the appendix, we consider two potential issues with our results above: response quality and heterogeneous effects by political party (see Appendix Tables A.12 through A.20 and discussion).With the former, our results may be less accurate due to possibly lower quality respondents from mTurk.With the latter, our results may be driven by effects within just one party, such that we are "preaching to the choir" in the words of Alesina et al. (2018).On the former issue, we find that restricting our sample to respondents who do not speed through the survey and those whose IP address is geocoded inside the U.S. does not dramatically affect our results above.Thus, poor data quality does not appear to be driving our results.On the latter, we find that the persuasive effects are concentrated among independents and Republicans, not Democrats, when it comes to persuasion, but that the treatments served to push partisans of both parties to write messages.That is, our treatments affect the views of respondents across the political spectrum, but primarily motivate 23 Table A.6 in the appendix contains the results on messages opposing the policies.those already in possession of strong viewpoints on these policies to express their pre-existing views in messages to their Senators.

Discussion & conclusion
We present the results of a survey experiment on Americans' support for redistributive policies.Supplying our respondents with information on either absolute or relative intergenerational economic mobility did not noticeably affect their policy views, their willingness to write to their elected officials, or the content of what they wrote to said officials.Arguments on specific redistributive policies, however, result in statistically significant increases in support for several policies.In particular, the arguments for increased cash assistance to poor families, housing vouchers for poor families to move to middle class neighborhoods, and universal pre- Covariates from Table 2 above and state-of-residence fixed effects included in all columns.Dependent variable in each column is the average of the codes of two research assistants on whether respondent wrote message concerning the policy in the column header (1 = yes, 0 = no).Robust standard errors in parentheses.***p < 0.01, **p < 0.05, *p < 0.1, corrected using Sankoh et al. (1997).
kindergarten all significantly increase support for those policies.And while all arguments lead to a greater proportion of messages to respondents' Senators on those topics, the arguments for raising the minimum wage, cash assistance, housing vouchers, and universal pre-kindergarten result in significantly more messages favoring their respective policies.
Our experiment demonstrates that arguments (particularly those relying on stronger empirical support) can persuade respondents to change their policy views, but it does not speak to the durability of that change.While Gerber et al. (2011) find that television advertisements in gubernatorial campaigns have large initial effects that fade after two weeks or less, Broockman and Kalla (2016) find that door-to-door canvassers were able to reduce subjects' transphobia significantly with a 10-min conversation, and that this effect persisted for at least 3 months.Thus, we suspect that the durability of our persuasion may rely on the message, but also on its matter of delivery; this would make an interesting topic for future investigation.
Furthermore, when we examine the results by party, our results are not driven by strengthening existing partisan differences, but by persuading respondents across the partisan spectrum.Indeed, many of the persuasive effects are largest among Republicans and independents, not Democrats, who are generally ideologically predisposed to redistribution.While the treatment effects on the content of these messages predominantly primes pre-committed partisans to express their views (rather than those with newfound beliefs to express them), the argument treatments nevertheless move the opinions of the previously unconvinced.
But what are we to make of the unresponsiveness of respondents' views to the mobility information treatments?Past research often finds that individuals' vote choice is driven by their evaluation not of their own economic position but of the overall economy (Lewis-Beck and Paldam 2000), and thus it is somewhat surprising that providing respondents with accurate mobility information has so little impact.We consider several possibilities here.
The first and simplest possibility is that the subjects did not attend to the information provided.While we did not perform a manipulation check, experiments that do so tend either to use treatments without direct quantitative information (Alesina et al., 2018) or check differences in views of mobility using a Likert-scaled opinion measure (Day and Fiske 2017).In contrast, we elicit subjects' subjective numerical beliefs, and then immediately provide the accurate information with a direct comparison to those beliefs, making the information obvious and salient.Respondents could choose not to believe-but not to avoid-the information. 24 second possibility is that our respondents did not react much to the information treatment because the information was not that new to them.That is, their ex ante beliefs were reasonably accurate.We have examined our respondents' accuracy regarding relative and absolute mobility.We find that the respondents in the Relative condition slightly underestimate both upward and downward mobility but are overall quite accurate at the task we pose for them.Our respondents in the Absolute treatment under-(over-) estimated the proportion of the poorest (richest) children who out-earn their parents but were much more accurate on the absolute mobility of middle-quintile children.That said, this explanation would imply that those respondents with the largest absolute errors in estimating mobility (be it absolute or relative) have the most room to revise their policy beliefs as well.We have regressed the absolute change in  Sankoh et al. (1997).
providing the correct information in the recycling information activity.One indicated her disappointment at not knowing the correct answers to these types of questions from past surveys.
policy beliefs on the absolute errors in both Relative and Absolute mobility, respectively, and found no economically or statistically significant relationship between the two.Even those for whom the information treatments provide a large correction do not exhibit a larger effect.This insignificant result on information echoes Day and Fiske (2019) claim that social mobility beliefs are unlikely to affect people's attitudes toward concrete policies, let alone their behavior.A third possibility is that, according to theory, it is not the overall level of mobility, but respondents' own experience or expectation of economic mobility that should inform their views on redistribution.While Kuziemko et al. (2015) do not find large effects from correcting individual misperceptions of income inequality, Karadja et al. (2017) and Cruces et al. (2013) find that some respondents change their positions on redistribution when experimental treatments correct their original misperceptions.The former finds among Swedish subjects that those who underestimated their relative position reduced their support for redistribution (particularly if they were predisposed toward right-leaning political parties), but no effect on those who overestimated their position.The latter finds among Argentinians that those who overestimated their position increase their support for redistribution, but effect on those who underestimated their rank in the income distribution.Thus, even the information most directly tied to canonical models of political support for redistribution does not always influence voters as anticipated.
A final possibility, however, is that the information treatments did not affect respondents' views because those views were not based on information to begin with.Traditional models of voting usually treat people as having some knowledge of how policies map to (self-interested or community-wide) outcomes, having some idea of how states of the world interact with policies, and then choosing the candidate or policy that maximizes those outcomes.Increasingly, however, there is evidence that people first choose a political affiliation, adopt policy views based upon this partisan identity, and finally seek information supportive of those views (Achen and Bartels 2017).Lenz (2012) documents across several instances in the United States and other western countries that, far from voters first evaluating policy positions and then choosing candidates, voters first choose candidates (often based on partisan identity), and then change their policy positions to match.Gerber et al. (2010) experimentally induce voters that have not declared a political party to do so.While they do not find that experimentally induced partisan identity affected policy views, it does affect candidate vote choice and retrospective evaluation of recent partisan figures.Barber and Pope (2019) exploit the ideological misalignment of U.S. President Trump with traditional Republican positions in a survey experiment, and find that partisan identification, rather than an internally consistent ideological position, is what drives support on individual issues, especially for low information voters.Even more recently, Akerlof (2020) argues that economists' failure to listen to the "stories [people tell] themselves at the time they make their decisions" (pp.413) has led us to insufficiently appreciate the role of narratives-of arguments-in determining human behavior.By this explanation, individuals often pick a side, absorb the worldview contained in that side's narrative, and then figure out what issues to support based on this story.If processing information through an internally consistent worldview is not what leads individuals to their positions in the first place, it is little wonder that providing them with new information does not lead them to revise those positions.That errors in estimating mobility are not associated with the magnitude of policy position changes is consistent with this explanation.Indeed, we also find nearly no statistically significant relationships between respondents' pre-existing policy views and their estimates of relative or absolute mobility. 25ote, however, that this rationale is distinct from Jonathan Swift's maxim that "reasoning will never make a man correct an ill opinion, which by reasoning he never acquired."Ultimately, what persuades respondents in our experiment is rhetoric-arguments, not just information.If one wants to influence people's views on a policy, perhaps one needs to lay out a rationale for it, not give individual facts and expect recipients to update their views using a model of the world that they may not possess.26This is consistent with Kuziemko et al. (2015) findings on the estate tax, and McCloskey (1983) approach to economic methodology.Political elites may be able to influence voters' positions simply by declaring their own stance (Broockman and Butler 2017;Barber and Pope 2019), but non-elites (i.e., most of us) likely must rely on piecing together a compelling narrative in support of their desired outcome.
Finally, while subsequent research could improve upon this aspect of our experiment, we note that the policies whose arguments respondents found most persuasive were those tied to better evidence.In a world where there is considerable concern over the spread of "fake news", it would be good to know the degree to which individuals lacking issue expertise can nevertheless discern between high-and low-quality evidence in policy arguments.

Disclosure statement: Jared Barton
This research was supported by an on-campus grant from California State University Channel Islands and was reviewed by the IRB at Bryant University.Jared Barton and his close family have no significant financial or in-kind support, nor any paid or unpaid positions from any interested party to this research.No party had the right to review this paper prior to its publication.

Disclosure statement: Xiaofei Pan
This research was supported by an on-campus grant from California State University Channel Islands and was reviewed by the IRB at Bryant University.Xiaofei Pan and her close family have no significant financial or in-kind support, nor any paid or unpaid positions from any interested party to this research.No party had the right to review this paper prior to its publication.

Table 1
Treatment conditions.

Table 2
Experiment sample characteristics by treatment and compared to U.S. population averages.Percentage male over age 18, age, (median) family income, marital status, race & ethnicity, and educational attainment (for population 25 years and over), from American Community Survey 2018) 1-year estimates.Employment status from Bureau of Labor Statistics' May 2019 Employment Situation News Release.Political views from the April 1-9, 2019, wave of the Gallup survey (party affiliation plus leaners).Median age of all U.S. residents is 38.2, while 47 is the average age of the U.S. adult population.The part-time employment percentage (0.099) is calculated based on seasonally adjusted persons in May 2019 who work as part time for economic or non-economic reasons.

Table 3
Pre-Treatment Policy Views (standard deviations in parentheses).

Table 4
Differences in Policy Views (5-point scale) Pre-to Post-Treatment.

Table 5
Change in Policy Views by Treatment Relative to Control, with and without covariates.

Table 6
Message writing by treatment.Covariates from Table2above and state-of-residence fixed effects included in all columns.Dependent variable in column 1 is a binary equal to one if respondent wrote anything in message box for letter to their U.S. Senators.Dependent variable in column 2 is a binary equal to one if respondent wrote an actual message to their U.S. Senators as described in text.Dependent variable in column 3 is average of research assistants' coding of messages as referencing mobility, inequality, or poverty.Robust standard errors in parentheses.***p < 0.01, **p < 0.05, *p < 0.1.(These are not corrected for multiple comparisons.)

Table 7
Messages sent on specific topic by treatment.

Table 8
Messages sent supporting a specific topic by treatment.Covariates from Table2above and state-of-residence fixed effects included in all columns.Dependent variable each column is the average of the codes of two research assistants on whether respondent wrote message supporting the policy in the column header (1 = yes, 0 = no).Robust standard errors in parentheses.***p < 0.01, **p < 0.05, *p < 0.1, corrected using