Activating an evidence-based identity increases the impact of evidence on policymaker beliefs about local climate policies

Evidence-based policymaking has the potential to improve the efficiency and impact of climate mitigation and adaptation policies, but that promise cannot be fulfilled if policymakers fail to change their minds (update their beliefs) when presented with new evidence. Research suggests that individuals often resist changing their mind, especially on polarized topics like climate action. Here we explore whether an ‘evidence-based policymaker’ intervention can reduce resistance when policymakers interpret new information. We hypothesize that, if policymakers wish to see themselves as ‘evidence-based’, reminding them of that identity can make changing their beliefs more comfortable. This is because belief-updating provides an opportunity to affirm their identity as an evidence-based policymaker. In two survey studies of state and local U.S. policymakers—a neutral policy pilot (n = 152) and a polarizing climate policy experiment (n = 356)—we show that the intervention was effective, even when evidence was incompatible with prior policy beliefs or party ideology. This finding suggests that making evidence-based identities salient when presenting new evidence could increase that information’s impact on climate policymaking.


Introduction
The 2022 Inflation Reduction Act allocates $370 billion for climate action in the U.S., an investment that is expected to cut greenhouse gas emissions by 31%-44% below 2005 levels by 2030 5,6,7 . However, the actual impact will depend upon the choice and design of policy tools like incentives for efficient or clean energy technologies. Climate and energy policies can vary widely in their impact and past programs demonstrate that popular approaches may be less (or more) effective than anticipated. For example, the 2009 Recovery Act provided $300 million to U.S. states to provide rebates for efficient household appliances, but roughly 90% of those rebates were claimed by households that would have already purchased efficient appliances in the absence of any incentive (Houde and Aldy 2017). As a result, the cost effectiveness of the policy was far lower than other typical energy efficiency programs (Arimura et al 2012, Knight et al 2022. Fortunately, scholarship continues to illuminate how factors like human behavior, split or perverse incentives, and engineering all contribute to the relative success of these types of policies (Christensen et al 2022). That research suggests new approaches that promise greater climate impacts per dollar of investment (Christensen et al 2021).
'Evidence-based policy'-defined as the 'process of using high-quality information to inform decisions that are made about government policies' 8 -has the potential to improve the efficacy and efficiency of climate policies by helping policymakers use evidence like the scholarship mentioned earlier to identify 'what works' (Davies et al 2000). Although evidence-based policymaking is broadly popular and multiple groups broker knowledge between policymakers and researchers (Bielak et al 2008), a range of barriers to evidence-based policymaking remain. These include fear of the political consequences of evaluation (Goerger et al 2020) and organizational limitations (DellaVigna et al 2022). Our work focuses on another barrier to evidence-based policymaking, psychological biases in how policymakers interpret evidence.
We present experimental evidence that suggests that policymakers resist belief updating when they encounter information about local energy policies that is incompatible with their prior beliefs. We also demonstrate that activating an 'evidence-based' policy maker identity can reduce that resistance. While this work is preliminary, it suggests one potential pathway to increasing the impact of climate policies.

Literature review 2.1. The use of scientific evidence in energy policymaking
An 'evidence-based policy' movement emerged in the 1970s in the U.S. and U.K. to improve the impact of social policy (Bulmer 2015, Rivlin 2015, Head 2016, Baron 2018). Since that time, evidence-based policymaking, the practice of evaluating policy outcomes and using that evidence to inform future policy adoption and expansion, has become broadly popular across parties and across multiple levels of government in the U.S. 9,10 and abroad 11 . While this movement has largely focused on social policies (Sorrell 2007), it has also had a clear impact on energy and environmental policy. Efforts to embed evidence in energy and climate policymaking range from academic efforts to synthesize and communicate research findings 12 to federal executive direction to embed evidence in government action on climate change 13 .

Psychological barriers to the use of scientific evidence in energy policymaking
Past research has identified knowledge synthesis and communication as a barrier to evidence-based climate and energy policymaking (Sorrell 2007). However, even once evidence has been communicated, psychological barriers may limit its impact. These barriers include motivated reasoning Nisbet 2012, Kahan et al 2017) and sensitivity to framing effects (Bechtel et al 2015) which can bias how individuals interpret and assimilate information.
A significant body of work suggests that motivated reasoning blunts the impact of climate change research. Motivated reasoning is a phenomenon where motivations (like a desire to form accurate beliefs or a desire to hold ideology consistent beliefs) shape the process of accessing, constructing, and evaluating information (Kunda 1990). Studies of the public suggest that this phenomenon emerges in the context of politically polarized issues like climate change (Campbell andKay 2014, Guilbeault et al 2018). However, the impact of motivated reasoning on climate attitudes is debated (Bayes and Druckman 2021) and it is not clear that the phenomenon affects how policymakers (rather than the general public) respond to evidence .
At least two studies have sought to establish whether motivated reasoning affects how policymakers respond to evidence and their findings are mixed. In a survey of 954 Danish policymakers on public service outsourcing, Baekgaard et al (2019) found that individuals are more likely to make cognitive mistakes when they encounter facts that are incompatible with their prior attitudes. This biased pattern of misinterpretation is one way in which motivated reasoning can arise (Kunda 1990). However, Lee (2022) conducted a survey of 690 local policymakers in the U.S., presenting them with information about expert consensus on a range of politically contested issues like rent control. That study did not find evidence that participants updated less when that information conflicted with their party ideology.
Apart from motivated reasoning, an emerging literature that specifically studies how policymakers respond to evidence suggests other biases. These include asymmetric updating and insensitivity to effect magnitude. In a field experiment run in collaboration with the World Bank and Inter-American Development Bank, Vivalt and Coville (2020) found that policymakers exhibited greater updating on 'good news' when they were presented with impact evaluations for development programs, an example of asymmetric updating. Using a survey of over 2,000 education policymakers, Nakajima (2021) found that while policymakers do update their views, that change is only persistent and large among those who receive an accessible description of the research methods used. Lastly, Toma and Bell (2022) found evidence of magnitude insensitivity. In a survey experiment of nearly 200 employees from U.S. federal agencies, participants were largely insensitive to the magnitude of impact described in an evaluation. However, Toma and Bell found that two decision aids, one comparing programs side by side, and another that translated impact into cost per person impacted, improved sensitivity to program impact.
The studies discussed earlier generally measure the effect of various types and presentations of evidence on policymaker stated beliefs rather than on their actions. Two field experiments however suggest that evidence can meaningfully affect policymaking behavior. Zelizer (2018) randomly assigned policy briefings for staffers in a U.S. state legislature and those briefings increased bill co-sponsorship 60% above baseline rates on relevant legislation. In an experiment with 2,150 Brazilian municipalities, Hjort et al (2021) found that providing information on the efficacy of taxpayer reminder letters increased actual uptake of the policy by 10%.
We believe that it is important to study actual policymakers because this population is distinct from the lay public. Policymakers differ from the general public in their fixed characteristics (like age and income) and the incentives associated with their decisions (Kertzer 2020). As a result, policymaker belief updating may exhibit distinct dynamics. For example, policymakers might be less prone to biases or mistakes in interpreting evidence due to their expertise and the high stakes of their decisions. The importance of studying decision-making and belief formation in that population has led to increasing numbers of experiments among political elites (Kertzer and Renshon 2022), including work that has found key differences in the magnitude of updating and sensitivity to information presentation between experts (e.g. policymakers and negotiators at Paris COP21) and non-experts (e.g. business students) (Bosetti et al 2017).

The role of directional and accuracy motivations in policymaking
As discussed briefly above, an individual's motivations can shape their responses to information and how they use that information to construct or change their beliefs (Kunda 1990). For example, when accuracy motivations are high (due to financial or reputational incentives), individuals may employ more laborious and less biased decision-making processes (Kruglanski andFreund 1983, Tetlock 1985). However, directional motivations can also shape reasoning. Directional motivation describes the desire to hold a certain belief. These desires can be straightforward. For example, individuals, unsurprisingly, hope that they do not have a life-threatening disease and are better at finding evidence that confirms that hope and less likely to find evidence that dashes it (Dawson et al 2002).
Motivations can also include the desire to hold beliefs consistent with one's worldviews, ideology, and group identity. These directional motivations appear to be influential in the political domain where individuals' sense of themselves is often intertwined with their politics (Greene 1999), and prior research reveals motivated reasoning acting along partisan cleavages (Redlawsk 2002, Taber and Lodge 2006, Cohen et al 2007, Kahan 2012. Research suggests incentives (Kunda 1990, Kraft et al 2015, Peterson and Iyengar 2021, individual differences (Druckman and Leeper 2012, Kahan et al 2017), and context (Bolsen et al 2014, Guilbeault et al 2018 can all affect the extent to which directional versus accuracy incentives will dominate cognition. When directional motivations dominate, individuals preferentially attend to evidence that reinforces their preferred beliefs and ignore, underweight, or denigrate incompatible evidence. When accuracy motivations dominate, individuals use new information to improve the accuracy of their beliefs (Kunda 1990).
Policymakers' incentives could potentially produce directional or accuracy motivations. For example, accuracy motives may dominate if policymakers are motivated by a desire to enact effective policies or believe that the success or failure of a given policy is likely to affect their success in the next election (Fiorina 1978, Weaver 1986, Healy and Malhotra 2013. However, policymakers' perceived electoral incentives may not necessarily encourage objectivity. Directional motivations might prevail if officials wish to conform to party positions since that conformity improves their perceived ability to fundraise or win votes. In addition to electoral incentives, policymakers may have greater directional motivation due to greater identification with their political party (Thomsen 2014)

'Evidence-based' identity may reduce psychological barriers to the use of scientific evidence in energy policymaking
The salience of a specific identity can vary based on context and cues and these changes in identity salience can change behavior (Forehand et al 2002, Reed 2004, LeBoeuf et al 2010, Wong-Parodi and Fischhoff 2015. For example, heightening awareness of a shared 'American' identity increases how much political partisans report liking members of the opposing party (Levendusky 2018). As a result, contexts that make political party salient appear to exacerbate polarized responses to information (James and Van Ryzin 2017, Guilbeault et al 2018), and contexts that invoke a deliberative mindset reduce polarized resistance to information (Már and Gastil 2020). While prior work has shown that identity-protective motivations may produce directional motivated reasoning, the motivation to protect an 'evidence-based' identity could increase accuracy motives.
Evidence-based policymaking appears to enjoy broad bipartisan support among policymakers in the U.S. For example, the Foundations for Evidence-Based Policymaking Act of 2018 passed in the U.S. House of Representatives with 95% of the votes cast 14 . Given this support, there is reason to believe that most policymakers see themselves as 'evidence-based.' However, this identity may be less salient than ideological or partisan identities which are often dominant in policymaking contexts (Cohen 2003). This suggests that the 'evidence-based' identity may be widely held but generally latent. Increasing the salience of that latent identity might increase policymaker updating due to the identity-behavior match. Salient identification as an evidence-based policymaker may transform an otherwise identity-threatening behavior (changing one's mind/updating in the face of challenging scientific evidence) into an identity-affirming behavior, while also orienting the individual toward accuracy rather than directional motivations.

Hypotheses and approach
We used two surveys of state and local policymakers in the U.S. to test whether an intervention that prompts participants to identify as 'evidence-based policymakers' would make them more likely to change their mind when presented with evidence relevant to a proposed policy focusing on local clean energy policies. In both surveys, participants are asked to estimate the impact of a specific proposed policy before and after reading research-based evidence about that policy. We used the difference between those two values as our measure of belief updating. We randomly assigned the intervention, a description of evidence-based policymaking and an opportunity to identify as an evidence-based policymaker, in order to measure its impact on updating.
Based upon the logic elaborated in section 2.4 we hypothesize the following: H1. The evidence-based identity intervention will increase belief updating among policymakers Since we believe that this effect is due to increased motivation to form accurate beliefs, we might encounter ceiling effects if accuracy motives are already high. We expect greater potential treatment effects in contexts where, at baseline, directional rather than accuracy motives are pronounced. Due to this, we expect to observe greater treatment effects when the evidence challenges the participant's priors or their party ideology. This leads to our second hypothesis.

H2. Greater updating will be observed when scientific evidence presented is incompatible with policymakers' party or prior beliefs
A pilot survey of U.S. state legislators was designed to explore the treatment in a policy domain that did not exhibit political polarization. Even though initial attitudes toward the policy were not polarized, we did find differences in updating and response to the intervention between political parties. We designed our second study to explore these dynamics and test how they might shape climate policy. That study, which surveyed local policymakers, randomly assigned participants to learn about a policy expected to be compatible with Democratic policy positions or one expected to be compatible with Republican policy positions.
While these studies were not designed to be an explicit test of directional motivated reasoning, it includes an 'evidence assessment' measure which can be used as an indicator of the phenomenon. This measure captures differences in how participants describe the research findings presented by the survey. For example, participants may indicate that the research indicates a clear (or unclear) relationship between the policy and key outcomes. Baseline differences in the 'evidence assessment' between groups is consistent with bias arising from directional motivated reasoning.
If directional motivated reasoning is affecting policymakers, we would expect the following pattern of results. First, we would expect participants receiving information that is incompatible with their party affiliation or prior beliefs to be more likely to assess the evidence as being unclear rather than definitive. Second, we would expect that the 'evidence-based' treatment would be more likely to increase belief updating when information is incompatible with party and/or priors.

Pilot-neutral policy: opportunity zones
The pilot was conducted with U.S. state legislators and staff. The survey measured beliefs about Opportunity Zones, a popular policy among Democrats and Republicans that provides tax incentives for investing in specific economically disadvantaged areas. Current research on similar place-based tax-incentives suggests that these programs may not produce the pronounced local employment or poverty reduction benefits that they were intended to (Bondonio and Engberg 2000, Neumark and Kolko 2010, Harger and Ross 2016. Due to this dynamic, we anticipated that there would be significant potential for belief updating among policymakers. That potential increases the likelihood that we will be able to observe updating and allow us to test H1, that the treatment will increase updating. The policy has enjoyed bi-partisan support so we did not anticipate that there would be political polarization. Given this assumed partisan neutrality we did not necessarily expect to observe the partisan differences in treatment effect predicted by H2.

Participants and recruitment
We recruited a total of 213 U.S. state legislators and legislative staff from 47 states attending the 2019 National Conference of State Legislators (NCSL) Annual Meeting for the pilot study. Recruitment was based on the procedure described in Anderson et al (2020) 15 . To recruit participants, our research team secured an exhibitor booth at the 2019 NCSL Legislator Summit in Nashville, Tennessee. The booth was staffed with three researchers who approached passing conference attendees in the exhibition hall. The researchers asked potential participants to take a survey about policymaker decision-making and they offered participants a $10 Starbuck gift card as a gift. Participants took the survey in person on tablets provided by the research team. We excluded incomplete surveys and surveys from participants who did not identify as Republican or Democrat, yielding a final sample size of 152.

Experimental design
We randomly assigned participants to either the identity treatment or control condition. We asked those who received the identity treatment to read a brief description of evidence-based policymaking (see S.I. figure 1). We then asked them to indicate if they identify as an evidence-based policymaker and to recall an instance when they used evidence. Those in the control condition described a policy preference but we did not expose them to any mention of 'evidence-based policymaking.' All participants subsequently read a short description of relevant scientific evidence from peer reviewed research papers including citations (see S.I. figure 3) and estimated the positive or negative impact of the policy. We then asked participants to assess the evidence, update their estimate of policy impact, and answer a series of questions about their attitudes and beliefs. We collected the covariate values after the dependent variables to avoid influencing responses on the key outcome measures. Because we collected these measures after participants read the policy information, their responses may have been affected by the increased salience of economic development issues or by the fact that participants may have inferred that the study designers are specifically interested in that issue.

Independent variables
• Evidence-based policymaking treatment. Those assigned to the evidence-based policymaking treatment read a description of evidence-based policymaking and we then asked them about their identification and use of evidence (See S.I. figure 1). We asked those in the control general questions about policymaking (See S.I. figure 2). • Political party. Political Party was measured by asking participants to respond to the question, 'Generally speaking, do you usually think of yourself as a Democrat, Republican, or an Independent?' . The answers were Democrat, Republican, Independent or Other. We coded responses as 'Democrat' = 1, 'Republican' = 0; we omitted all other responses from the analysis.

Dependent variables
• Evidence Assessment. To measure evidence assessment, we asked participants the following question after they read the policy-relevant information: 'How would you summarize the research described above?' . We gave them five potential options to choose from. These included the accurate summary, 'The economic development incentives did appear to increase investment in targeted areas but appear not to have increased employment or otherwise improved the well-being of residents' , the unclear summary, 'The economic development incentives did appear to increase investment in targeted areas but it is not clear whether they increased employment or otherwise improved the well-being of residents' , and three other clearly inaccurate options given the information provided. We code three categories of response as 'Accurate' , 'Unclear' , and 'Inaccurate' . In the analysis, 'Accurate' = 1 and 'Unclear' = 0. 'Inaccurate' was excluded from the analysis since it was rarely chosen. • Belief updating. To assess belief updating, we asked participants for their beliefs about the policy's impact before and after reading the information about the policy (1 = A large negative impact, 5 = A large positive impact). To create the updating measure, we took the absolute value of the difference between preinformation belief and post-information belief. We used this measure because, depending upon the individual's prior beliefs, the information could reasonably lead them to update in either direction.

Covariates:
• Policy priority. To assess whether economic development was a policy priority, we asked participants to rank order the following policy issues based on their own policy priorities: economic development, infrastructure, education, criminal justice, and health. The ranking was operationalized as a continuous numeric variable. • Opportunity Zone policy. We asked participants if they had an Opportunity Zone in their district where 0 = they did not, 1 = they were unsure if they did, and 2 = they did have one. • Job type. We asked participants to indicate their status as an elected official (=0) or legislative staff (=1).

Data analytic plan
To test H1 that the treatment generally increases belief updating, we ran an OLS (ordinary least squares) regression with treatment predicting belief updating. To test H2 that updating will be affected by (a) political party or (b) information compatibility, we conducted an OLS regression with treatment interacted with political party predicting belief updating. To assess whether directional motivating reasoning is operating (in the absence of treatment), we conducted an exploratory analysis that estimates the correlation between evidence assessment and political party. A multinomial logistic regression was conducted with treatment interacted with political party predicting evidence assessment, with evidence assessment as the reference case. All analyses controlled for policy priority, Opportunity Zone policy, and job type. We pre-registered analyses at Open Science Forum (https://osf.io/cqnyd/).

Descriptive statistics
In the sample, 104 participants identified as Democrats and 48 identified as Republicans. The remaining 42 participants did not identify with either party and we excluded them from the analysis. Those who did not belong to either party were predominately legislative staff, some of whom work in non-partisan roles. For reference, in 2019, 3465 U.S. state legislators were Democrats while 3844 were Republicans 16 . We observed no evidence of political polarization of prior beliefs about the impact of Opportunity Zones (p = 0.40).

Testing H1 and H2
Counter to H1, eliciting an evidence-based policymaker identity did not seem to increase belief updating on average in this scenario, see S.I. table 1, model 1. However, our findings were consistent with H2. H2 predicts that we might observe larger treatment effects among groups that (at baseline) experience higher levels of directional motivated reasoning. When the interaction between treatment and political party was included, the estimated treatment effect was positive (B = 0.39; SE = 0.15; p < 0.01; See S.I. table 1 model 2). We observed a significant interaction between treatment and political party on belief updating (B = − 0.41; SE = 0.18; p = 0.025; See S.I. table 1 model 2) suggesting that the identity treatment increased belief updating among Republicans. These effects are visualized in figure 1.

Exploratory analysis
This analysis explored whether motivated reasoning was active in the absence of the treatment. We found differences in evidence assessment across political parties, consistent with directional motivated reasoning. Political party was correlated with evidence assessment (B = 0.20; SE 0.091; p = 0.027; See S.I. table 2 model 1). Democrats were more likely to select the accurate summary than Republicans. This suggests that even though Democrats and Republicans did not differ in their initial beliefs, Republican participants may have been more resistant to changing those beliefs compared to Democratic participants.

Discussion
In the pilot, we found that activating an evidence-based policymaker identity increased updating among Republicans but did not detect an effect on Democrats. Our exploratory analysis found party differences in evidence evaluation that suggest that (in the absence of the treatment) Republicans were more likely to engage in directional motivated reasoning to maintain positive views of the policy. Taken together, these finding suggest that the treatment may be effective in increasing updating when directional motivations are present. In the absence of directional motivations, accuracy motivations might already be high, creating a ceiling effect. While the findings were consistent with H2, some other factor correlated with political party could interact with the treatment and have produced this pattern of results.
This study included a policy priority measure to control for differences in issue interest. Unfortunately, this measure could potentially be influenced by exposure to the Opportunity Zone information treatment. Participants may have been more likely to indicate that they prioritize economic development because they had been prompted to think about these issues or because they suspect that the researchers were interested in these issues. While we cannot rule out these effects, the main findings hold both when the measure was and wass not included in the regression.
The main study explored the interaction of information compatibility and the treatment to address that concern. That study largely replicated the design of the pilot but with energy policies subject to greater political polarization. Hence, we focused on the interaction of information compatibility and the treatment.

Main study-polarized energy policy
The main study was conducted with U.S. policymakers using the CivicPulse platform 17 and deliberately focused on two climate change-related energy policy issues that were expected to be highly polarized to assess the performance of the intervention in a polarized domain: energy efficiency rebate and renewable energy procurement policy. The study randomly assigned participants to one of two policy domains. We framed bboth policies as a response to climate change, a domain that is polarized by political party in the U.S. (Stokes andWarshaw 2017, Feldman andHart 2018). The energy efficiency scenario was intended to be challenging to Democrats while the renewable energy condition presented information was intended to be challenging to Republicans.

Participants and recruitment
The survey was fielded by CivicPulse, a non-profit research organization that regularly surveys public officials in the United States. CivicPulse maintains a dynamically updated contact list of elected executives (like Mayors) and elected legislators associated with all townships, municipalities, and counties in the United States with populations of 1000 or more. We surveyed a random sample of officials from this list for this study. CivicPulse was founded to help gather and share data among local governments. It provides its members with insights on the issues facing local jurisdictions as well as the approaches others are taking to tackle those issues. We anticipated that those policymakers that participated in our study would differ from non-participants, potentially in their interest in research and data, in their administrative capacity (since taking surveys takes time), or in other unobservable ways. A total of 561 local and policymakers in the U.S. using the CivicPulse Platform completed the survey out of the 770 that began the survey. We excluded incomplete surveys and surveys from those not identifying as Republican or Democrat were excluded, yielding a final sample size of 356.

Experimental design
The design for the main study followed that of the pilot. However, the evidence section that participants were asked to read was shortened in order to reduce the estimated time needed to complete the survey to comply with CivicPulse requirements. In this survey, evidence was summarized in one paragraph which included citations of peer reviewed research papers (see S.I. figures 4 and 5).

Independent variables
• Evidence-based policymaking treatment. The treatment was the same as in Study 1. We coded those randomly assigned the treatment as '1' , and those assigned to the the control as '0' . • Policy Condition. We randomly assigned participants to one of two climate change energy policy issue scenarios: energy efficiency rebate and renewable energy procurement. The energy efficiency rebate policy scenariopresented participants with information suggesting that these programs may result in smaller than projected energy savings. The renewable energy procurement policy scenario provided information suggesting that the proportion of renewable energy could likely be increased without increasing rates. • Political party. Participants were asked 'Generally speaking, do you generally think of yourself as a…?) 'Democrat' , 'Republican' , 'Independent' , or 'Other' . For those participants who identified as 'Independent' , or 'Other' , they were then asked if they 'leaned toward either of the major parties and were offered the options of 'Democratic Party' , 'Republican Party' , and 'Neither' . We coded participants as 'Republican' = 1 or 'Democrat' = 0 if they identified with or leaned toward one of these two parties. We coded all others as 'Other' . This information on participants was collected at an earlier point by CivicPulse. • Party compatibility. For Republican participants randomly assigned to the renewable energy policy condition and for Democrat participants randomly assigned to the energy efficiency policy condition, we coded compatibility as '1' . For Democrat participants randomly assigned to the renewable energy policy condition and for Republican participants randomly assigned to the energy efficiency policy condition, we coded compatibility as '0' . • Prior compatibility. Information about residential energy efficiency rebates suggests that those programs are less cost effective than alternatives, thus this information was compatible with beliefs that the policy would have a neutral or negative impact on participants' community (pre-information beliefs). For those in the in the energy efficiency condition, we coded their prior compatibility as '1' if they initially rated the impact as neutral or negative and '0' if they anticipated positive effects. The renewable energy condition provides evidence that renewable energy generation can be increased without increasing costs. For those in the renewable condition, we coded their prior compatibility as '1' if they anticipated that the policy would have a positive impact on their community (pre-information beliefs) , and as '0' if they anticipated neutral or negative impacts.

Dependent variables
• Evidence assessment. We asked participants in the energy efficiency condition to choose between the following assessments after reading policy relevant evidence (See S.I. figure 4). We coded participants who chose the option 'Energy efficiency programs that have been studied appear not to be cost effective' as 'Accurate' . We coded participants who chose the option 'Energy efficiency programs that have been studied appear to be cost effective' as 'Inaccurate' . We coded participants who chose the option 'It is unclear whether energy efficiency programs that have been studied are cost effective' as 'Unclear' . We asked participants in the renewable energy condition to choose between similar summaries after reading the evidence (See S.I. figure 5). We coded participants who chose the option 'Projections suggest that increasing renewable energy in the future will not increase electricity rates on average' as 'Accurate' . We coded participants who chose the option 'Projections suggest that increasing renewable energy in the future will increase electricity rates on average' as 'Inaccurate' . We coded participants who chose the option 'Projections suggest that it is unclear what impact increasing renewable energy in the future would have on electricity rates on average' as 'Unclear' . In the analysis, 'Accurate' = 1, 'Unclear' = 0, and we excluded 'Inaccurate' observations from the analysis. • Belief updating. To assess belief updating, we asked participants for their beliefs about the policy's impact before and after reading the information about the policy: 'How positive or negative an impact do you think the policy will have?' measured on a 7-point Likert scale (1 = A large negative impact, 7 = A large positive impact). To create the updating measure, we took the absolute value of the difference between preinformation belief and post-information belief. We used this measure because, depending upon the individual's prior beliefs, the information could reasonably lead them to update in either direction.

Covariates
• Policy priority. To assess whether energy and environmental policy was a policy priority, we asked participants to rank order the following policy issues based on their own policy priorities: energy and environmental, transportation, business, social, and crime and safety, or other. We coded this as a continuous numerical variable. We collected this information after we administered the treatment and key outcome measures. We did not include this measure in the primary analysis but used it to assess covariate balance. • Competition. We asked participants 'When you last ran for office, did you face an opponent in the general election' . The 'Yes' response was coded as '1' , the 'No' response was coded as '0' , and the 'Other' response was coded as 'NA' . This information on participants was collected at an earlier point by CivicPulse.

Data analytic plan
To test H1 that the treatment increases belief updating, we ran an OLS regression with treatment predicting belief updating. To test H2 that the treatment has a greater effect on updating when information is incompatible, we conducted an OLS regression with treatment interacted with (a) prior compatibility and (b) party compatibility predicting belief updating. To explore whether directional motivated reasoning might occur in the absence of treatment, we ran multinomial logistic regressions with treatment interacted with (a) prior compatibility and (b) party compatibility predicting evidence assessment, with accurate assessment as the reference case. Analyses were pre-registered at Open Science Forum (https://osf.io/m974g/).

Descriptive statistics
In this sample, 161 participants identified as or leaned toward identifying as Democrats and 198 identified as or leaned toward identifying as Republicans. The average length of time participants had served in a government position was 12.99 years. With respect to ambition, 34 indicated no interest, 201 indicated an openness to the possibility, and 184 indicated that they were actively considering running for higher office. Most indicated (n = 256) that they faced competition in the last election, while 153 indicated that they did not.
We analyzed party differences in initial beliefs about the policy (measured before participants read the information treatment) to confirm that there was polarization at baseline. Among those assigned to the energy efficiency rebate policy scenario, as predicted, we observed partisan polarization in pre-information beliefs with Democrats believing, on average, that the policy will have a more positive impact than Republicans. This suggests that Democrats will find the information challenging since it suggests that these programs may not have a large positive impact. On a seven-point Likert scale from 'a large negative impact' (1) to 'a large positive impact' (7), Republicans on average provided an estimate that was 1.48 points less than Democrats and the p-value was less than 0.01.
Among those assigned to the renewable energy procurement policy scenario, we also observed partisan polarization in pre-information beliefs with Democrats believing, on average, that the policy will have a more positive impact than Republicans. Republican's estimate of policy impact was 2.58 points less than Democrats with a p-value less than 0.01 on a seven-point Likert scale from 'a large negative impact' to 'a large positive impact' .

Testing H1 and H2
As predicted in H1, eliciting an evidence-based policymaker identity significantly increased belief updating for polarizing energy policies when presented with new scientific evidence (B = 0.27; SE = 0.11 p = 0.015; See S.I. table 3, model 2). We also found support for H2 that the treatment will have greater impact on updating when the evidence is challenging. We did not observe a significant interaction between treatment and party compatibility on belief updating (B = − 0.25; SE = 0.22; p = 0.25; See S.I. table 3, model 2). However, we did observe a significant interaction between treatment and prior compatibility (B = − 0.47; SE = 0.22; p = 0.034; See S.I. table 3, model 3). This indicates that the treatment was most effective at increasing updating when information challenges the policymakers baseline beliefs about the policy. This relationship is illustrated in figure 2.

Exploratory analysis
We tested whether compatibility predicted evidence assessment to explore whether directional motivated reasoning was at work. We found that prior compatibility predicted evidence assessment. Individuals were more likely to choose the accurate rather than the unclear assessment when information was compatible with their prior beliefs (B = 0.22; SE = 0.069 p < 0.01; See S.I. table 4, Model 1). This suggests that individuals adopted higher evidentiary standards upon encountering prior challenging information. Party compatibility was also found to be correlated with assessment.

Discussion
This work suggests that prompting policymakers to identify as 'evidence-based' has the potential to increase belief updating when they receive evidence on climate policies like local clean energy programs. Encouragingly, this intervention appears to be effective in the polarized domain of climate change policy and works for both Republican and Democratic policymakers. In fact, the treatment effect appears to be largest among those for whom the evidence was incompatible with their prior beliefs.
This pattern of effects along with the exploratory analysis of evidence assessment suggests that the intervention may work by reducing directional motivated reasoning. We found that participants with priors that were incompatible with the evidence exhibited motivated evidentiary standards when assessing the evidence. Those individuals were more likely to describe the implications of the evidence as 'unclear' , likely because they were motivated to dismiss evidence that challenged their initial view of the policy. We suspect that the intervention is most effective within that population because it introduces a countervailing motivation. At baseline, we expected that individuals would be motivated to defend their prior beliefs but that the intervention would reduce that defensive motivation because changing their mind offers an opportunity to see oneself as an evidence-based policymaker who is willing to 'follow the facts' . This pattern is found in both the pilot and the study, increasing our confidence in the result.
One difference between the pilot and the main study is that the pilot found that political party was the primary predictor of both summary (which we use as a measure of motivated reasoning) and the efficacy of the treatment in increasing updating, whereas we found that prior compatibility played an analogous role in the main study. Both the desire for belief consistency and a desire to hold party consistent views are plausible drivers of directional motivated reasoning. As a result, this work is not well positioned to identify the 'true' source of motivated reasoning which likely varies across populations and policy contexts. Our findings do however provide evidence that directional motivated reasoning (that largely corresponds with party lines) may suppress the impact of evidence among U.S. policymakers. Importantly, that directional motivated reasoning does not appear to be limited to one political party, members of both parties appear to be resistant to evidence that challenges their policy views.
These findings suggest that an intervention that prompts policymakers to identity as evidence-based could increase the impact of policy impact evaluations, academic studies, and other evidence. Ultimately, that could lead to the adoption of more effective energy and climate policies. However, this work represents only an initial step toward testing the potential effects of this type of intervention. There are several limitations of the survey experiment methodology used in this study.
While the internal validity of the experimental design is generally high, one potential concern is experimenter demand or desirability effects. Demand effects occur when participants guess the hypotheses of the experimenter and adjust their responses in order to confirm those hypotheses. While demand effects are a much-discussed concern in experimental research, online survey experiments have been shown to be generally robust to demand effects (Mummolo and Peterson 2019). We believe that policymakers are no more likely than general populations to be subject to demand effects and likely would not wish to appear to be swayed by the intervention.
Another concern is that participants may differ from policymakers at large. The CivicPulse sample used in the main survey is fairly representative of the demographics of local policymakers in the U.S. , however participants in the survey may differ in unobservable ways from the general population of policymakers. For example, policymakers who are willing to take a survey may be more likely to identify as 'evidence-based policymakers.' If that is the case, our results could overstate the effect of the intervention in the general population if those individuals are more responsive to the intervention. However, the participant population could potentially be less responsive to the intervention if the evidence-based identity is already salient for those participants.
Our reliance on a survey-based design also limits our ability to predict how an evidence-based identity intervention would affect actual policymaking behavior. For example, we cannot speak to the durability of the intervention or the relative role of beliefs about policy efficacy in policymaking behavior relative to strategic motivations and other factors. We also cannot speak to how other factors like underlying trust in science interact with the treatment. More work is needed to understand how the treatment interacts with other characteristics and incentives and how treatment affects behavior over time.
We do believe that the simplicity of the treatment makes it well suited to science communication and or evidence-based policymaking interventions in the field. The treatment in these studies involves asking participants if they identify as an evidence-based policymaker and it would be difficult to systematically build that prompt into climate science communications. However, rhetorical appeals to an evidence-based policymaker identity could be strategically embedded in webinars, reports, and other modes of research communication. Building the general strength and salience of the identity, rather than simply activating it prior to research communication might also be effective. That goal might be accomplished through efforts like evidence-based policymaker pledges or the formation of affiliation groups. Field experimentation will be a critical next step in both understanding the psychological dynamics of the treatment and in designing tools that can increase the impact of evidence in climate policymaking.

Data availability statement
The data cannot be made publicly available upon publication because they contain sensitive personal information. The data that support the findings of this study are available upon reasonable request from the authors.