Balance as bias, resolute on the retreat? Updates & analyses of newspaper coverage in the United States, United Kingdom, New Zealand, Australia and Canada over the past 15 years

We live in embattled times regarding media reporting on climate change. In terms of frequency of reporting, media accounting of the changing climate has fluctuated in coverage and fought for attention amid other issues (figure 1). In terms of quality and content of coverage, many challenges to provide accurate and contextualized coverage have persisted.

Zoom In Zoom Out Reset image size

In this study, we systematically assessed the accuracy of media coverage of human contributions to climate change. We conducted a content analysis of coverage across 4856 articles over 15 years (January 2005 to December 2019) by analyzing 17 high-circulation national print media sources in five countries: the United Kingdom (U.K.), Australia, New Zealand, Canada, and the United States (U.S.). We selected these five countries because the phenomenon of climate contrarianism is primarily an Anglo-Saxon phenomenon (Painter 2011). Therefore, focusing specifically on newsprint sources in Anglophone countries enables a meaningful exploration of whether the previously observed norm of 'balanced' reporting has continued to amplify outlier perspectives that bely the scientific consensus.

This research extends previous analyses conducted by Boykoff and Boykoff (2004) that examined media coverage of climate change and global warming in leading U.S. newspapers from 1990 to 2002. The study found evidence of reporting that was out of step with the scientific consensus that humans contribute to climate change ⁸ . The informational bias was attributed to over-reliance on the journalistic norm of 'balanced reporting'. Subsequent research examined U.S. and U.K. print media from 2003 to 2006 (Boykoff 2007b). This study found an ongoing and significant divergence from scientific consensus in the U.S. in 2003–2004, followed by a decline in 2005–6, along with no significant divergence in U.K. reporting over this same four-year time period ⁹ .

There have been several studies that have replicated parts of this previous research, often with a single western country-focus (i.e. Nisbet 2011, Bacon and Nash 2012, McGaurr et al 2013, Painter and Gavin 2016, Schmid-Petri et al 2015, Boykoff and Farrell 2019, Merkley 2020, Ruiu 2021). However, there has not been systematic follow up research in the U.S. that has also expanded longitudinally to multiple country contexts. In these interim years, two critical issues have emerged:

• (a)  
  there has been continued extrapolation based on these earlier studies to evaluate current conditions. The Boykoff and Boykoff (2004) paper continues to be cited frequently—808 times since 2016 and 160 times since 2020 alone ¹⁰ . In these citations, the 2004 article is often used as evidence of ongoing problems with climate coverage due to the journalistic norm of balanced reporting.
• (b)  
  while some follow up studies have indicated a decline in the prevalence of balanced reporting in climate coverage (Boykoff (2007b), Brüggemann and Engesser 2017) others have found a continued prevalence of this phenomenon in some contexts (Gurwitt et al 2017, Bolsen and Shapiro 2018, Ruiu 2021), resulting in continued uncertainty about the magnitude and scope of the problem.

To address these issues, this study has methodically updated the past 15 years of coverage (2005–2009) while expanding analyses to influential national-level newspaper outlets across five countries.

We focus on anthropogenic climate change as an issue where there is clear convergent agreement among relevant expert researchers. This provides testable hypotheses regarding how this consensus is subsequently represented in prominent news media outlets (see section 1).

Expert consensus on anthropogenic climate change has emerged and strengthened over the past three decades. Relevant expert communities and institutions, peer-reviewed scientific research, and government reports have consistently found that human activity has driven the observed increase in globally averaged temperatures since the Industrial Revolution (Allan et al 2000, Houghton et al 2001, Solomon et al 2007, Pachauri et al 2014, Gaffney and Steffen 2017, Commonwealth Academies of Science 2018). Furthermore, social science research that empirically measures the strength of this consensus has found that 97% of relevant expert scientists agree that humans contribute to climate change (Oreskes 2004, Anderegg et al 2010, Cook et al 2013, Verheggen et al 2014).

Moving from this consensus to media portrayals of anthropogenic climate change, the journalistic values and norms of balanced reporting and objectivity shape representational practices and habits. Balanced reporting is a value that often appears to fulfill pursuits of objectivity (Cunningham 2003). Objectivity and balance developed through the professionalization of journalism amid a rapid expansion of modern media communications (Starr 2004). Professional media institutions—punctuated by standards of accuracy, accountability, and objectivity through social and ethical responsibility—were seen to contrast with partisan press (McQuail 2005). Balance also provided a 'validity check' for reporters on deadline or lacking the capacity to assess the veracity of nuanced or complex claims themselves (Dunwoody and Peters 1992). Over the past few decades, many scholars and practitioners have called for reassessments of objectivity and balance in the context of accurate and reliable reporting (i.e. Gerken 2020).

Media deployment of objectivity and balance when covering various dimensions of climate change have been widely acknowledged as problematic (Boykoff and Boykoff 2004, Boykoff 2007a). Research examining objectivity and balance in the context of climate change reporting has, in part, motivated related studies.

Analyzing coverage of press releases in the New York Times, USA Today, and Wall Street Journal, Wetts (2020) found that press releases that opposed climate action earned about twice the amount of coverage as those that called for action on climate change. These findings indicate the propensity for a continued 'false balance' in the U.S. print media, thereby raising questions about whether the norms of balance and objectivity are truly a 'thing of the past' when it comes to climate change coverage (Wetts 2020, 19 059). In another study, Bohr (2020) ran a computational analysis of sources of bias in coverage of climate change across 52 newsprint sources in the U.S. The study found that the ideological orientation and circulation size of the newspaper outlet significantly influenced the attention given to topics and framings. In a similar vein, Brüggemann and Engesser (2017) found that right-leaning media outlets, in addition to the national-political context, and contrarian-leaning journalists contribute to niches of climate denialism (66). Brüggemann and Engesser (2017) also found a shift from 'objective/balanced' journalism in climate change reporting, toward interpretative journalism, where contrarian voices are quoted and placed in the context of a dismissal of their stance (65).

These recent studies illustrate the persistence of inaccurate, contested or biased coverage of climate change. However, there remains a pressing need to re-evaluate the occurrence of this problem more recently and in a broader context. Additionally, cross-national comparisons often rely on purely quantitative or automated content analysis (i.e. Engesser and Brüggemann 2016). Notable exceptions include Painter and Ashe (2012) and Brüggemann and Engesser (2017). However, both studies covered a shorter time frame, where the former study focused on several months' worth of coverage in 2007 and between 2009 and 2010, and the latter study covered the years 2011 and 2012. In this study, we provide an updated and expanded assessment of the prevalence of overly 'balanced' coverage of climate change across major national print media sources in Australia, New Zealand, Canada, the U.K., and the U.S. from 2005 to 2019. Further, the study combines a relatively large sample (N = 4856) with a qualitative discourse analytic approach, thereby balancing both scope and sensitivity of analysis, to make significant contributions to the broader field.

We analyzed coverage across 17 sources in newspapers from five countries, January 2005—December 2019, selecting for sources that met the criteria of: (a) high circulation and (b) accessibility of the entire article text through the Nexis Uni database in 2019. Based on these criteria, we included the following sources:

• U.K.: Guardian and Observer, Daily Mail and Mail on Sunday, Telegraph and Sunday Telegraph, Times and Sunday Times
• Australia: Sydney Morning Herald, Courier Mail and Sunday Mail, Daily Telegraph and Sunday Telegraph, The Age
• New Zealand: New Zealand Herald, Dominion Post, The Press
• Canada: Globe and Mail, Toronto Star, National Post
• U.S.: New York Times, USA Today, Washington Post

The unit of analysis was the individual article. We searched articles with keywords 'global warming' or 'climate change' and deployed content analysis to code the full sample set. Searches were conducted in Nexis Uni and duplicates were not grouped to ensure replicability ¹¹ . The population was N = 246 320 articles. A random number generator was used on each month of data to take an initial sample set of 20%, which yielded N = 48 603 articles. To further randomize article selection, we chronologically analyzed 10% of the sample set using a random number generator again on each month of data. Our sample set was 4856 articles (i.e. 1.97% of the total sample; see tables 1 and 2).

We manually coded a representative sample of the population of articles (N = 4856). This is an abbreviated version of that codebook (see supplemental materials (available online at stacks.iop.org/ERL/16/094008/mmedia)):

• Category 1: anthropogenic global warming only contributes to climate change (distinct from natural variations)
• Category 2: anthropogenic global warming significantly contributes to climate change (in combination with natural variation)
• Category 3: anthropogenic global warming and natural variability equally contribute to climate change
• Category 4: anthropogenic global warming negligently contributes to climate change
• Category 5: not applicable: includes articles that are not about anthropogenic climate change ¹² . Category 5 articles were removed from the final sample.

Category 2 refers to articles that portrayed the scientific consensus view that humans are the main driver of currently observed climate change, while Category 3 refers to articles that provide false balance between scientific consensus and contrarian views regarding the existence or causes of climate change. Category 4 refers to articles that either refute the existence of climate change or else make claims that observed climate change is not driven primarily by humans.

This coding scheme was consistent with previous investigations of media coverage of anthropogenic climate change (Boykoff and Boykoff 2004, Boykoff 2007b, Boykoff and Mansfield 2008, Boykoff 2008).

Coding was conducted using a critical discourse analytic approach, rather than tallying frequencies of words or phrases. Critical discourse analysis has been widely used in the field of media studies for decades to analyze discrete media texts to examine which voices and discourses are privileged over others (Phelan 2017). Following Boykoff and Boykoff (2004, p 126), we define discourse as a 'recognizable network of questions, assumptions, reference points, and language games employed by a given group of people that enables them to organize and understand the world.' Coding therefore focused on applying topics (see codebook) to textual units by identifying combinations of terminology, framing techniques, salience of elements in the text, tone, and relationships between clusters of messages to discern larger intertextual patterns. The validity of discourse analysis is established when linguistic details converge to support agreement (Gee 2014).

To ensure validity, each article was independently coded by two coders in the larger group of six coders on the team. Pairing of coders was rotated once during analysis to ensure consistency across the entire coding team. Coding within a joint research project requires iteratively refining the codebook to ensure shared interpretation and application within individual coding efforts (Saldaña 2016). Following this period of independent coding, the coding team convened at regular intervals to assess interpretive convergence and coding agreement and to update the codebook.

Following the coding process, intercoder agreement was calculated using Krippendorff alpha and Scott's Pi tests of intercoder reliability. The intercoder reliability correspondence was 89.6% and Krippendorff alpha rate was 0.81, while Scott's Pi test was 0.809, meeting acceptability standards (Lombard et al 2002). This reliability and validity testing accounted for spuriousness while also integrating past knowledge and familiarity with various facets of climate change science, policy, and news coverage (Metag 2016).

Coders also assembled notes as they proceeded through the coding process, capturing illustrative passages in articles, and drawing representative examples from each of the sources. Consistent with previous research, 'balance' was determined to be coverage that provided roughly equal attention and emphasis to competing viewpoints on anthropogenic climate change, however not necessarily equal time and space (Entman 1989, Dunwoody and Peters 1992, Boykoff 2011). Table 3 includes article quotes illustrative of Categories 1 through 4.

In addition to news articles, the analysis included letters, editorials, and other publications that contained the keywords 'global warming' or 'climate change.' These latter units of analysis may be outside the bounds of journalistic norms—for example, the author of a letter or editorial may not follow guidelines on balance or 'truth' in reporting—but these still reflect the overall content of the sources in which they are published and, thereby, impact readers.

Throughout the coding process, coders discussed and addressed coding differences, revising the codebook and coding process as necessary. How to code for tone proved a common point for discussion. For example, an article may not directly deny anthropogenic climate change, but may read in a mocking or sarcastic tone with respect to human-caused climate change. With some exceptions, coders generally coded based on the content of the article itself, as opposed to tone, as tone can be difficult to interpret. On the other hand, if tone came through strongly in an article, then this was reflected in the final category assignment. Given the large sample size for this project, the researchers determined that it is not necessary to achieve a 'perfect formula' for article category assignments with respect to tone; among the general pool of readers of a published piece, one reader may pick up on sarcasm in an article, whereas another reader may not.

Following coding, we first analyzed the percentage of articles in each category and the percent difference between Category 2 articles (i.e. accurate reporting) and Category 1, 3, and 4 articles combined (i.e. biased or otherwise inaccurate reporting). Per our codebook, 2194 articles were coded as Category 5 (not applicable) and were removed from the sample, yielding a total of N = 2662 articles (i.e. 1.08% of the total sample) that were directly relevant to the causes of climate change (i.e. Categories 1, 2, 3 and 4 combined). Across the 15 years, there were two articles in Category 1, 2397 articles in Category 2, 150 articles in Category 3, and 113 articles in Category 4. Only those articles directly relevant to climate change (i.e. N = 2662) were used in the calculation of percentages per category and percent difference scores in tables 4 and 5.

The null hypothesis was that newspaper coverage reflected the convergent agreement in the scientific community regarding anthropogenic climate change in each year (H₀: π₁ =π₂). The alternative hypothesis was that media portrayals significantly diverged from this scientific consensus (H_a: π₁ ≠ π₂).

The study utilized z-score significance testing (comparing ratios) similar to a t-test (z ⩾ 1.96, p = 0.05; z ⩾ 2.575, p = 0.01; z ⩾ 3.29, p = 0.001):

$$\begin{align*}z & = \frac{{\left( {{\Pi _1} - {\Pi _2}} \right)}}{{{{{{\tilde {O^{^{\prime}}}}}}_{{\Pi _1} - {\Pi _2}}}}}\,{\text{and}}\,{{{\tilde {O^{^{\prime}}}}}_{{\Pi _1} - {\Pi _2}}}\\ & = \sqrt {{\Pi ^{\wedge} }\left( {1 - \Pi ^{\wedge} } \right)\left( {\frac{1}{{{n_1}}} + \frac{1}{{{n_2}}}} \right)} \end{align*}$$

where π^{^} = pooled estimate based on the whole sample.

We asked the essential question across 17 national print media sources in five countries over 15 years (2005–2019): has coverage of climate change or global warming been 'balanced,' and therefore actually perpetrating an informational bias?

We analyzed the significance of differences in discourse between the consensus in the relevant expert scientific community and media representations of anthropogenic climate change by coding for scientifically accurate reporting (Category 2) and biased or otherwise inaccurate reporting (Categories 1, 3, and 4). We examined these categories across (a) time and (b) source by country.

To start, we analyzed differences in newspaper coverage of climate change or global warming, as related to the scientific consensus, by year across all sources (see table 4). As a percentage of total coverage, the average difference between accurate representations of the scientific consensus (Category 2) and less accurate or biased coverage (Categories 1, 3, and 4 combined) was 79.61% ± 7.81. The lowest difference between accurate and less-accurate reporting was 63.64% in 2010. This was statistically significant at the 95% confidence level (z-score = −2.05, p < 0.05) ¹³ . We found that the highest difference between accurate and less accurate reporting, 92.55%, occurred in 2015. This was statistically significant at the 90% confidence level (z-score = 1.66, p  <  0.1). Additionally, the upward trend of the fit line in figure 2 shows an approximately five percentage point increase (from 87% to 92%) in Category 2 articles (anthropogenic global warming significantly contributes to climate change in combination with natural variation) over the 15 year period, suggesting that scientifically accurate climate coverage is improving over time.

Zoom In Zoom Out Reset image size

Next, we analyzed differences in the accuracy of reporting across the 17 sources by country of publication (see table 5). As a percentage of the total coverage, the average difference between accurate coverage and less accurate coverage was 75.53% ± 18.13. We also found that the Daily Mail and Mail on Sunday, Telegraph and Sunday Telegraph, and the National Post all had significantly less accurate coverage of climate change than the other sources. The National Post of Canada had the least accurate coverage with 70.83% of articles representing scientific consensus on climate change (Category 2), compared to 9.17% in Category 3 (i.e. anthropogenic global warming and natural variability equally contribute to climate change) and 20.00% in Category 4 (i.e. anthropogenic global warming negligently contributes to climate change). This was statistically significant at the 90% confidence level (z-score = −1.87, p < 0.1). In Australia's Telegraph and Sunday Telegraph, 71.15% of articles were in Category 2 (i.e. accurate reporting), 15.38% in Category 3, and 13.46% in Category 4 (i.e. less accurate or biased reporting). This was statistically significant at the 90% confidence level (z-score = −1.83, p < 0.1). Finally, for the U.K.'s Daily Mail and Mail on Sunday 72.50% of coverage was Category 2 (i.e. accurate reporting), 8.75% was Category 3, and 18.75% was Category 4 (i.e. less accurate or biased reporting). This was statistically significant at the 90% confidence level (z-score = −1.68, p < 0.1). It is also interesting to note that across all sources, Canada's Toronto Star and the U.K.'s Guardian and Observer, which had the greatest number of articles in the sample (N = 520), had the best or most scientifically accurate coverage of climate change, with 97.37% of the Toronto Star's coverage in Category 2 and 95.00% of the Guardian and Observer's coverage in Category 2. Thus, both Canada and the U.K. had sources with the best and worst coverage of climate change within the sample.

Figure 3 shows the percentage difference between Category 2 (i.e. accurate reporting) and Categories 1, 3, and 4 (i.e. less accurate or biased reporting) for all the sources, from most accurate coverage of climate change to least accurate. The middle bar represents the average difference score (75.53%), while the top bar is the average difference score plus one standard deviation (93.67%) and the bottom bar is the average difference score minus one standard deviation (57.40%).

Zoom In Zoom Out Reset image size

Nonetheless, there remain fluctuations in how accurately the scientific consensus is represented from year to year. For example, 2010 had the lowest percentage of scientifically accurate articles (Category 2), at 81.82%, while 2015 had the highest, at 96.28% (figure 4). Figure 5 shows the % difference between accurate (Category 2) and less accurate reporting (Categories 1, 3, and 4). We observe signification variation across the five countries. The accuracy of coverage has increased steadily in the U.S. and Canada since 2012, and to a lesser extent in Australia. We also observe significant interannual variation in the U.K. and New Zealand with no clear growth trend ¹⁴ . Moreover, while U.S. and New Zealand newspapers reveal similar trends across all sources, it is notable that the outlets in Australia (Telegraph and Sunday Telegraph), Canada (National Post) and the U.K. (Daily Mail and Mail on Sunday) were found to publish significantly less accurate coverage of climate change over time (table 5). These outlets also ran fewer articles about human contributions to climate change over this time period in comparison with their counterparts within these respective countries (table 1).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

Overall, there are significant differences in the percentage of articles that accurately represented the scientific consensus about anthropogenic climate change (Category 2) over time and sources across our entire sample. However, this is underscored by the fact that the vast majority of reporting—an average of 90% across all years and sources—accurately reflected the consensus view (figure 4).

This research has sought to move from limited, dated, or speculative characterizations of media coverage of anthropogenic climate change over the past 15 years to a methodical and empirically driven analysis of newspaper media portrayals in Australia, New Zealand, Canada, the U.K., and the U.S.

We conclude that in the 17 major print news outlets analyzed here, the journalistic norm of balanced reporting has not informationally biased coverage of anthropogenic climate change from 2005 to 2019. In fact, we found that on average scientifically accurate climate coverage improved (see figure 2). Thus, we argue that it is time to retire blanket statements regarding the continued dominance of 'balanced' coverage of the scientific consensus on climate change that gives equal credence to 'both sides.' However, our correlational data show that it is still important to proceed with caution, as there have been significant fluctuations in the accuracy of reporting across time, within countries, and by newspaper outlets.

For example, after the University of East Anglia email hacking scandal (known also as 'climategate') and the Copenhagen Accord in November/December 2009, among other climate-related events receiving substantial press coverage, 2010 was the least accurate year of climate coverage in our sample, 81.82% of coverage represented the scientific consensus view, 9.63% of coverage represented a false balance view, and 8.56% portrayed denialist or contrarian views. The most accurate coverage of climate change in our sample was in 2015 with 96.28% of coverage portraying the scientific consensus view, 1.06% of coverage representing a false balance view, and 2.66% representing denialist or contrarian views. Interestingly, this period preceded the successful negotiation of the Paris Agreement in December 2015. Furthermore, 2019 saw a significant rise in coverage across both the world and the U.S. due to a confluence of high profile climate-related disasters and political events. Sub-Saharan African drought, Central American migration pressures, South American deforestation, Asian public health concerns, European decarbonization, UN climate talks, Australian bushfires, Canadian federal elections, U.S. withdrawal from the Paris Climate Agreement, and global youth-led climate social movements punctuated the 2019 media and climate change landscape and lofted coverage into unseen territories, especially as the year came to an end (see also Boykoff et al 2020). 2019 was also the third most accurate year of reporting (after 2015 and 2018) in terms of coverage of climate change in our sample: 93.47% of coverage portrayed the scientific consensus view while 3.26% of coverage represented a false balance view and 3.26% represented denialist or contrarian views.

As a second example—and similar to the findings of other research (i.e. Boykoff and Mansfield 2008, Painter and Gavin 2016, Stoddart et al 2016, Ruiu 2021)—the sources with significantly less accurate climate change coverage, Canada's National Post, Australia's Telegraph and Sunday Telegraph (owned by Rupert Murdoch) and the U.K.'s Daily Mail and Mail on Sunday, all have a right leaning political orientation, while the left-leaning sources within the U.K., the Guardian and Observer, and Canada, the Toronto Star, had the most accurate climate coverage. As studied elsewhere (i.e. Carvalho 2007, Dunlap 2008, Boykoff 2011), ideological cultures and stances are among several factors that shape media coverage of climate change. Combined with our finding that these outlets have run fewer stories on anthropogenic climate change than their in-country counterparts over this time-period, the coverage in these conservative outlets has comparatively remained more 'silent' on these critical issues while these sources have continued to contribute to informational biases in the public arena.

These findings also raise additional questions for future studies. This research did not examine other aspects of journalistic coverage, such as placement and prominence of articles or differences between article types. Additionally, this study only included newspaper reporting on climate change, though it is well recognized that the print media industry has undergone significant decline and transformation in the last several decades ¹⁵ . While outside the scope of this study, it will be important to expand longitudinal and cross-national studies to include other types of media (i.e. cable news, digital native media, radio, etc) and to further interrogate the underlying features that shape uneven accuracy of reporting we observed over time and across sources in subsequent research.

Furthermore, while our data show that accurate reporting vastly outweighed less accurate reporting by a factor of 9–1 across all years included in our analysis, this may not be a cause for complacency. As Lamb et al (2020) have pointed out, the terrain of climate debates has shifted in recent years away from strict denial of the scientific consensus on human causes of climate change toward 'discourses of delay' that focus on undermining support for specific policies meant to address climate change (see also Schmid-Petri et al 2015). Additionally, other studies have shown that there is not always a direct relationship between the accuracy of individuals' climate knowledge and their beliefs about climate change (Fischer and Said 2021). This is compounded by the fact that individuals receive and take up knowledge about climate change via multiple information sources, with complex and varying effects on behavioral change (Arlt et al 2011). Thus, accurate reporting of the scientific consensus in print media alone is not sufficient to ensure that the public is both well-informed and willing to take individual or collective action to address climate change.

This research provides a valuable and timely update to previous studies examining the issue of balanced climate change reporting, which can help focus research efforts on pressing, current issues—a few of which we have outlined above. This research also contributes to comparative investigations of media coverage of climate change across national, political, and cultural contexts (i.e. Petersen et al 2019, Vu et al 2019) and consideration of its effects on perceived consensus that humans contribute to climate change (i.e. Koehler 2016, Bertoldo et al 2019). More widely, this research adds to ongoing research regarding issue salience and agenda-setting, as well as prioritization at the science-policy interface (i.e. Bromley-Trujillo and Karch 2019).

Going forward, this type of systematic research can inform ongoing discussions and debates about media coverage of climate change, helping those discussions focus elsewhere to other ongoing challenges inside as well as outside the newsroom. There is no doubt that there will remain many challenges associated with news reporting about climate change in the 21st century.

Olivia Pearman, Midori Aoyagi, Anne Gammelgaard Ballantyne, Kaori Doi, Rogelio Fernández-Reyes, Lauren Gifford, Erin Hawley, Isidro Jiménez Gómez, Jennifer Katzung, Gabi Mocatta, Ami Nacu-Schmidt, David Oonk, Jeremiah Osborne-Gowey, Lars Kjerulf Petersen, Anne Hege Simonsen, and Andreas Ytterstad. Publication of this article was funded by the University of Colorado Boulder Libraries Open Access Fund and the University of Colorado Boulder Environmental Studies Program.

The data that support the findings of this study are available upon reasonable request from the authors.

The authors declare no competing interests.