Comment on 'Assessing ExxonMobil's climate change communications (1977–2014)' Supran and Oreskes (2017 Environ. Res. Lett. 12 084019)

In 'Assessing ExxonMobil's climate change communications (1977–2014)' [1], Supran and Oreskes claim that ExxonMobil¹ purposely deceived the public by communicating a position on climate change in a series of published advertorials that is inconsistent with other statements contained in internal documents and published in technical journals. To reach that conclusion, the authors conducted textual content analysis and compared 187 documents generated between 1977 and 2004.

The publication is premised on at least two methodological flaws. First, the publication largely compared data from two different companies to determine whether there was a discrepancy between them. This comparison ignores that before 1999, Exxon Corporation and Mobil Oil Corporation were two separate companies; they were incorporated and headquartered in different states and did not share management or employees.

It would be illogical therefore to compare the internal documents of one company during the pre-merger period (Exxon) with the contemporaneous public statements of the other company (Mobil). Yet the authors have done just that, obscuring the separateness of the two corporations by 'refer[ring] to ExxonMobil Corporation, Exxon Corporation, and Mobil Oil Corporation as "ExxonMobil"' throughout the publication and regardless of whether the companies were independent at the relevant point in time.

Tables 1–4 show the data set that was used for the publication. The vast majority of the public statements evaluated by the authors were issued solely by Mobil before the merger. Of the public statements, 25 of 36 (approximately 70%) pre-date the merger and were generated entirely by Mobil. Conversely, the overwhelming majority of internal documents from the pre-merger period were generated by Exxon. In fact, only two out of 78 documents (approximately 3%) can be attributed to Mobil. All (29 out of 29) peer-reviewed articles, virtually all (17 out of 18) non-peer-reviewed documents, and virtually all (30 out of 31) internal documents considered by the publication in the pre-merger time period are attributable to Exxon. Accordingly, the bulk of the publication is devoted to comparing Mobil's public statements to Exxon's publications and internal documents during a period of time when Exxon bore no responsibility for Mobil's public statements and Mobil bore no responsibility for Exxon's publications and internal documents. Having compared fundamentally dissimilar data sets in search of a discrepancy, the authors committed a fundamental methodological error that renders their conclusions invalid.

Second, the publication assessed only a small subset of available advertorials. The authors report that the 'the company [Mobil] took out an advertorial every Thursday between 1972 and 2001' or approximately 1560 times. Yet they chose to review only 36 advertorials (or less than 3%) that were cherry-picked by another entity, Greenpeace, an activist group engaged in a long running anti-ExxonMobil campaign.² This reliance on limited and non-representative data sets (generated with undisclosed selection criteria) further calls into question the validity of the data used to support the publication's conclusions.

Based on these two methodological flaws, it is clear that valid conclusions cannot be drawn from the data sets analyzed.

As further proof that the article is fundamentally flawed, at ExxonMobil's request, Dr Kimberly Neuendorf, a professor at Cleveland State University who developed the content analysis method the authors relied on and cited in their research, conducted a review of the publication [2], and found the content analysis contained 'numerous fundamental and fatal flaws.'

Dr Neuendorf concluded the content analysis used in the publication 'is unreliable, invalid, biased, not generalizable, and not replicable.' Dr Neuendorf said the publication did not provide scientific support for either a discrepancy among ExxonMobil's climate change communications, or a claim that ExxonMobil misled the public.

In addition to data selection deficiencies, Dr Neuendorf identified defects in the coding of documents. According to Dr. Neuendorf, 'To maintain objectivity, content analysis coding ought to be conducted by coders who are at arm's-length with regard to the research.' Dr Neuendorf observed that the authors' 'selection of themselves as coders is inappropriate because they are not blind to the purpose of the research or independent of each other.'

Dr Neuendorf's comments on the process and the coding further reinforce our assertion that several errors were made in the process of developing the argument. The publication does not provide scientific support for either a discrepancy among ExxonMobil's climate change communications, or a claim that ExxonMobil misled the public.

In light of the authors' comparison of two unlike data sets, their reliance on limited and targeted data sets, and their questionable coding practices, the conclusions set forth in the publication cannot be credited.

Any data that support the findings of this study are included within the article.