How do we know about resilience? An analysis of empirical research on resilience, and implications for interdisciplinary praxis

Data were retrieved from the Institute of Scientific Information's (ISI) Web of Science between March and November 2010. The Social Sciences Citation Index and the Arts and Humanities Citation Index were used to identify the 'social science papers' and the Science Citation Index Expanded was used to identify the 'ecological papers'.

A key word search using the term 'resilien*' to search the title, keywords and abstract entries was employed. It excluded 'book reviews' in the case of the social science papers and was limited to 'articles' in the case of the ecological papers. The time span for the papers in the search was not limited.

The data were then refined according to subject area. In the case of the social science papers, papers in the subject areas deemed peripheral to the social science focus or that were known to be included in the ecology search were excluded. The search of the social science papers included those with the following subject areas (the following list is those fields that returned more than 10 papers in this initial search): Anthropology or Archaeology or Area Studies or Behavioural Sciences or Business (and Finance) or Communication or Criminology and Penology or Demography or Economics or Education (and Education and Special) or Environmental Studies or Ethics or Ethnic Studies or Family Studies or Industrial Relations and Labour International Relations or Geography or Gerontology or History or History and Philosophy of Science or Humanities or Philosophy or Planning and Development or Law or Management or Political Science or Psychiatry or Psychology (Applied and Developmental and Educational and Multidisciplinary and Social) or Public Administration or Public Health or Social Issues or Social Sciences (Biomedical and Interdisciplinary) or Social Work or Urban Studies or Women's Studies.

The search of the ecological papers included those papers with the following subject areas: Ecology or Environmental Studies or Environmental Sciences or Marine and Freshwater Biology or Limnology or Zoology or Water Resources or Forestry or Plant Sciences or Mathematical and Computational Biology or Oceanography or Toxicology or Soil Science or Evolutionary Biology or Fisheries or Biodiversity Conservation or Biology or Soil Science or Urban Studies or Entomology or Parasitology or Palaeontology.

Data sets from the social science and the ecological searches were compared, removing duplicates. Following this process, some 3759 social science entries and 2789 ecological entries formed our population data.

A pilot study of a sample of papers was undertaken by all the authors to review the range and type of papers to determine appropriate categories for analysis.

A random selection of the total social and ecological data sets (using Excel random selection formulae) was generated, resulting in a sample of 337 papers.

Of these 337 papers, 18 social and 116 ecological papers were excluded because they were not relevant (i.e., they were not actually about resilience), based on the expert judgements of the authors Ellemor and Glaister. A further seven were excluded because they were written in a language other than English, leaving 106 and 91 social and ecological studies, respectively.

Only empirical papers were included in our study, as we were interested to discern the empirical foundations for theories of resilience, and to see, in turn, if resilience is a concept that gives rise to empirical investigation. The test of empirical applied here is if a publication both explains its method and presents primary data. Of the 106 and 91 social and ecological papers, a further 27 social and 15 ecological papers were excluded because they did not contain any original data, leaving a sample of 79 and 76 social and ecological papers, respectively. Finally, another 6 papers were excluded from the social set because closer examination showed that they investigated social–ecological systems (i.e., sat at the intersections of ecological and social research rather than being clearly in one discipline or the other).

The data set, though randomly generated, only contains those articles that appear in the ISI databases. It is primarily limited by the fact that it does not contain books and book chapters. It also excludes non-English language papers.

Each journal article was read by one of the five authors. Codes were allocated to answer 15 questions, the most pertinent of which queried the sources of change events (e.g. human or ecological), the temporal and spatial scales of studies, and study design (table 2). Equivalent, though not always the same, codes/categories were used for each data set. Basic statistical analysis of the data using contingency tables and χ² with α set to 0.05 (Sokal and Rohlf 1995) was undertaken to explore prospective differences between ecological and social studies, which we now discuss.

Comparisons between the two sets of studies revealed some marked differences in the way resilience is studied. Social science studies were overwhelmingly focused on individuals whereas ecological research focused more on ecological populations and communities (table 3) and this difference in the dominant entities of study was statistically significant. Social studies focused almost entirely on human sources of change events (such as economic and political change, sources of stress such as illness or family violence) (table 3(b)). In contrast, both human (e.g. climate change) and ecological change (e.g. hurricane, drought) events were well represented in ecological research (table 3(b)), and this difference between ecological and social studies was also statistically significant (table 3(b)).

Few ecological studies (21%) defined resilience or provided a reference to the meaning of the word, whereas 40.5% of social science studies provided a definition, a difference between social and ecological studies that is once again statistically significant (χ² = 6.2,P = 0.013).

The temporal and spatial scales over which studies were carried out differed between social science and ecological research. Unsurprisingly, studies over short time periods were the most common overall but ecological studies comprised many more long term studies than found in the social sciences (table 4(a)). Similarly, both research fields were dominated by work done at small spatial scales (local scales) but particularly so in ecological research, whereas in the social sciences studies at a national level were almost as common as locally based research (table 4(b)). Global-scale and international studies were uncommon.

Perhaps the most interesting results lie in the comparison of research designs used in the social and ecological studies, and their means of inference about cause-and-effect. Social science research was dominated by one-off, explorative surveys that investigated attributes of entities (table 4(c)). There were a few studies of entities over time, but before/after research around a particular change event was uncommon. Unsurprisingly given ethical considerations, experiments were unusual in social science research but there were also no examples in our sample of research done over spatial gradients of a change event (table 4(c)). In contrast, ecological research was spread much more evenly across types of study designs, with one-off surveys comprising the least common study type.

Whilst there are diverse definitions of resilience, one of the common features of most definitions is that the concept refers to how a system experiences and responds to change (Manyena 2006, Thrush et al 2009, and table 1). Our analysis of the studies reviewed indicate that understanding of how systems respond to change is limited by an overwhelming focus at the level of individual (social studies) or population and community (ecological studies). For the latter, a focus on communities is somewhat understandable given that testing hypotheses on entire ecosystems is logistically difficult (Thrush et al 2009). Nevertheless, we found it surprising that the number of studies on resilience that put results into an ecosystem context were few. Systems, whether understood in a social sense (in terms of the structure and function of relationships at a family, neighbourhood, workplace or community) or in an ecological sense (i.e. at an ecosystem level), do not form a large proportion of our sample, although more so for the ecological than social papers. Consequently resilience research that can address questions about systems are a lot less common than the literature about resilience assumes. Masten and Obradovic (2008) also note that in human development research on resilience there is little research connecting individual resilience to larger social systems.

A key insight generated by resilience research concerns the largely unpredictable and non-linear nature of change (Holling 1996, Manyena 2006, Masten and Obradovic 2008). Yet, one of the disturbing findings of our review is that the majority of studies in our sample consider very short temporal scales of change. Whilst a larger proportion of the ecological studies are undertaken over a longer period, the limited temporal scale considered in the studies overall potentially restricts the extent to which system processes and responses to change events can be understood. For instance, delayed responses to change, and the synergistic and cumulative interaction of multiple change events may not be captured within such studies. There is also a critical time element in the way in which social systems response to change in terms of how people and institutions anticipate and prepare for change, recover and learn, which influence resilience. These are important temporal dimensions of change that are theorized about in the resilience literature, yet our study concludes that the empirical body of work on this dimension is limited.

The spatial scale of analysis is significant in determining the extent to which relevant processes of change and response are identified and explained, and indeed, scale and cross-scale dynamics are key concerns within resilience (see Berkes et al 2002, Cumming et al 2006, Peterson et al 1998). The overwhelmingly local spatial focus of the studies reviewed, particularly for the ecological studies, limits the consideration of wider scale processes. While local studies may consider the broader, spatial context in which they sit, this is not equivalent to capturing data over large spatial scales, which can provide definitive information about when and how findings at small scales can be (if ever) scaled up or applied to other seemingly, similar locations.

There is more diversity in the methods applied to study resilience in the ecological studies we reviewed compared with the social science work, which largely draws upon one-off, explorative surveys. There are surprisingly few 'before/after' studies undertaken in the social sciences, which may reflect the way in which resilience is most often interpreted in the reviewed papers as a condition (rather than an outcome) and distinct from particular change events, in contrast to interpretations in the ecological sciences that focus on how entities respond to change over time. Nevertheless, the predominance of one-off surveys in social studies of resilience may signal that the benefits of using study designs that can provide greater inferential capacity have not been fully appreciated. Before/after studies or those designed to take advantage of spatial or temporal gradients in a change event can provide much more direct tests of hypotheses of cause-and-effect than open-ended, explorative surveys (Downes et al 2002).

It is often said that research on resilience should seek to enhance understanding of the interactions between social and ecological systems across a range of spatial and temporal scales (Leslie and McLeod 2007, Armitage et al 2009, Fischer et al 2009, Chapin et al 2010, Polasky et al 2011). Many studies emphasize the importance of having multidisciplinary teams in which ecological and social scientists work together to improve conservation of ecosystems and the protection of the societies dependent upon them. Increasingly there is a recognition amongst resilience scholars of the need for trans-disciplinary approaches to research (Miller et al 2008, Vogel et al 2007), that rethink not only disciplinary boundaries but also the research–policy–practice interface. The success of such approaches depend on there being not only a recognition of the great diversity in meanings of words such as 'resilience' but also the considerable difference in understandings of the characteristics of resilient systems and the evidence upon which such understandings are based. Our research suggests that such approaches may suffer difficulties in realizing a robust understanding of what factors confer resilience upon systems, unless this diversity in the types of evidence available in the social and ecological literatures is recognized, appreciated and reconciled. Consequently, such approaches will need to devote considerable time and effort to building such an appreciation and reconciliation, but highly beneficial, new research directions and approaches are likely to result. It is suggested that future research needs to give greater attention to capturing cross-scale processes and change in systems over time, using research designs that are to some degree common with respect to spatial and temporal scale and methods for collecting evidence about cause-and-effect.