Frontiers in Ecology
and the Environment
Coordinated distributed experiments:
an emerging tool for testing global
hypotheses in ecology and
environmental science
Lauchlan H Fraser, Hugh AL Henry, Cameron N Carlyle, Shannon R White, Carl Beierkuhnlein,
James F Cahill Jr, Brenda B Casper, Elsa Cleland, Scott L Collins, Jeffrey S Dukes, Alan K Knapp,
Eric Lind, Ruijun Long, Yiqi Luo, Peter B Reich, Melinda D Smith, Marcelo Sternberg, and
Roy Turkington
Front Ecol Environ 2012; doi:10.1890/110279
This article is citable (as shown above) and is released from embargo once it is posted to the
Frontiers e-View site (www.frontiersinecology.org).

Please note: This article was downloaded from Frontiers e-View, a service that publishes fully edited
and formatted manuscripts before they appear in print in Frontiers in Ecology and the Environment.
Readers are strongly advised to check the final print version in case any changes have been made.

esa
© The Ecological Society of America

www.frontiersinecology.org

REVIEWS REVIEWS REVIEWS

Coordinated distributed experiments: an
emerging tool for testing global hypotheses
in ecology and environmental science
Lauchlan H Fraser1*, Hugh AL Henry2, Cameron N Carlyle1,3, Shannon R White4, Carl Beierkuhnlein5,
James F Cahill Jr4, Brenda B Casper6, Elsa Cleland7, Scott L Collins8, Jeffrey S Dukes9, Alan K Knapp10,
Eric Lind11, Ruijun Long12, Yiqi Luo13, Peter B Reich14,15, Melinda D Smith16, Marcelo Sternberg17, and
Roy Turkington3
There is a growing realization among scientists and policy makers that an increased understanding of today’s
environmental issues requires international collaboration and data synthesis. Meta-analyses have served this
role in ecology for over a decade, but the different experimental methodologies researchers use can limit the
strength of the meta-analytic approach. Considering the global nature of many environmental issues, a new
collaborative approach, which we call coordinated distributed experiments (CDEs), is needed that will control
for both spatial and temporal scale, and that encompasses large geographic ranges. Ecological CDEs, involving
standardized, controlled protocols, have the potential to advance our understanding of general principles in
ecology and environmental science.
Front Ecol Environ 2012; doi:10.1890/110279

E

cology as a science began well over a century ago,
when it was based primarily on observational studies
(McIntosh 1985). Toward the end of the 20th century,
this led to a more experimental, reductionist approach,
focused on testing hypotheses and developing theories.
Controlled, manipulative experiments have shaped our
understanding of ecological principles considerably over
the past few decades. However, although individual studies are good for testing and developing theories, they
yield site-specific information that is difficult to extrapolate to provide broad generalizations. Today, emerging
globally relevant questions in basic and applied ecology
require a reconsideration of what approaches would be
best for understanding large-scale patterns and processes.

In a nutshell:
• Many environmental issues have become global problems that
require collaborative experimental research at an international
scale
• We recommend the use of coordinated distributed experiments (CDEs), standardized experiments to control for spatial
and temporal scales across a wide, international geographic
range
• The use of CDEs will lead to more efficient international collaborations, addressing fundamental basic and applied questions in ecology and environmental science

1

Department of Biological Sciences and Natural Resource Sciences,
Thompson Rivers University, Kamloops, Canada *(lfraser@tru.ca);
2
Department of Biology, University of Western Ontario, London,
Canada; 3Department of Botany and Biodiversity Research Centre,
University of British Columbia, Vancouver, Canada (for the address
details of the remaining authors please see the last page of this paper)
© The Ecological Society of America

It is no longer sufficient to ask whether global warming
can alter plant or animal growth, whether acid rain has
impacts on aquatic trophic structure, or whether habitat
fragmentation reduces biodiversity. Society requires more
than theory – it requires an understanding of “how, how
much, and when”. A major limitation to our ability to
extrapolate the results from experimental approaches has
been the scopes and scales of the studies that are practically and logistically feasible. For this reason, there have
been shifts in methodologies in ecology and the environmental sciences; for example, we have seen a return to
observational studies through the use of technological
advances in the monitoring and analysis of data (Collins
et al. 2006; Sagarin and Pauchard 2010). Another technique that has become widely used is meta-analysis, a
quantitative approach to reviewing, integrating, and
summarizing large numbers of independent studies
(Gurevitch and Hedges 2001).
Over the past two decades, a progressive increase in the
use of meta-analysis has advanced most academic disciplines, including ecology and the environmental sciences. The strength of the meta-analytic approach lies in
the objective and quantitative statistical methodology
that allows the synthesis of large numbers of studies
(Gurevitch et al. 1992; Arnqvist and Wooster 1995;
Harrison 2011). A few examples of ecological topics that
have been ideally suited to the meta-analytic approach
include the question of whether amphibian declines are a
worldwide phenomenon (Houlahan et al. 2000), the
effect of dispersal on species diversity (Cadotte 2006), the
response of ecosystem nitrogen (N) cycles to N addition
(Lu et al. 2011), the effects of grazing on plant traits (Díaz
et al. 2007), the relationship between plant species richwww.frontiersinecology.org

Coordinated distributed experiments
(a)

LH Fraser et al.
(b)

(c)

Figure 1. Examples of different rainfall manipulation methods conducted on grasslands with corresponding map locations in Figure 3;
(a) Germany, (b) Israel, and (c) Canada.

ness and productivity (Mittelbach et al. 2001), the
effects of controlling plant traits on litter decomposition
rates (Cornwell et al. 2008), and grassland biomass
responses to carbon dioxide (CO2) and N enrichment
(Lee et al. 2010). While there has been an increasing use
of meta-analysis, there is also a growing literature that is
critical, or at least cautionary, of this quantitative
approach. Indeed, Whittaker (2010) referred to metaanalyses as “mega-mistakes”, not necessarily to discourage the use of meta-analysis but to urge a more rigorous
approach to its use.
The limitations and criticisms of meta-analysis are well
documented (Hillebrand and Cardinale 2010; Harrison
2011). Ultimately, the robustness of any meta-analysis
relies on the individual studies selected for inclusion. For
this reason, it is important to exclude methodologically
weak studies and to ensure that the studies chosen are representative of all the methodologically robust studies that
have been conducted on a research topic. Even when a
representative sample of studies is obtained, meta-analysis
is only successful if the studies can be combined in a
meaningful manner. Two critical issues influence the subsequent success (in terms of scientific accuracy and
progress) of meta-analyses: (1) effect size metrics, based on
the calculation used to standardize the different types and
scales of possible data measurements and the power of the
effects, and (2) standardization of methodology amongst
studies. Although it is easy to imagine using a meta-analysis to test a basic ecological question – for instance, the
richness–productivity relationship among plant communities – the specific effect size metrics used can make a difference to the outcome and interpretation of the analysis.
For example, Whittaker (2010) reviewed three metaanalyses, all focused on plant species richness–productivity relationships, and found widely divergent results
among the analyses. Similarly, Hungate et al. (2009), in an
assessment of the outcome of four published meta-analyses
gauging the effect of elevated CO2 on soil carbon (C), also
found that each study reached a different conclusion.
One of the great benefits of meta-analysis is its quantitative approach to hypothesis testing and the collaborative nature of data compilation. Data used in meta-analyses are not necessarily published in the primary
peer-reviewed literature. Even if a published paper may
www.frontiersinecology.org

seem appropriate for inclusion in a meta-analysis, the
researcher conducting the meta-analysis often needs to
contact the original authors for further information on
methodology or may even request the raw data. This
engenders a positive spirit of inclusion and often encourages collaboration. In some cases, papers used in metaanalyses are not cited but rather included as supplemental
material and therefore are not indexed by ISI Web of
Science, likely because the list of references would otherwise be longer than journals usually allow (Kueffer et al.
2011). Meta-analyses provide a basis for important and
meaningful results that can also lead to an appropriate
concept for the future design of ecological experiments.
We contend that the answers to many questions in
ecology and the environmental sciences cannot be
addressed by meta-analysis alone. A core limitation in
the meta-analysis approach is that details of the original
study are beyond the control of those who conduct the
meta-analyses – instead, they are imposed at the time
each individual study was conducted. For examples of differences in experimental design see Figure 1 for precipitation treatments and Figure 2 for warming treatments.
Issues of scale, both temporal and spatial, including geographic variation, cannot be controlled retrospectively. A
logical solution is to use a study framework that reduces
these issues before the data are collected. Long-term
experiments, such as the Park Grass Experiment in the
UK, which began in 1856 (Silvertown et al. 2006), and
the US Long Term Ecological Research Network program
(Knapp et al. 2012), have for years been recognized as
vital to understanding ecological processes. However,
important global issues, such as climate change, N deposition, non-native invasive species, habitat fragmentation
and degradation, and biodiversity loss, urgently require
standardized controlled experiments on wide geographic
scales, preferably global in some cases, and ones that
range across biomes and latitudes in others. Coordinated
distributed experiments (CDEs) – experiments run in
parallel by several research groups in multiple locations
around the globe – have the advantage of simultaneously
addressing global environmental problems and exploring
general ecological theory, while offering the inherent precision of controlled experiments.
Multi-site investigations that we believe resemble eco© The Ecological Society of America

LH Fraser et al.
(a)

Coordinated distributed experiments
(b)

(c)

Figure 2. Examples of different warming manipulation methods conducted on grasslands with corresponding map locations in Figure
3; (a) China, (b) Canada, and (c) Canada.

logical CDEs are not unknown (eg Hector et al. 1999;
Pywell et al. 2002; Wright et al. 2004; Royer et al. 2009;
Heisler-White et al. 2009). However, experiments that
include multiple geographically distinct sites are often
limited spatially by the logistical challenges and cost
when a single group conducts the research. Recently,
however, a handful of collaborative ecological research
networks have been established that facilitate integration
of data across large spatial scales. For example,
FLUXNET (Baldocchi et al. 2001), a global network of
500 micrometeorological tower sites from 30 regional
networks across five continents, has been effective at
combining ecosystem CO2 flux data. Likewise, the US
National Ecological Observatory Network (NEON;
Keller et al. 2008), while restricted to the confines of a
single country, is being designed to specifically include
sites from 20 eco-climate domains, and aims to facilitate
the coordinated study of ecological responses to globalchange factors across a broad geographic range.
While the two examples above describe networks of
observational studies, networks of experimental studies
have also been established, including PRECIPNET
(Weltzin et al. 2003), a network of rainfall manipulation
studies carried out primarily in the US; NitroEurope
(Sutton et al. 2007), a network of N manipulation sites
across Europe; ECOCRAFT (Medlyn et al. 1999), a network of field experiments to examine the effects of elevated CO2 on European forest trees; and TreeDivNet, an
international network of forest researchers who study the
relationship between tree diversity and ecosystem function. The Free-Air Carbon dioxide Enrichment (FACE)
studies, although not a single coordinated network, also
represent a consortium approach to a network. Techniques were not standardized, but most studies in the
network did attempt to achieve uniformity, so that syntheses and meta-analyses would be relatively simple to
carry out (eg Reich et al. 2006; Leakey et al. 2009).
Nevertheless, networks of experiments have typically
been restricted geographically (primarily to the US or
Europe) and they have not featured consistent experimental designs or data collection methods across sites. In
addition, global meta-analyses may not reflect important
patterns within specific biomes. The latter can also be
said of TERACC (Rustad 2008; now replaced by
© The Ecological Society of America

INTERFACE), an international research network of
global change experiments linking experimentalists,
ecosystem modelers, and Earth-system modelers. So far,
the main outcome of research networks has been the
building of data repositories for data sharing, often leading to more meta-analyses.

n Examples of ecological CDEs
Perhaps the first ecological study that we are aware of
that could be termed a CDE was a series of ozone experiments conducted from 1980 to 1987 as part of the
National Crop Loss Assessment Network (Heagle 1989).
At each of five locations in the US, open-top chambers
were used to expose important agronomic crop species to
a realistic range of ozone treatments. These studies, when
complemented by companion experiments, provided the
best evidence that low-level ozone pollution can affect
plant growth and plant community composition, and
they also uncovered the mechanisms behind species differences in sensitivity (Reich 1987; Heagle 1989). A second example of an early ecological CDE was a Canadianled investigation to determine whether plant
competition intensity was independent of plant biomass
(Reader et al. 1994). The experiment was inspired by two
prevailing theories of plant community organization that
differed in their prediction regarding the relationship
between competition and site productivity. One theory
(Grime 1973) predicted that the intensity of competition
increases with an increase in productivity, while the
other (Newman 1973; Tilman 1988) predicted that competition would remain consistent with increasing site
productivity. Through a standard neighbor removal
experiment carried out in 12 locations on three continents by 20 research teams, it was found that competition
intensity was not related to neighbor biomass, except at a
single site where the range in neighbor biomass was
greater than all the other sites. This study highlighted the
importance of an adequate range in the productivity gradient and the use of a consistent index for the measurement of competition.
There have been other examples of the CDE approach
(Table 1). A number of research topics have been or are
being addressed, including: the effects of biodiversity on
www.frontiersinecology.org

Coordinated distributed experiments

LH Fraser et al.

ecosystem function (BIODEPTH; Hector et al. 1999),
global warming (ITEX; Henry and Molau 1997), combinations of warming and rainfall manipulation
(CLIMOOR/VULCAN/INCREASE; Beier et al. 2004;
Wessel et al. 2004), and nutrient additions (NITREX and
EXMAN, Wright and Rasmussen 1998; NutNet, Adler et
al. 2011). Other networks, such as LIDET (Gholz et al.
2000), which examined differences in rates of litter
decomposition, and LINX I and II (Webster et al. 2003;
Mulholland et al. 2008), which used 15N tracers to explore
stream N cycling, provided useful comparisons of ecosystem processes among sites but without experimental
manipulation of the environment. For all of these CDEs,
the emphasis has been on simple, testable questions and
standardized methodology. However, in most cases, with
the exception of the Nutrient Network (NutNet) and the
International Tundra Experiment (ITEX), the geographic
range has been limited.
ITEX provides some of the first evidence of the effects
of global warming on ecosystems and was the first international effort to understand the potential future impacts
of climate change. ITEX is circumpolar, involving 10
countries and 26 sites across the Arctic tundra.
Considering that global warming is predicted to impact
the higher latitudes first and with the greatest intensity,
the experiment was instrumental in gaining an under-

standing of climate-change effects. All plant species
investigated under ITEX responded to warming in terms
of either biomass changes, phenology, or both; there was,
however, no general pattern in terms of type or magnitude of response over the limited time span of the experiment (Henry and Molau 1997), although the cumulative
effects of long-term warming may be greater than
expected (Elmendorf et al. 2011). The results suggest that
not only are ecological CDEs important for a general
understanding of pattern and process, but that long-term,
multi-site investigations are needed to understand
changes in response and how those changes vary among
ecosystems (Knapp et al. 2012).
NutNet was developed to address how grasslands are
affected by global changes in eutrophication, mainly
through atmospheric nutrient deposition, and large herbivore communities (Firn et al. 2011; Adler et al. 2011;
Stokstad 2011). NutNet began partly in response to a
frustration among researchers with the lack of any data
on terrestrial systems that could be used in meta-analyses
(Hillebrand et al. 2007; Gruner et al. 2008). The experimental approach was to study the addition of three nutrient categories (N, phosphorus, and potassium plus
micronutrients), and the multiple combinations of all
nutrient categories and fencing to exclude herbivores.
Currently, 40 sites on five continents are implementing

Table 1. Summary of network ecological CDEs in ecology and environmental sciences
Network meta-experiment

Treatments

Technology

Years active

Number of sites (location)

BIODEPTH

Species diversity

Constructed
communities

1996–1999

8 (grasslands – Europe)

CLIMOOR/VULCAN/INCREASE

Warming, rain
exclusion

Retractable curtains

1998–present

6 (heathland/shrubland –
Europe)

Competition Intensity

Neighbor removal

Herbicide

1992

12 (old field/grassland – North
America/Europe/Australia)

GLOWA Jordan River

Rain addition/
exclusion

Irrigation and rainout
shelters

2001–present

4 (shrubland/dwarf shrubland –
Israel)

ITEX

Warming

Open top chambers

1992–present

26 (Arctic tundra)

LINX I

Site comparison
only

15

N tracer addition

1996–2001

16 (streams – US)

LINX II

Site comparison
only

15

N tracer addition

2001–2006

8 (9 streams per location – US)

LIDET

Site comparison
only

Decomposition –
litter bags

1990–2000

28 (North and Central
America)

NCLAN

Ozone

Open top chambers

1980–1987

5 (agricultural field – US)

NITREX

N addition or
removal

Irrigation and rainout
shelters

Established
late 1980s

8 (coniferous forest – northwest Europe)

EXMAN

N quantity and
quality

Irrigation and
rainout shelters

Established
1980s

6 (coniferous forest – Europe)

NutNet

Factorial N, P, K
addition/vertebrate
herbivore exclosure

Fertilizer application
and fencing

2006–present

45 (grasslands – global)

www.frontiersinecology.org

© The Ecological Society of America

LH Fraser et al.

the full experimental protocol, while five more sites are
implementing only the factorial nutrient experiment.

n What is an ecological CDE?
Based in part on the examples cited above, we offer the
following defining attributes (1–5) and probable characteristics (6–8) of ecological CDEs.

Coordinated distributed experiments
(6) Synchronized data collection

Research groups should run the experiment simultaneously if the different study sites share similar seasonal climates. Alternatively, studies should run during the same
growing season if northern and southern hemispheres are
included, to reduce the likelihood of temporal variation
in global climate or atmospheric composition affecting
research outcomes.

(1) Hypothesis-driven experimental study

The research question must be direct, testable, and determined through consensus of the collaborative team. This
requires that the problem be condensed to an overarching
general question that has potential validity in different
geographic locations and in ecosystems with differing
species pools, legacies (eg natural and anthropogenic disturbances), biodiversity, climates, and so forth. This
approach forces the research team to identify general and
specific research questions.

(7) Multiple investigative teams

Networks that include teams of investigators from many
locations and disciplines reduce costs for each individual
team while promoting multidisciplinary collaborations,
and increasing the potential for new insights to develop
from the sharing of ideas. Having researchers close to
each proposed study location reduces travel costs and
individual time commitments.
(8) Low cost and low maintenance

(2) Multiple geographic locations

Global issues require multi-site experiments at a transcontinental spatial scale. The majority of meta-analyses
seem to be drawn from studies in North America and
Europe, leading to the possibility of a bias in outcomes
(Martin et al. 2012). The larger the geographic range a
CDE encompasses, the greater the potential for understanding general principles or identifying the underlying
causes of different responses between site locations.
(3) Standardized research design

Every research team at each location must follow the
same methodology and protocol (based on collaborative
consensus) outlined in the experimental design.
(4) Standardized data and coordinated data
management

Ecologists use a variety of data management methods
even when recording identical phenomena. Standardizing input forms and data storage allows for more
straightforward data compilation and analysis and is critical for taking advantage of the power of CDEs. Such data
standardization also facilitates the generation of metadata
and increases the potential for alternative uses of the
experimental data.
(5) Intellectual property sharing

A common agreement on data ownership and intellectual property must be based on a consensus among the
research teams, so as to avoid conflicts and to maximize
commitment of all the partners to the common
approach.
© The Ecological Society of America

Low-cost activities will encourage collaboration among
researchers around the globe, including those in developing countries with limited finances and in sites that are
remote and potentially difficult to access by North
American and European researchers. It is currently a
challenge to attract dedicated funding for global collaborative projects, so that many ecological CDEs will need to
be conducted on existing operating budgets. However,
ecologists and environmental scientists must be more
vocal about educating policy makers and research agencies about the importance of this research, even though it
may be expensive by historical ecological standards.
Ecological CDEs require coordination and uniformity
in design and application. Because all the data are collected from the same replicated experimental design, the
statistical approach to data interpretation is very different
from that used in meta-analyses. Meta-analysis involves
complex statistics specific to and designed for this purpose (Hedges and Olkin 1985). Statistical analysis of ecological CDEs relies on arguably simpler, traditional tests,
such as ANOVA, mixed-effects models, and multilevel/
hierarchical models (Gelman and Hill 2007).
Perhaps the most challenging aspect of ecological
CDEs is developing the networks and funding the experiments. Ultimately, it takes one individual, or a small
group of people, to assume a leadership role in recruitment and coordination of the network. At the recruitment stage, it is imperative that the basic research question is well formulated and that it addresses a critical area
of interest. Once the question is established, a starting point
for recruitment should include personal contacts and internet searches for researchers that are currently engaged in
similar research, ascertained through recent publications.
Protocol development will require a critical mass of
www.frontiersinecology.org

Coordinated distributed experiments

LH Fraser et al.

Figure 3. World map identifying the locations of climate-change experiments in grasslands, numbered from west to east: 1 –
Vancouver Island, Canada; 2 and 3 – California; 4 – British Columbia, Canada; 5 – Nevada; 6 – California; 7 – Alberta,
Canada; 8 – Utah; 9 – Colorado; 10 – New Mexico; 11 – Wyoming; 12 – Colorado; 13 and 14 – Saskatchewan, Canada; 15 –
Manitoba, Canada; 16 – Oklahoma; 17 and 18 – Kansas; 19 – Minnesota; 20 – Tennessee; 21 and 22 – Ontario, Canada; 23 –
Chubut, Argentina; 24 – Wales, UK; 25 – Cumbria, UK; 26 – Derbyshire, UK; 27 – Oxfordshire, UK; 28 – LanguedocRoussillon, France; 29 – Switzerland; 30 – Bayreuth, Germany; 31 – Israel; 32 – Tibetan Plateau, China; 33 – Northern
Mongolia; 34 and 35 – Tibetan Plateau, China; 36 – Tasmania, Australia; 37 – Bogong High Plains, Australia; 38 – New
Zealand. Labels for some experiments have been moved slightly from their true location for legibility.

invested researchers, and while this can be done “remotely”
via e-mail, listservs, and social media, we recommend an
organized workshop, which can be coordinated to run during an international conference to maximize attendance.
We encourage organizing committees of international ecology conferences to accommodate CDE workshops. A welldesigned protocol can serve as a recruitment tool to enlarge
the network, especially in identified geographical gaps, and
as the basis for subsequent grant proposals. Once a CDE is
underway, annual or biennial workshops can serve to
strengthen the network and maximize productivity. Few
funding agencies support large intercontinental ecological
experiments, but we expect that the international demand
for suitable funding mechanisms will increase as CDEs
become more common. We strongly recommend that
national and transnational research agencies and foundations establish and promote international agreements for
the funding of participating researchers in CDEs.

n A proposal for a CDE to test the effects of drought
in grasslands

In grassland ecosystems, water is the major limiting factor; changes in precipitation patterns and increased risk
of drought will therefore likely have a major impact on
grassland ecosystems. Greenhouse-gas emissions have led
www.frontiersinecology.org

to increases in atmospheric CO2 concentration and mean
annual global temperature, causing alterations in mean
annual global precipitation (Meehl et al. 2007). A key prediction of altered precipitation is an increased risk of
drought (Reichstein et al. 2007), and yet relatively few
studies have investigated the effect of potential changes in
precipitation on terrestrial ecosystems (Weltzin et al.
2003). Predicted changes in precipitation vary across grasslands globally (increase, no change, decrease) but all models agree that, globally, precipitation will be more variable
with more extreme events (eg droughts). A global chronic
drought experiment is therefore needed. Regions of the
world can vary widely in projected trends in precipitation changes (Meehl et al. 2007), which may partially
explain why there are so few precipitation studies and
the problems of comparing those study results.
Despite an increase in the number of published papers
investigating climate change in grasslands over the past
two decades, it is still difficult to compare treatment
effects between studies. The ideas from this review
paper largely emerged from a recent symposium and
workshop, Climate Change Experiments in Temperate
Grasslands, held at the International Association for
Vegetation Science meeting in Lyon, France, in 2011
(see White et al. [2011] for meeting report). At the symposium, independent results from 14 field climate© The Ecological Society of America

LH Fraser et al.

change experiments on three continents were inconsistent and it was not possible to identify broad patterns of
grassland response to temperature and precipitation
manipulations. One of the likely reasons were the many
differences in experimental methodology, including the
size of rain-out shelters (structures built to reduce or
restrict precipitation from reaching the ground), the
design of the rain-out shelters, the magnitude of water
additions, and the design of open-top chambers to
increase temperature (White et al. 2011).
Most grassland climate-change experiments have
been carried out in North America and Europe (Figure
3). This geographic bias limits our ability to determine
general global patterns of grassland response to global
change. The large variation in design and scale of experiments further limits the potential for comparison and
development of general principles (Figures 1 and 2).
Two meta-analyses have investigated experimental climate-change effects on plant communities (Rustad et al.
2001; Wu et al. 2011), but the same problems inherent
in all meta-analyses apply (explained above). We identify the need for a collaborative effort of researchers
from around the globe to tackle what we consider to be
the most relevant and pressing question about grassland
ecosystem responses to climate change, and to do so in
the context of a CDE.
The proposed CDE would ask the general question
“are the effects of drought on temperate grassland ecosystems consistent across study sites?” The research design
would involve installations of standardized, low-cost,
passive rain-out shelters (eg Heisler-White et al. 2009;
Figure 4) over grassland ecosystems around the world.
Passive rain-out shelters require little maintenance and
allow for relatively easy set-up and monitoring. Response
variables to measure could include plant abundance and
diversity, soil microbial communities, levels of soil C and
N, and soil moisture. Of course, other response variables
would also likely be measured, including plant traits and
potentially soil biota, but a bare minimum of site characteristics (such as soil properties) and response variables
would be required for a site to be included in the CDE.
Such an approach will facilitate testing for general
effects across wide site differences.
Advantages of the CDE approach over the meta-analysis approach are evident when examining the consequences of rainfall manipulations. Unlike ecological
CDEs, meta-analyses are not designed to test for withinsite differences or the complexity of interaction effects.
For example, if plant biomass within half of the rain-out
shelters in one site responded negatively and the other
half in the same site responded positively, a meta-analysis
might interpret the site response as having no overall
effect. Because data from an ecological CDE are standardized, it is possible to test for interactive site differences,
both within a site and among sites, as well as accounting
for covarying climate and habitat variables, assuming
these properties are also monitored as a part of the CDE.
© The Ecological Society of America

Coordinated distributed experiments

Figure 4. Example of the style of rain-out shelter proposed for
the CDE to test the effects of drought in grasslands (from US,
map location in Figure 3 is 12).

A potentially important property that varies between
geographic locations is year-to-year variation in annual
precipitation. If the coefficient of variation over two
decades in one site is greater than another, the rain-out
shelter treatment might have less of an effect on the site
with a greater coefficient of variation. An ecological
CDE will be able to answer this question by comparing
different grassland sites with different year-to-year variation in annual precipitation; thus, temporal and spatial
scale can be controlled. Standardized control of experiments and control of detail in measurement of response
across a wide geographic range enhances the power of the
CDE approach over that of meta-analysis.

n Conclusions
Anthropogenic activities are having effects at global
scales. For many of the environmental issues we face it is
no longer possible for a single investigator, or even a team
of investigators, to address large scale ecological questions
at the local or regional level. Ecological CDEs that rely
on international collaboration are needed to address
worldwide problems; these will help us to understand ecological patterns and processes and to develop solutions
that will help with environmental management.

n References

Adler PB, Seabloom EW, Borer ET, et al. 2011. Productivity is a
poor predictor of plant species richness. Science 333: 1750–53.
Arnqvist G and Wooster D. 1995. Statistical power of methods of
meta-analysis. Trends Ecol Evol 10: 460–61.
Baldocchi DD, Falge E, Gu L, et al. 2001. FLUXNET: a new tool to
study the temporal and spatial variability of ecosystem-scale
carbon dioxide, water vapor and energy flux densities. B Am
Meteorol Soc 82: 2415–35.
Beier C, Emmett BA, Gundersen P, et al. 2004. Novel approaches
to study climate change effects of terrestrial ecosystems in the
field – drought and passive night time warming. Ecosystems 7:
583–97.
Cadotte MW. 2006. Dispersal and species diversity: a meta-analysis. Am Nat 167: 913–24.
www.frontiersinecology.org

Coordinated distributed experiments
Collins SL, Bettencourt LMA, Hagberg A, et al. 2006. New opportunities in ecological sensing using wireless sensor networks.
Front Ecol Environ 4: 402–07.
Cornwell WK, Cornelissen JHC, Amatangelo K, et al. 2008. Plant
species traits are the predominant control on litter decomposition rates within biomes worldwide. Ecol Lett 11: 1065–71.
Díaz S, Lavorel S, McIntyre S, et al. 2007. Plant trait responses to
grazing – a global synthesis. Glob Change Biol 13: 313–41.
Elmendorf SC, Henry GHR, Hollister RD, et al. 2011. Global
assessment of experimental climate warming on tundra vegetation: heterogeneity over space and time. Ecol Lett 15:
164–75.
Firn J, Moore JL, MacDougall AS, et al. 2011. Abundance of introduced species at home predicts abundance away in herbaceous
communities. Ecol Lett 14: 274–81.
Gelman A and Hill J. 2007. Data analysis using regression and
multilevel/hierarchical models. Cambridge, UK: Cambridge
University Press.
Gholz HL, Wedin DA, Smitherman SM, et al. 2000. Long-term
dynamics of pine and hardwood litter in contrasting environments: toward a global model of decomposition. Glob Change
Biol 6: 751–65.
Grime JP. 1973. Competitive exclusion in herbaceous vegetation.
Nature 242: 344–47.
Gruner DS, Smith JE, Seabloom EW, et al. 2008. A cross-system
synthesis of consumer and nutrient resource control on producer biomass. Ecol Lett 11: 740–55.
Gurevitch J, Morrow LL, Wallace A, et al. 1992. A meta-analysis
of field experiments on competition. Am Nat 140: 539–72.
Gurevitch J and Hedges LV. 2001. Meta-analysis: combining the
results of independent experiments. In: Scheiner SM and
Gurevitch J (Eds). Design and analysis of ecological experiments. New York, NY: Chapman & Hall.
Harrison F. 2011. Getting started with meta-analysis. Methods Ecol
Evol 2: 1–10.
Heagle AS. 1989. Ozone and crop yield. Annu Rev Phytopathol 27:
397–423.
Hector A, Schmid B, Beierkuhnlein C, et al. 1999. Plant diversity
and productivity of European grasslands. Science 286:
1123–27.
Hedges LV and Olkin I. 1985. Statistical methods for meta-analysis. New York, NY: Academic Press Inc.
Henry GHR and Molau U. 1997. Tundra plants and climate
change: the International Tundra Experiment (ITEX). Glob
Change Biol 3: 1–9.
Heisler-White JL, Blair JM, Kelly EF, et al. 2009. Contingent productivity responses to more extreme rainfall regimes across a
grassland biome. Glob Change Biol 15: 2894–2904.
Hillebrand H and Cardinale BJ. 2010. A critique for meta-analysis
and the productivity–diversity relationship. Ecology 91:
2545–49.
Hillebrand H, Gruner DS, Borer ET, et al. 2007. Consumer versus
resource control of producer diversity depends on ecosystem
type and producer community structure. Proc Natl Acad Sci
104: 10904–09.
Houlahan JE, Findley CS, Schmidt BR, et al. 2000. Quantitative
evidence for global amphibian population declines. Nature
404: 752–55.
Hungate BA, van Groenigen K, Six J, et al. 2009. Assessing the
effect of elevated carbon dioxide on soil carbon: a comparison
of four meta-analyses. Glob Change Biol 15: 2020–34.
Keller M, Schimel DS, Hargrove WW, Hoffman FM. 2008. A continental strategy for the National Ecological Observatory
Network. Front Ecol Environ 6: 282–84.
Knapp AK, Smith MD, Hobbie SE, et al. 2012. Past, present and
future roles of long-term experiments in the LTER network.
BioScience 62: 377–89.
www.frontiersinecology.org

LH Fraser et al.
Kueffer C, Niinemets U, Drenovsky RE, et al. 2011. Fame, glory
and neglect in meta-analyses. Trends Ecol Evol 26: 493–94.
Leakey ADB, Ainsworth EA, Bernacchi CJ, et al. 2009. Elevated
CO2 effects on plant carbon, nitrogen, and water relations: six
important lessons from FACE. J Exp Bot 60: 2859–76.
Lee M, Manning P, Rist J, et al. 2010. A global comparison of
grassland biomass responses to CO2 and nitrogen enrichment.
Philos T Roy Soc B 365: 2047–56.
Lu M, Yang Y, Luo Y, et al. 2011. Responses of ecosystem nitrogen
cycle to nitrogen addition: a meta-analysis. New Phytol 189:
1040–50.
Martin LJ, Blossey B, Ellis E. 2012. Mapping where ecologists
work: biases in the global distribution of terrestrial ecological
observations. Front Ecol Environ 10: 195–201.
McIntosh RP. 1985. The background of ecology: concept and theory. Cambridge, UK: Cambridge University Press.
Medlyn BE, Badeck FW, De Pury DGG, et al. 1999. Effects of elevated [CO2] on photosynthesis in European forest species: a
meta-analysis of model parameters. Plant Cell Environ 22:
1475–95.
Meehl GA, Stocker TF, Friedlingstein WDCP, et al. 2007. Global climate projections. In: Solomon S, Qin D, Manning M, et al. (Eds).
Climate change 2007: the physical science basis. Contribution of
Working Group I to the Fourth Assessment Report of the
Intergovernmental Panel on Climate Change. Cambridge, UK
and New York, NY: Cambridge University Press.
Mittelbach GG, Scheiner SM, Steiner CF, et al. 2001. What is the
observed relationship between species richness and productivity? Ecology 82: 2381–96.
Mulholland PJ, Helton AM, Poole GC, et al. 2008. Stream denitrification across biomes and its response to anthropogenic
nitrate loading. Nature 452: 202–05.
Newman EI. 1973. Competition and diversity in herbaceous vegetation. Nature 244: 310–11.
Pywell RF, Bullock JM, Hopkins A, et al. 2002. Restoration of
species-rich grassland on arable land: assessing the limiting
processes using a multi-site experiment. J Appl Ecol 39:
294–309.
Reader RJ, Wilson SD, Belcher JW, et al. 1994. Plant competition
in relation to neighbor biomass: an intercontinental study
with Poa pratensis. Ecology 75: 1753–60.
Reich PB. 1987. Quantifying plant response to ozone: a unifying
theory. Tree Physiol 3: 63–91.
Reich PB, Hungate BA, Luo Y. 2006. Carbon–nitrogen interactions in terrestrial ecosystems in response to rising atmospheric carbon dioxide. Annu Rev Ecol Evol S 37: 611–36.
Reichstein M, Ciais P, Papale D, et al. 2007. Reduction of ecosystem productivity and respiration during the European summer
2003 climate anomaly: a joint flux tower, remote sensing and
modelling analysis. Glob Change Biol 13: 634–51.
Royer DL, Kooyman RM, Little SA, et al. 2009. Ecology of leaf
teeth: a multi-site analysis from an Australian subtropical
rainforest. Am J Bot 96: 738–50.
Rustad LE. 2008. The response of terrestrial ecosystems to global
climate change: towards an integrated approach. Sci Total
Environ 404: 222–35.
Rustad LE, Campbell JL, Marion GM, et al. 2001. A meta-analysis
of the response of soil respiration, net nitrogen mineralization,
and aboveground plant growth to experimental ecosystem
warming. Oecologia 126: 543–62.
Sagarin R and Pauchard A. 2010. Observational approaches in
ecology open new ground in a changing world. Front Ecol
Environ 8: 379–86.
Silvertown J, Poulton P, Johnston AE, et al. 2006. The Park Grass
Experiment 1856–2006: its contribution to ecology. J Ecol 94:
801–14.

© The Ecological Society of America

LH Fraser et al.
Stokstad E. 2011. Open-source ecology takes root across the
world. Science 334: 308–09.
Sutton MA, Nemitz E, Erisman JW, et al. 2007. Challenges in
quantifying biosphere–atmosphere exchange of nitrogen
species. Environ Pollut 150: 125–39.
Tilman D. 1988. Plant strategies and the dynamics and structure of
plant communities. Princeton, NJ: Princeton University Press.
Webster JR, Mulholland PJ, Tank JL, et al. 2003. Factors affecting
ammonium uptake in streams – an inter-biome perspective.
Freshwater Biol 48: 1329–52.
Weltzin JF, Loik ME, Schwinning S, et al. 2003. Assessing the
response of terrestrial ecosystems to potential changes in precipitation. BioScience 53: 941–52.
Wessel WW, Tieteman A, Beier C, et al. 2004. A qualitative
ecosystem assessment for different shrublands in western

Coordinated distributed experiments
Europe under impact of climate change. Ecosystems 7: 662–71.
White SR, Carlyle CN, Fraser LH, et al. 2011. Climate change
experiments in temperate grasslands: synthesis and future
directions. Biol Lett; doi:10.1098/rsbl.2011.0956.
Whittaker R. 2010. Meta-analyses and mega-mistakes: calling
time on meta-analysis of the species richness–productivity
relationship. Ecology 91: 2522–33.
Wright IJ, Reich PB, Westoby M, et al. 2004. The worldwide leaf
economics spectrum. Nature 428: 821–27.
Wright RF and Rasmussen L. 1998. Introduction to the NITREX
and EXMAN projects. Forest Ecol Manag 101: 1–7.
Wu Z, Dijkstra P, Koch GW, et al. 2011. Responses of terrestrial
ecosystems to temperature and precipitation change: a metaanalysis of experimental manipulation. Glob Change Biol 17:
927–42.

Department of Biological Sciences, University of Alberta, Edmonton, Canada; 5Bayreuth Center for Ecology and Environmental Research,
University of Bayreuth, Bayreuth, Germany; 6Department of Biology, University of Pennsylvania, Philadelphia, PA; 7Section of Ecology,
Behavior and Evolution, University of California, San Diego, San Diego, CA; 8Department of Biology, University of New Mexico,
Albuquerque, NM; 9Department of Forestry and Natural Resources, Purdue University, West Lafayette, IN; 10Department of Biology and
Graduate Degree Program in Ecology, Colorado State University, Fort Collins, CO; 11Department of Ecology, Evolution and Behavior,
University of Minnesota, St Paul, MN; 12International Centre for Tibetan Plateau Ecosystem Management, Lanzhou University, Lanzhou,
China; 13Department of Botany and Microbiology, University of Oklahoma, Norman, OK; 14Department of Forest Resources, University of
Minnesota, St Paul, MN; 15Hawkesbury Institute for the Environment, University of Western Sydney, Richmond, Australia; 16Department of
Ecology and Evolutionary Biology, Yale University, New Haven, CT; 17Department of Molecular Biology and Ecology of Plants, Tel Aviv
University, Tel Aviv, Israel

© The Ecological Society of America

www.frontiersinecology.org