ROBITT: A tool for assessing the risk-of-bias in studies of temporal trends in ecology

Robin J. Boyd*, Gary D. Powney, Fiona Burns, Alain Danet, François Duchenne, Matthew J. Grainger, Susan G. Jarvis, Gabrielle Martin, Erlend B. Nilsen, Emmanuelle Porcher, Gavin B. Stewart, Oliver J. Wilson, Oliver L. Pescott

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Aggregated species occurrence and abundance data from disparate sources are increasingly accessible to ecologists for the analysis of temporal trends in biodiversity. However, sampling biases relevant to any given research question are often poorly explored and infrequently reported; this can undermine statistical inference. In other disciplines, it is common for researchers to complete ‘risk-of-bias’ assessments to expose and document the potential for biases to undermine conclusions. The huge growth in available data, and recent controversies surrounding their use to infer temporal trends, indicate that similar assessments are urgently needed in ecology. We introduce ROBITT, a structured tool for assessing the ‘Risk-Of-Bias In studies of Temporal Trends in ecology’. ROBITT has a similar format to its counterparts in other disciplines: it comprises signalling questions designed to elicit information on the potential for bias in key study domains. In answering these, users will define study inferential goal(s) and relevant statistical target populations. This information is used to assess potential sampling biases across domains relevant to the research question (e.g. geography, taxonomy, environment), and how these vary through time. If assessments indicate biases, then users must clearly describe them and/or explain what mitigating action will be taken. Everything that users need to complete a ROBITT assessment is provided: the tool, a guidance document and a worked example. Following other disciplines, the tool and guidance document were developed through a consensus-forming process across experts working in relevant areas of ecology and evidence synthesis. We propose that researchers should be strongly encouraged to include a ROBITT assessment when publishing studies of biodiversity trends, especially when using aggregated data. This will help researchers to structure their thinking, clearly acknowledge potential sampling issues, highlight where expert consultation is required and provide an opportunity to describe data checks that might go unreported. ROBITT will also enable reviewers, editors and readers to establish how well research conclusions are supported given a dataset combined with some analytical approach. In turn, it should strengthen evidence-based policy and practice, reduce differing interpretations of data and provide a clearer picture of the uncertainties associated with our understanding of reality.

Original languageEnglish
Pages (from-to)1497-1507
Number of pages11
JournalMethods in ecology and evolution
Issue number7
Early online date6 Apr 2022
Publication statusPublished - 1 Jul 2022

Bibliographical note

Funding Information:
G.D.P. and S.G.J. were supported through Natural Environment Research Council (NERC) award number NE/V006878/1 as part of the DRUID (Drivers and Repercussions of UK Insect Declines) project. R.J.B. and O.L.P. were supported by the NERC award number NE/R016429/1 as part of the UK Status, Change and Projections of the Environment (UK-SCAPE) program delivering National Capability; O.L.P. also acknowledges the support of the British Council's Alliance Hubert Curien program, award number 515719745, and the National Plant Monitoring Scheme funded by the UK Joint Nature Conservation Committee (ref. A17-0291-1205). F.D. was funded by the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement number 787638, granted to C.H. Graham). E.P. and G.M. acknowledge the support of the PHC Alliance award number 44779VJ.

Publisher Copyright:
© 2022 The Authors. Methods in Ecology and Evolution published by John Wiley & Sons Ltd on behalf of British Ecological Society.


  • essential biodiversity variables
  • indicators
  • insect declines
  • risk-of-bias
  • species occurrence data
  • temporal trends
  • uncertainty

Cite this