2019 EMDataResource Model Metrics Challenge Dataset

  • Catherine L. Lawson (Creator)
  • Andriy Kryshtafovych (Creator)
  • Paul D. Adams (Creator)
  • Pavel V. Afonine (Creator)
  • Matthew L. Baker (Creator)
  • Benjamin A. Barad (Creator)
  • Paul Bond (Creator)
  • Tom Burnley (Creator)
  • Renzhi Cao (Creator)
  • Jianlin Cheng (Creator)
  • Grzegorz Chojnowski (Creator)
  • Kevin Douglas Cowtan (Creator)
  • Ken Dill (Creator)
  • Frank DiMaio (Creator)
  • Daniel P. Farrell (Creator)
  • James S Fraser (Creator)
  • Mark A Jr Herzik (Creator)
  • Soon Wen Hoh (Creator)
  • Jie Hou (Creator)
  • Li Wei Hung (Creator)
  • Maxim Igaev (Creator)
  • Agnel P. Joseph (Creator)
  • Daisuke Kihara (Creator)
  • Dilip Kumar (Creator)
  • Sumit Mittal (Creator)
  • Bohdan Monastryrskyy (Creator)
  • Mateusz Olek (Creator)
  • Colin M Palmer (Creator)
  • Ardan Patwardhan (Creator)
  • Alberto Pérez (Creator)
  • Jonas Pfab (Creator)
  • Greg D Pintilie (Creator)
  • Jane S. Richardson (Creator)
  • Daipayan Sarkar (Creator)
  • Luisa U. Schäfer (Creator)
  • Michael F Schmid (Creator)
  • Gunnar F. Schroeder (Creator)
  • Mrinal Shekhar (Creator)
  • Dong Si (Creator)
  • Abishek Singharoy (Creator)
  • Genki Terashi (Creator)
  • Thomas C. Terwilliger (Creator)
  • Andrea Vaiana (Creator)
  • Liguo Wang (Creator)
  • Stephanie A. Wankowicz (Creator)
  • Christopher J. Williams (Creator)
  • Martyn Winn (Creator)
  • Tianqi Wu (Creator)
  • Xiaodi Yu (Creator)
  • Kaiming Zhang (Creator)
  • Helen M. Berman (Creator)
  • Wah Chiu (Creator)



This is the full dataset of the 2019 Cryo-EM Map-based Model Metrics Challenge sponsored by EMDataResource (www.emdataresource.org, challenges.emdataresource.org, model-compare.emdataresource.org). The goals of this challenge were (1) to assess the quality of models that can be produced using current modeling software, (2) to check the reproducibility of modeling results from different software developers and users, and (3) compare the performance of current metrics used for evaluation of models. The focus was on near-atomic resolution maps with an innovative twist: three of four target maps formed a resolution series (1.8 to 3.1 Å) from the same specimen and imaging experiment. Tools developed in previous challenges were expanded for managing, visualizing and analyzing the 63 submitted coordinate models, and several new metrics were introduced. File Descriptions: 2019-EMDataResource-Challenge-web.pdf: Archive of News, Goals, Timeline, Targets, Modelling Instructions, Process, FAQ, Submission Instructions, Submission Summary Statistics source from the EMDR Challenges website correlation-images.tar.gz: Pairwise correlation tables for selected metric scores from the EMDR Model Compare website maps.tar.gz: The maps used for Fit-to-Map analyses in the Challenge models.tar.gz: The 63 models submitted by the modelling teams results.tar.gz: The output logs for all of the analysis methods Scores.xlsx: Scores for each model and analysis method, compiled into spreadsheet format targets.tar.gz: The reference models used in the analysis Post submission correction to the web archive PDF document: The full list of EMDataResource members on the model committee is as follows: Cathy Lawson, Andriy Kryshtafovych, Greg Pintilie, Mike Schmid, Helen Berman, Wah Chiu.

External deposit with Zenodo
Date made available28 Oct 2020
  • Cryo-EM model validation recommendations based on outcomes of the 2019 EMDataResource challenge

    Lawson, C. L., Kryshtafovych, A., Adams, P. D., Afonine, P. V., Baker, M. L., Barad, B. A., Bond, P., Burnley, T., Cao, R., Cheng, J., Chojnowski, G., Cowtan, K., Dill, K. A., DiMaio, F., Farrell, D. P., Fraser, J. S., Herzik, M. A., Hoh, S. W., Hou, J., Hung, L. W., & 34 othersIgaev, M., Joseph, A. P., Kihara, D., Kumar, D., Mittal, S., Monastyrskyy, B., Olek, M., Palmer, C. M., Patwardhan, A., Perez, A., Pfab, J., Pintilie, G. D., Richardson, J. S., Rosenthal, P. B., Sarkar, D., Schäfer, L. U., Schmid, M. F., Schröder, G. F., Shekhar, M., Si, D., Singharoy, A., Terashi, G., Terwilliger, T. C., Vaiana, A., Wang, L., Wang, Z., Wankowicz, S. A., Williams, C. J., Winn, M., Wu, T., Yu, X., Zhang, K., Berman, H. M. & Chiu, W., 4 Feb 2021, In: Nature Methods. 18, 2, p. 156-164 9 p.

    Research output: Contribution to journalArticlepeer-review

    Open Access

Cite this