Atomic model refinement and completion at low resolution (cryo-EM or crystallographic) is often a challenging task. This is mostly because the experimental data aren’t sufficiently detailed to describe using atomic models. To make refinement practical and ensure a refined model is geometrically meaningful additional a priori information about model geometry needs to be used. This information includes restraints on Ramachandran plot distributions or side chain rotameric states. However, using Ramachandran plot or rotameric states as refinement targets diminish the validating power of these tools. Therefore finding additional model validation criteria that are not used or difficult to use as refinement goals is desirable. Hydrogen bonds are one of most important non-covalent interactions that shape and maintain protein structure. These interactions can be characterized by specific geometry of hydrogen donor and acceptor atoms. Systematic analysis of these geometries performed for all quality-filtered high-resolution models of proteins from PDB shows they have distinct and conserved distribution that can be characterized by only two parameters. Here we demonstrate how these two parameters can serve as unique validation metrics and how they can pinpoint severe modeling problems that no other validation tools can detect. This tool is now a part of Phenix model validation suite; guidelines to its use and interpretation will be given.

Hybrid and integrative modeling methods that combine computational molecular mechanics simulations with experimental data are powerful in describing the structure and dynamics of large biomolecules. In particular, flexible fitting is a powerful technique to build the 3D structures of biomolecules from cryo-electron microscopy (cryo-EM) density maps. While flexible fitting methods work nicely with very high-resolution maps, there are limitations for medium resolution maps (~5-10 angstrom) in the case of complex conformational transitions. To overcome such issues, we proposed a refinement based on conformational ensemble, i.e., performing multiple fittings trials using various parameters. An automatic adjustment of the biasing force constants during the fitting process was introduced via a replica-exchange scheme to improve the success rate. From such multiple fittings, clustering analysis of the models obtained can be an effective approach to avoid over‐fitting. In addition, we have looked into the pixel size parameter as it can impact the resolution and accuracy of a cryo-EM map, and we proposed a computational protocol to estimate the appropriate pixel size parameter. In our protocol, we fit and refine atomic structures against cryo-EM maps at multiple pixel sizes. The resulting fitted and refined structures are evaluated using the GOAP score. We have demonstrated the efficacy of this protocol in retrieving appropriate pixel sizes via several examples.

Structural biology, the study of the 3D structures of biological entities on scales from small molecules to cells, has had an enormous impact on our understanding of biology and biological processes in health and disease. The results of these structural studies (mainly by MX, NMR and 3DEM) have been captured in the single global archive of atomistic models of biomacromolecules and their complexes, the PDB, operated by the wwPDB consortium. In addition, since 2002 the cryo-EM community has been depositing their maps and tomograms in EMDB.

A few years before the resolution revolution, wwPDB and EMDB jointly convened an EM Validation Task Force (VTF) which met in 2010 to discuss initial recommendations (published in 2012) regarding validation of cryo-EM data and models. In the following decade, the resolution revolution happened, EMDB grew from 1,000 to 15,000 entries, an archive for raw cryo-EM data was established (EMPIAR), community challenges related to EM validation were organised, and many labs began to develop new approaches to validating EM structures. This made it clear that a second EM VTF meeting was urgently needed. This meeting took place (in person!) in January 2020. During two days several dozen experts from all over the globe discussed cryo-EM data
management, deposition and validation.

A white paper summarising the discussions and recommendations of the second EM VTF meeting is currently in preparation. I will provide an overview of the major consensus recommendations emanating from the meeting and also address how wwPDB and EMDB are implementing these.

Cryo-EM is becoming an increasingly popular method of structure determination in structural biology. As the number of cryo-EM structures increases, it is important to maintain standards that measure the quality of those structures. The correctness of atomic models is very important because they often serve as targets for novel drugs or the knowledge base of such developments. Also, such standards are important to prevent the accumulation of errors of the structures in the databases. Thus, careful curation and validation of cryo-EM maps and derived atomic models are of utmost importance.

We have developed the EMDA Python package [1] that includes tools for cryo-EM map and model manipulation. In this presentation, the emphasis is given to those for validation. The majority of the current validation tools used in single-particle cryo-EM analyses are global metrics. They provide summaries of the global quality of maps or map-model fits. In order to reveal the local variation of the signal in maps and map-model fits, a new set of tools based on the local correlation have been developed. To calculate the local correlation, a spherical kernel is convolved with the map in image space to yield a correlation value at each voxel resulting in a three-dimensional (3D) correlation map. The variation of calculated correlation depends on the size of the kernel. The local correlation calculated using half maps captures the local variations in the signal, whereas the local correlation calculated between a map and a model indicates the quality of their fit. Map-model local correlation can be used to identify model regions outside the density or poorly fitted. Also, it can highlight unmodeled regions on the map. While the half map local correlation is useful to identify the presence/absence of the signal, its comparison with the map-model local correlation can be used to validate the map-model fit. In this presentation, we will demonstrate the use of local correlation through several examples. Also, EMDA includes several tools based on the maximum likelihood method. EMDA’s map-overlay and map magnification refinement are based on the maximisation of the joint probability distribution between two maps by a quasi-Newton method. We will demonstrate the use of map overlay and magnification refinement implemented in EMDA through examples.

To obtain more accurate atomic models from cryoEM and increase their impact on biomedical research, metrics are needed that carefully evaluate these constructed models. In this poster we present further developments on FSC-Q, a map-to-model quality issue recently introduced [1], with the capability to detect those areas of the model that are better supported by the experimental data (Figure1). The algorithm performs a careful analysis of the Signal-to-Noise Ratio in the half maps and in map generated from the proposed model through local resolution. It is intuitive and, yet, very precise, introducing quality information that we have quantitatively shown is new, in the sense that some of it was not captured in previous quality assessment metrics.

Electron cryo-microscopy (cryo-EM) is rapidly becoming a mainstream area of structural biology and medicine, enabling visualization and modelling of a wide variety of biologically important complexes. This recent explosion of new cryo-EM structures raises several important questions. How accurate are these maps and their model interpretations? What criteria are currently being used and are they good enough? This paper describes the outcomes of the 2019 Model Metrics Challenge sponsored by EMDataResource (https://challenges.emdataresource.org). The goals of this challenge were two-fold: (1) to evaluate the quality of models that can be produced using current modelling software, and (2) to assess the performance of metrics currently in use to evaluate cryo-EM models. In both instances the focus was on map targets selected the near-atomic resolution regime (1.8-3.1 Å), with an innovative twist: three of four maps formed a resolution series from the same specimen/imaging experiment.The results permit several specific recommendations to be made about validating near-atomic cryo-EM structures, both in the context of an individual laboratory experiment and for in the context of a structure data archive. We will also touch on preliminary results from our ongoing 2021 Ligand Model Challenge.

Q-scores are calculated locally for individual atoms in a model fitted or built into a cryo-EM map. They can be averaged over groups of atoms to represent resolvability of larger features such as residues in proteins, nucleotides in nucleic acids, and ligands. Plotting of residue or nucleotide Q-scores helps to identify which parts of a model are resolved in the map, and which parts may be unresolved or may need further refinement. A useful property of Q-scores is that for well-fitted models, they correlate strongly to the resolution of the map estimated by FSC; this answers the question ‘what is a good score’ for a map at a certain resolution. Several examples and related structural insights are shown with models and maps ranging from 2 to 5Å resolution. The connection between Q-scores and atomic B-factors is also explored. Finally, Q-scores are used to help detect and assess water and ion molecules in maps at 3Å and higher resolutions.