Troubleshooting Simulations#

This guide covers tips and strategies for troubleshooting simulation failures. If you encounter a failure not covered here, please consider contributing it to the openfe-failure-cases repository or opening an issue.

Periodic box size / nonbonded cutoff errors#

You may see errors like:

openmm.OpenMMException: The periodic box size has decreased to less than twice the nonbonded cutoff.

openmm.OpenMMException: NonbondedForce: The cutoff distance cannot be greater than half the periodic box size.

These errors have the same cause: the system is too small for the chosen nonbonded cutoff.

Possible causes

  • Insufficient solvent padding.

  • Incorrect periodic box vectors.

  • A solvated system that does not leave enough space around the solute.

Suggested fixes

  • If the system is solvated by the OpenFE protocol, increase the protocol’s solvent_padding setting.

  • If the system is explicitly solvated, e.g. when using SolvatedPDBComponent or ProteinMembraneComponent:

    • Verify that the correct box vectors were supplied.

    • Ensure there is sufficient solvent around the system.

We do not recommend changing the nonbonded cutoff to fix this issue, since force fields are typically parameterized for these values and changing them may affect accuracy. If you have modified the nonbonded cutoff, it may be that it is too large for the simulation box.

SMIRNOFF force field parameter assignment errors#

You may see errors like:

openff.interchange.exceptions.UnassignedBondError: BondHandler was not able to find parameters for the following valence terms: - Topology indices (0, 1): names and elements (C1 C), (Si1 Si)

This usually means the SMIRNOFF-style force field does not have parameters for part of the chemistry in the system. In the example above, the force field was unable to assign bond parameters for bonds involving silicon.

Possible causes

  • The system contains atoms or functional groups that are not covered by the selected force field.

Suggested fixes

  • Check whether the chemistry in the system is expected to be supported by the chosen force field.

  • Consider using a molecule specific force field with custom parameters to guarantee coverage of the chemistry in your system.

If the error lists specific bonds or valence terms, those terms are the ones that could not be assigned parameters.

NaN errors during simulation#

You may see errors like:

openmm.OpenMMException: Particle coordinate is NaN.

openmmtools.multistate.utils.SimulationNaNError: Propagating replica 0 at state 10 resulted in a NaN!

For more information on the OpenMM error, see the OpenMM FAQ entry on NaNs: openmm/openmm

These errors usually mean that the simulation became numerically unstable during minimization or propagation. In some cases the simulation can recover by restarting from the last stable state. By default the protocols will attempt this up to 20 times before giving up, so you may see multiple NaN errors in the logs even if the simulation ultimately completes.

A successful restart does not necessarily mean that the resulting free-energy estimate is unreliable. However, repeated NaN errors may indicate an underlying stability problem. Before trusting the result, investigate the transformation and input structures for likely causes of instability and check the usual diagnostics, including agreement between independent repeats.

Possible causes

  • Missing capping groups or other issues in the receptor structure.

  • A poor atom mapping, including mappings that break bonds, map too few heavy atoms or map atoms whose hybridization changes due to a single to double/triple bond transformation.

  • An initial clash between the ligand and a crystal water, the receptor, or another part of the system that the minimizer could not relax.

  • A poor input structure for the ligand that the minimizer could not fix.

Suggested fixes

  • Run the validation script on the transformation JSON to try to identify the source of the problem.

  • Inspect the receptor for missing residues or missing capping groups.

  • Review the atom mapping for bond-breaking transformations or mappings with too few heavy atoms, try to improve the mapping by aligning the ligands before generating the mapping.

    • Consider running the transformation using the SepTop protocol which does not require an atom mapping and is robust to poor ligand alignment.

  • Check the input ligand and starting pose for clashes or poor geometry, try to relax the ligand in the receptor before running the protocol.

  • If possible, rebuild or re-prep the input structures before rerunning the protocol.

The state of the system and integrator before the error are often saved in a nan-error-logs directory, which can help with debugging.

Validating your inputs with a plain MD simulation#

Many of the errors above (e.g. parameter assignment failures, NaN errors during simulation, box size errors) are due to problems with the input structures which are not identified until protocol execution. A quick way to check this before running an expensive free energy protocol is to run a short MD simulation of your system with the PlainMDProtocol (see the Plain MD protocol guide).

This helps check that:

  • The receptor can be parameterized together with the ligand (e.g. no missing residues, caps or unsupported chemistry) by the same force fields you intend to use in your production protocol.

  • The non-alchemical system can be minimized and equilibrated producing a stable simulation without clashes or NaN errors.

If a system is stable in plain MD it is much more likely to be suitable for other protocols, for failures see sections NaN errors during simulation and SMIRNOFF force field parameter assignment errors for next steps. We recommend running this for a representative subset of the ligands with the receptor, using the same force fields and solvent settings you intend to use in your production protocol.

The simplest way to run this check is via the Python API with the example below which uses short equilibration and production lengths, which can catch common setup problems:

import openfe
from gufe.protocols import execute_DAG
from openff.units import unit
from openfe.protocols.openmm_md.plain_md_methods import PlainMDProtocol
import pathlib

ligand = openfe.SmallMoleculeComponent.from_sdf_file("ligand.sdf")  # load your ligand from your SDF file
protein = openfe.ProteinComponent.from_pdb_file("protein.pdb")  # load your protein from your PDB file
solvent = openfe.SolventComponent()  # use the same settings as your production run

# If you have cofactors in the system load them here as SmallMoleculeComponents and include them in the ChemicalSystem below.
# cofactor = openfe.SmallMoleculeComponent.from_sdf_file("cofactor.sdf")
system = openfe.ChemicalSystem(
    {
    "ligand": ligand,
    "protein": protein,
    "solvent": solvent,
    # "cofactor": cofactor  # include any cofactors here
    },
    name="validation",
)

settings = PlainMDProtocol.default_settings()
settings.simulation_settings.equilibration_length_nvt = 1 * unit.picosecond
settings.simulation_settings.equilibration_length = 1 * unit.picosecond
settings.simulation_settings.production_length = 1 * unit.picosecond
# You will need a GPU with CUDA installed to run this quick check, we recommend using the same platform as your production simulations.
settings.engine_settings.compute_platform = "CUDA"

protocol = PlainMDProtocol(settings=settings)
dag = protocol.create(stateA=system, stateB=system, mapping=None)
output_dir = pathlib.Path("plain_md_check")
output_dir.mkdir(exist_ok=True, parents=True)

result = execute_DAG(
    dag,
    shared_basedir=output_dir,
    scratch_basedir=output_dir,
    keep_shared=True,
    raise_error=True,
    n_retries=0
)

# Check for any errors in the execution
if not result.ok():
    for failure in result.protocol_unit_failures:
        # If we find a failure print the traceback to help with debugging
        print(f"Protocol unit {failure.name} failed with error: {failure.traceback}")

Resuming Interrupted Simulations#

In the case where your openfe quickrun execution has timed out, was cancelled, or was otherwise interrupted, you can attempt to resume the simulation using the --resume flag, which is available as of openfe v1.11.0.

See Resuming Interrupted Simulations for more details.

Log Debug information#

Note

When using a scheduler (e.g. SLURM), be sure to specify output files for standard out and standard error. For example, when using SLURM both --output= and --error= must be set to view errors.

One of the first troubleshooting steps is to increase the verbosity of the logging. openfe uses Python’s native logging library which can be configured either using a Python API or a configuration file.

Warning

We do not recommend setting the log level to debug for production runs, as the logging may slow down the simulation and add a lot of noise to the output.

When using openfe quickrun, the configuration file is more convenient. Below is an example logging configuration file that can be used to set the log level to DEBUG:

[loggers]
keys=root

[handlers]
keys=stdout

[formatters]
keys=standardFormatter,msgOnly

[handler_stdout]
class=StreamHandler
level=DEBUG
formatter=standardFormatter
args=(sys.stdout,)

[logger_root]
level=DEBUG
handlers=stdout

[formatter_standardFormatter]
format=%(asctime)s %(levelname)s %(name)s: %(message)s

[formatter_msgOnly]
format=%(message)s

Save this configuration file as debug_logging.conf and then run openfe quickrun with the --log flag, for example:

$ openfe --log debug_logging.conf quickrun -d results/ -o results/result_lig_ejm_31_solvent_lig_ejm_42_solvent.json transformations/rbfe_lig_ejm_31_solvent_lig_ejm_42_solvent.json

Note that the --log debug_logging.conf argument goes between openfe and quickrun on the command line.

This will cause every package to log at the debug level, which may be quite verbose and noisy but should aid in identifying what is going on right before the exception is thrown.

JAX warnings#

We use pymbar to analyze the free energy of the system. pymbar uses JAX to accelerate computation. The JAX library can utilize a GPU to further accelerate computation. If the necessary libraries for GPU acceleration are not installed and JAX detects a GPU, JAX will print a warning like this:

WARNING:2025-06-10 09:01:40,857:jax._src.xla_bridge:966: An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.

This warning does not mean that the molecular dynamics simulation will fall back to using the CPU. The simulation will still use the computing platform specified in the settings.

PYMBAR_DISABLE_JAX#

Due to a suspected memory leak in the JAX acceleration code in pymbar we disable JAX acceleration by default. This memory leak may result in the simulation crashing, wasting compute time. The error message may look like this:

LLVM compilation error: Cannot allocate memory
LLVM ERROR: Unable to allocate section memory!

We have decided to disable JAX acceleration by default to prevent wasted compute. However, if you wish to use the JAX acceleration, you may set PYMBAR_DISABLE_JAX to FALSE (e.g. put export PYMBAR_DISABLE_JAX=FALSE in your submission script before running openfe quickrun). For more information, see these issues on github: