使用GROMACS做模拟，在nvt中遇到CUDA error #400的问题

仙台叶月

在运行： gmx mdrun -v -deffnm nvt -nb gpu -pme gpu -ntmpi 1 -ntomp 8 -gpu_id 0后 。
遇到了错误：
Program:     gmx mdrun, version 2023.2Source file: src/gromacs/gpu_utils/device_stream.cu (line 100)
Function:    DeviceStream::synchronize() const::<lambda()>

Assertion failed:
Condition: stat == cudaSuccess
cudaStreamSynchronize failed. CUDA error #400
(cudaErrorInvalidResourceHandle): invalid resource handle.

For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors
我查了一下可能是有PME的计算量太大问题、GPU占用问题（我在服务器上跑的，id0的GPU下有一个lammps的任务）、em能量爆炸问题。我后面用了另一个体系，用相同的命令和gpuid下完全可以跑，大概排除gpu占用的问题和em的问题。然后又把PME换成了Cuf - off但是也不行。 auto.sh是自动添加气体并且模拟的脚本，和另一个体系的区别就是将N2替换成了CH4，并且将原子数量从3（N2中有一个虚拟原子）改成了5。

具体报错内容如下：

:-) GROMACS - gmx grompp, 2023.2 (-:


Executable:   /usr/local/gromacs/bin/gmx
Data prefix:  /usr/local/gromacs
Working dir:  /home/amax/DATA/zzf/temp
Command line:
  gmx grompp -f em.mdp -c system.gro -p topol.top -o em.tpr


Setting the LD random seed to -343146529


Generated 108249 of the 108345 non-bonded parameter combinations
Generating 1-4 interactions: fudge = 1


Generated 74154 of the 108345 1-4 parameter combinations


Excluding 3 bonded neighbours molecule type 'AB_RESP'


Excluding 2 bonded neighbours molecule type 'CO2'


Excluding 3 bonded neighbours molecule type 'CH4'


Searching the wall atom type(s)


NOTE 1 [file topol.top, line 22]:
  In moleculetype 'AB_RESP' 4 atoms are not bound by a potential or
  constraint to any other atom in the same moleculetype. Although
  technically this might not cause issues in a simulation, this often means
  that the user forgot to add a bond/potential/constraint or put multiple
  molecules in the same moleculetype definition by mistake. Run with -v to
  get information for each atom.


Analysing residue names:
There are:    61      Other residues
Analysing residues not classified as Protein/DNA/RNA/Water and splitting into groups...


NOTE 2 [file em.mdp]:
  There are 576 atoms that are fully frozen and part of COMM removal
  group(s), removing these atoms from the COMM removal group(s)


Number of degrees of freedom in T-Coupling group rest is 173.54
The integrator does not provide a ensemble temperature, there is no system ensemble temperature


The largest distance between excluded atoms is 0.390 nm between atom 472 and 476
Calculating fourier grid dimensions for X Y Z
Using a fourier grid of 36x28x384, spacing 0.107 0.119 0.117


Estimate for the relative computational load of the PME mesh part: 0.95


NOTE 3 [file em.mdp]:
  The optimal PME mesh load for parallel simulations is below 0.5
  and for highly parallel simulations between 0.25 and 0.33,
  for higher performance, increase the cut-off and the PME grid spacing.






This run will generate roughly 0 Mb of data


There were 3 NOTEs


GROMACS reminds you: "What If None Of Your Dreams Come True ?" (E. Costello)


                      :-) GROMACS - gmx mdrun, 2023.2 (-:


Executable:   /usr/local/gromacs/bin/gmx
Data prefix:  /usr/local/gromacs
Working dir:  /home/amax/DATA/zzf/temp
Command line:
  gmx mdrun -v -deffnm em -nb gpu -ntmpi 1 -ntomp 8 -gpu_id 0


Reading file em.tpr, VERSION 2023.2 (single precision)
1 GPU selected for this run.
Mapping of GPU IDs to the 1 GPU task in the 1 rank on this node:
  PP:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PP task will update and constrain coordinates on the CPU
Using 1 MPI thread
Using 8 OpenMP threads




NOTE: The number of threads is not equal to the number of (logical) cpus
      and the -pin option is set to auto: will not pin threads to cpus.
      This can lead to significant performance degradation.
      Consider using -pin on (and -pinoffset in case you run multiple jobs).


Steepest Descents:
   Tolerance (Fmax)   =  1.00000e+03
   Number of steps    =        50000
Step=    0, Dmax= 1.0e-02 nm, Epot=  5.86995e+34 Fmax= 3.21922e+02, atom= 605


writing lowest energy coordinates.


Steepest Descents converged to Fmax < 1000 in 1 steps
Potential Energy  =  5.8699507e+34
Maximum force     =  3.2192212e+02 on atom 605
Norm of force     =            nan


GROMACS reminds you: "C has the power of assembly language and the convenience of... assembly language." (Dennis Ritchie)


                      :-) GROMACS - gmx grompp, 2023.2 (-:


Executable:   /usr/local/gromacs/bin/gmx
Data prefix:  /usr/local/gromacs
Working dir:  /home/amax/DATA/zzf/temp
Command line:
  gmx grompp -f nvt.mdp -c em.gro -p topol.top -o nvt.tpr


Setting the LD random seed to -1711603761


Generated 108249 of the 108345 non-bonded parameter combinations
Generating 1-4 interactions: fudge = 1


Generated 74154 of the 108345 1-4 parameter combinations


Excluding 3 bonded neighbours molecule type 'AB_RESP'


turning H bonds into constraints...


Excluding 2 bonded neighbours molecule type 'CO2'


turning H bonds into constraints...


Excluding 3 bonded neighbours molecule type 'CH4'


turning H bonds into constraints...


Searching the wall atom type(s)


NOTE 1 [file topol.top, line 22]:
  In moleculetype 'AB_RESP' 4 atoms are not bound by a potential or
  constraint to any other atom in the same moleculetype. Although
  technically this might not cause issues in a simulation, this often means
  that the user forgot to add a bond/potential/constraint or put multiple
  molecules in the same moleculetype definition by mistake. Run with -v to
  get information for each atom.




NOTE 2 [file topol.top, line 22]:
  The bond in molecule-type AB_RESP between atoms 20 C10 and 26 C13 has an
  estimated oscillational period of 1.8e-02 ps, which is less than 10 times
  the time step of 2.0e-03 ps.
  Maybe you forgot to change the constraints mdp option.


Analysing residue names:
There are:    61      Other residues
Analysing residues not classified as Protein/DNA/RNA/Water and splitting into groups...


NOTE 3 [file nvt.mdp]:
  There are 576 atoms that are fully frozen and part of COMM removal
  group(s), removing these atoms from the COMM removal group(s)


Number of degrees of freedom in T-Coupling group System is 123.78


The largest distance between excluded atoms is 0.390 nm between atom 472 and 476


Determining Verlet buffer for a tolerance of 0.005 kJ/mol/ps at 300 K


Calculated rlist for 1x1 atom pair-list as 1.200 nm, buffer size 0.000 nm


Set rlist, assuming 4x4 atom pair-list, to 1.200 nm, buffer size 0.000 nm


Note that mdrun will redetermine rlist based on the actual pair-list setup
Calculating fourier grid dimensions for X Y Z
Using a fourier grid of 36x28x384, spacing 0.107 0.119 0.117


Estimate for the relative computational load of the PME mesh part: 0.95


NOTE 4 [file nvt.mdp]:
  The optimal PME mesh load for parallel simulations is below 0.5
  and for highly parallel simulations between 0.25 and 0.33,
  for higher performance, increase the cut-off and the PME grid spacing.






This run will generate roughly 0 Mb of data


There were 4 NOTEs


GROMACS reminds you: "Meet Me At the Coffee Shop" (Red Hot Chili Peppers)


                      :-) GROMACS - gmx mdrun, 2023.2 (-:


Executable:   /usr/local/gromacs/bin/gmx
Data prefix:  /usr/local/gromacs
Working dir:  /home/amax/DATA/zzf/temp
Command line:
  gmx mdrun -v -deffnm nvt -nb gpu -pme gpu -ntmpi 1 -ntomp 8 -gpu_id 0


Reading file nvt.tpr, VERSION 2023.2 (single precision)
Changing nstlist from 10 to 100, rlist from 1.2 to 1.2


Update groups can not be used for this system because atoms that are (in)directly constrained together are interdispersed with other atoms


1 GPU selected for this run.
Mapping of GPU IDs to the 2 GPU tasks in the 1 rank on this node:
  PP:0,PME:0
PP tasks will do (non-perturbed) short-ranged interactions on the GPU
PP task will update and constrain coordinates on the CPU
PME tasks will do all aspects on the GPU
Using 1 MPI thread
Using 8 OpenMP threads




NOTE: The number of threads is not equal to the number of (logical) cpus
      and the -pin option is set to auto: will not pin threads to cpus.
      This can lead to significant performance degradation.
      Consider using -pin on (and -pinoffset in case you run multiple jobs).
starting mdrun 'AB_RESP'
10000 steps,     20.0 ps.


-------------------------------------------------------
Program:     gmx mdrun, version 2023.2
Source file: src/gromacs/gpu_utils/device_stream.cu (line 100)
Function:    DeviceStream::synchronize() const::<lambda()>


Assertion failed:
Condition: stat == cudaSuccess
cudaStreamSynchronize failed. CUDA error #400
(cudaErrorInvalidResourceHandle): invalid resource handle.


For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors
-------------------------------------------------------

仙台叶月

sobereva 发表于 2026-2-2 06:20
有别人回复之前若需要对帖子进行修改、补充，应直接编辑原帖，不要通过回帖进行补充，导致信息零零碎碎， ...

感谢老师！已经解决了！

仙台叶月

Loading0760 发表于 2026-2-1 23:23
建议先只用cpu跑一遍，确认是体系的问题还是环境的问题

感谢帮助 已经找到问题了

sobereva

仙台叶月 发表于 2026-1-30 14:17
用到了这些文件

有别人回复之前若需要对帖子进行修改、补充，应直接编辑原帖，不要通过回帖进行补充，导致信息零零碎碎，这点在置顶的新社员必读贴里明确说了。

可以先尝试3L说的。
但你这个问题最大概率是拓扑文件有硬伤，参考http://sobereva.com/soft/Sobtop#FAQ8说的排查和解决。尤其是注意grompp显示的NOTE 1

Loading0760

建议先只用cpu跑一遍，确认是体系的问题还是环境的问题