Simulations with >100k particles.

[Spela Ivekovic]

Hi Peter,

I have run the Python script successfully and it reports 792450 atoms in its original setting (boxSize = 20). I tested both the CUDA platform and the OpenCL platform.

Here's the complete output:

Code: Select all

Adding solvent
Building system
792450 atoms
Creating context
Computing
initial energy: -9582238.80569 kJ/mol
final energy: -12133964.7558 kJ/mol

792,450 is quite a bit more than I can get if I run the HelloWaterBox example directly as a C++ executable. In the HelloWaterBox C++ case, the simulation crashes if increased over 45x45x45 water molecules (= 91,125 water molecules = 273375 atoms).

Why would there be such a discrepancy?

Spela

[Peter Eastman]

Hi Spela,

It looks like HelloWaterBox needs to be cleaned up. The real problem is that it's creating water molecules that are very tightly packed together in a very high energy configuration, and the simulation is blowing up. Line 244 reads

Code: Select all

const double WaterSizeInNm = 0.3107; // edge of cube containing one water, in nanometers

If I increase the spacing very slightly to 0.3207, it has no problem simulating over a million atoms.

Peter

[Spela Ivekovic]

Hi Peter,

The real problem is that it's creating water molecules that are very tightly packed together in a very high energy configuration, and the simulation is blowing up.

yes, this sorted it out. Phew

I guess the issue is numerical round-off. When I find the time, I'll see if I get the same problem with the double-precision version of OpenMM. I expect the problems we have been experiencing in our other simulations then also have to do with the spacing of the atoms. We are using OpenMM to speed up a CPU-based MD library which runs in double precision and I suspect this is where most of the issues arise from, as we pass the initial simulation settings, generated in this library, to OpenMM. It's good to know where to look...

BTW, just for the record, with the correction you suggested, I managed to run a 70x70x70 water box in the HelloWaterBox example on a 2GB Quadro 4000. Not bad!

Thank you for your help!

Spela

[Peter Eastman]

Hi Spela,

I don't think numerical roundoff is the issue. The spacing used in the original version is actually an accurate value for liquid water, and would be entirely reasonable if it were starting from an equilibrated configuration. The problem is that it just arranges the water molecules in a uniform grid, which is nothing like liquid water and is an incredibly high energy configuration. If you could run it long enough, it would eventually equilibrate to something reasonable, but there's a really good chance that before that happens, some atom will get shoved right on top of another atom and the whole simulation will blow up. And the bigger your box is, the more likely that is to happen (since it just has to happen once anywhere in the box).

Presumably if you decreased the time step and increased the friction coefficient of the thermostat, that would also solve the problem. I haven't tested it though.

Peter

[Maxim Imakaev]

Hi Peter,

I'm just asking if you fixed the "long chain" type error for the large number of particles (probably, stack overflow for the connected component search)?
We want to run some very long polymer simulations around the end of the summer.

Thanks,
Max

[Peter Eastman]

No I haven't. Thanks for reminding me. I'll see if I can do that in the next few days.

Peter

[Peter Eastman]

I've just merged the fix into the main repository.

Peter

[Maxim Imakaev]

Thank you Peter!
It works now with a million particles, which is more than enough for us.

Out of curiosity I checked it further, and found that CUDA breaks down between 1.25 and 1.5 million due to a segfault in nonbondedForce.
OpenCL seems to be working up to 3M, but it explodes after 2M - maybe, because of my setup, I'll have to experiment with it if I need to.

Thanks,
Max