simtk.org

Dear All,

Have now verified on a new, clean MacBook Pro (15" Retina display Quad core Intel I7) with NVIDIA GT750M GPU that OpenMM5.2 and OpenMM6.0 do not work with CUDA on platforms Mavericks (OSX 10.9) running the latest Xcode (5.02) and CUDA ToolKit/Drivers from cuda-mac-5.5.28_10.9_64.pkg. While the Cuda examples all run and find the GPU card, OpenMM5.2 will install, however, the python interface appears to be hosed (trying to run testInstallation.py gives an error:

20% python testInstallation.py
Failed to import OpenMM packages: dlopen(/Library/Python/2.7/site-packages/simtk/openmm/_openmm.so, 2): no suitable image found. Did find:
/Library/Python/2.7/site-packages/simtk/openmm/_openmm.so: mach-o, but wrong architecture
Make sure OpenMM is installed and the library path is set correctly.).

I expect this is a clang/gcc/g++ incompatibility. The test example HelloArgon.cpp, compiled using just make w/o arguments, doesn't compile and gives the error:

31% make
g++ -g -I/Users/brookscl/OpenMM/openmm_5.2/include HelloArgon.cpp -L/Users/brookscl/OpenMM/openmm_5.2/lib -lOpenMM -o HelloArgon
Undefined symbols for architecture x86_64:
"OpenMM::Context::setPositions(std::__1::vector<OpenMM::Vec3, std::__1::allocator<OpenMM::Vec3> > const&)", referenced from:
simulateArgon() in HelloArgon-4Cv30G.o
"OpenMM::Platform::loadPluginsFromDirectory(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)", referenced from:
simulateArgon() in HelloArgon-4Cv30G.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [HelloArgon] Error 1

The code HelloArgonInC.c compiles and runs, but runs on the OpenCL platform:

37% HelloArgonInC
REMARK Using OpenMM platform OpenCL

For OpenMM6.0 the situation is somewhat different, the existing pre-compiled libraries seem more compatible, but the CUDA platform is still not recognized.

Running the python test scrips yields:

46% python testInstallation.py
There are 3 Platforms available:

1 Reference - Successfully computed forces
2 CPU - Successfully computed forces
3 OpenCL - Successfully computed forces

Median difference in forces between platforms:

Reference vs. CPU: 3.41782e-06
Reference vs. OpenCL: 0.00462904
CPU vs. OpenCL: 0.0046304

The other examples all compile in c++ and c, but run on the OpenCL platform.

Thus, for some reason. the CUDA platform is not recognized. I note that we have also seen errors in the computation of PME energies on the OpenCL platform with OpenMM5.2, and thus I am hesitant to continue to do code development on OpenCL since I am unsure of the results. This will be the subject of another post once we have more data.

Any help/insights into how to get the CUDA platform to be recognized on the MacBook running Mavericks with either OpenMM5.2 or OpenMM6.0 would be appreciated.

Charles Brooks

Mavericks changed a bunch of things that broke OpenMM in lots of ways. So 5.2 won't work on it. All those problems are fixed in 6.0 though.

Apple's OpenCL is just a mess. It's been completely broken ever since 10.7.5. We're trying to get them to fix the problems, but I don't know how soon that will happen. In the mean time, I strongly recommend not using OpenCL on OS X.

CUDA should work fine though, and I don't know what's causing that problem. Unfortunately, I don't currently have a machine I can test that on. My computer has an older GPU (a 330M), and Nvidia dropped support for that generation in their drivers for Mavericks. But it still loads the CUDA platform, it just throws an exception when I try to create a Context saying that no device is available.

What happens if you try to load the plugin library directly?

> python
>>> from simtk.openmm import *
>>> Platform.loadPluginLibrary('/usr/local/openmm/lib/plugins/libOpenMMCUDA.dylib')

If it can't load the plugin, that should throw an exception with a hopefully useful error message.

Peter

Dear Peter,

Thanks for the suggestion. The problem appears to be:

python
Python 2.7.5 (default, Aug 25 2013, 00:04:04)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from simtk.openmm import *
>>> Platform.loadPluginLibrary('/Users/brookscl/OpenMM/openmm_6.0/lib/plugins/libOpenMMCUDA.dylib')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/simtk/openmm/openmm.py", line 11656, in loadPluginLibrary
return _openmm.Platform_loadPluginLibrary(*args)
Exception: Error loading library /Users/brookscl/OpenMM/openmm_6.0/lib/plugins/libOpenMMCUDA.dylib: dlopen(/Users/brookscl/OpenMM/openmm_6.0/lib/plugins/libOpenMMCUDA.dylib, 9): Library not loaded: @rpath/libcufft.dylib
Referenced from: /Users/brookscl/OpenMM/openmm_6.0/lib/plugins/libOpenMMCUDA.dylib
Reason: Incompatible library version: libOpenMMCUDA.dylib requires version 1.1.0 or later, but libcufft.dylib provides version 0.0.0

I note that this appears to be an incompatibility with existing libOpenMMCUDA.dylib and CUDA 5.5 on the Mac:

See that indeed I am linking to the 5.5 libraries

ls -al /usr/local/cuda/lib/libcufft.dylib
lrwxr-xr-x 1 root wheel 45 Feb 12 17:36 /usr/local/cuda/lib/libcufft.dylib -> /Developer/NVIDIA/CUDA-5.5/lib/libcufft.dylib

However, it appears the libcufft.dylib in the CUDA 5.5 release for the Mac are not compatible with the same version.

chem-clb3.chem.lsa.umich.edu 57% otool -L /usr/local/cuda/lib/libcufft.dylib
/usr/local/cuda/lib/libcufft.dylib:
@rpath/libcufft.5.5.dylib (compatibility version 0.0.0, current version 5.5.28)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 635.21.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)

Ideas?

Post my posting I investigated a bit more and find that if I run the otool -L on the libcufft.dylib from the CUDA 5.0 ToolKit directory, it shows compatibility version 1.0.0

pwd
/Developer/NVIDIA/CUDA-5.0/lib
chem-clb3.chem.lsa.umich.edu 63% cd ../../CUDA-5.5/lib
chem-clb3.chem.lsa.umich.edu 64% otool -L libcufft.dylib
libcufft.dylib:
@rpath/libcufft.5.5.dylib (compatibility version 0.0.0, current version 5.5.28)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 635.21.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 52.0.0)

I see the problem. On my computer:
Apparently when CUDA updates itself, it doesn't actually update all the libraries! I've done a full re-install of CUDA 5.5 and rebuilt the Mac libraries for OpenMM 6.0. Try the new ones. Does it work now?

Peter

Have you had a chance to test the new build? As soon as you confirm the problem is fixed, we can release 6.0.

Peter

Has anybody successfully installed and run OpenMM on a Mac using OS X Yosemite?

Hi,

I'm seeing this exact same problem on OSx Yosemite now. When I run the test, python -m simtk.testInstallation, I get the same output:

There are 3 Platforms available:

1 Reference - Successfully computed forces
2 CPU - Successfully computed forces
3 OpenCL - Successfully computed forces

Median difference in forces between platforms:

Reference vs. CPU: 1.99866e-06
Reference vs. OpenCL: 2.1526e-05
CPU vs. OpenCL: 2.13618e-05

as previously reported by Prof. Brooks for Mavericks. I've installed cuda as well as the toolkit, and my $DYLD_LIBRARY_PATH points to the correct /usr/local/cuda/lib, where all the cuda libraries are located. Another graduate student in our lab, who is running OpenMM on an identical machine, but on Mavericks instead of Yosemite isn't facing this problem. His version does find the cuda libraries, and runs against them. Has the previous Mavericks problem resurfaced in Yosemite? Is there any way to fix it?

Thank you.

Hi Saurabh,

Try this command:
(If you installed somewhere else, give the correct path, of course.) Post the full output. That will hopefully tell us what the problem is.

Make sure you have the right version of CUDA installed. The OpenMM 6.3 binaries were built against CUDA 7.0. Different CUDA versions are not binary compatible with each other, so you need the exact version it was built against.

Peter

Hi Dr. Eastman,

OpenMM can find CUDA now. I had previously compiled it against CUDA 6.5, which was why it wasn't finding the cuda driver. Installing CUDA 7.0 and reinstalling OpenMM has now solved the problem.

Thank you for your help.

Ok, great.

Peter