openmm2.0 and cuda 3.1 64 bits in Mac

[Alan Wilter Sousa da Silva]

Hi there,

Compiling openmm2.0 from source worked fine in 32 bits. Now that Cuda 3.1 is full 64 bits in Mac I gave a try in 64 bits, by putting in ccmake "CMAKE_OSX_ARCHITECTURES x86_64"

...
[ 96%] Building CXX object plugins/freeEnergy/platforms/cuda/sharedTarget/CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateObcGbsaSoftcoreForces2.cu_OpenMMFreeEnergyCuda_generated.cpp.o
cd /Users/alan/Downloads/build_openmm/plugins/freeEnergy/platforms/cuda/sharedTarget && /usr/bin/c++ -DOpenMMFreeEnergyCuda_EXPORTS -DOPENMM_LIBRARY_NAME=OpenMM -DOPENMM_MAJOR_VERSION=1 -DOPENMM_MINOR_VERSION=0 -DOPENMM_BUILD_VERSION=0 -DOPENMM_SVN_REVISION=\"exported\" -DOPENMM_COPYRIGHT_YEARS=\"2008\" -DOPENMM_AUTHORS=\"Peter.Eastman\" -DOPENMM_FREE_ENERGY_LIBRARY_NAME=OpenMMFreeEnergy -DOPENMM_FREE_ENERGY_MAJOR_VERSION=1 -DOPENMM_FREE_ENERGY_MINOR_VERSION=0 -DOPENMM_FREE_ENERGY_BUILD_VERSION=0 -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -O3 -DNDEBUG -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -fPIC -I/Users/alan/Downloads/OpenMM2.0-Source/src/plugins/freeEnergy/src -I/Users/alan/Downloads/OpenMM2.0-Source/src/platforms/reference/src/SimTKReference -I/Users/alan/Downloads/OpenMM2.0-Source/src/platforms/reference/src -I/Users/alan/Downloads/OpenMM2.0-Source/src/plugins/freeEnergy/platforms/reference/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/plugins/freeEnergy/openmmapi/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/plugins/freeEnergy/./include -I/Users/alan/Downloads/OpenMM2.0-Source/src/src -I/Users/alan/Downloads/OpenMM2.0-Source/src/libraries/validate/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/platforms/reference/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/libraries/sfmt/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/libraries/lepton/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/libraries/quern/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/libraries/jama/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/olla/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/openmmapi/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/./include -I/Users/alan/Downloads/OpenMM2.0-Source/src/plugins/freeEnergy/platforms/cuda/./include -I/Users/alan/Downloads/OpenMM2.0-Source/src/platforms/cuda/include -I/Users/alan/Downloads/OpenMM2.0-Source/src/platforms/cuda/src -I/Users/alan/Downloads/OpenMM2.0-Source/src/platforms/cuda/src/kernels -I/Users/alan/Downloads/OpenMM2.0-Source/src/plugins/freeEnergy/platforms/cuda/src -I/usr/local/cuda/include -DOPENMMCUDAFREEENERGY_BUILDING_SHARED_LIBRARY -o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateObcGbsaSoftcoreForces2.cu_OpenMMFreeEnergyCuda_generated.cpp.o -c /Users/alan/Downloads/build_openmm/src/cuda/kCalculateObcGbsaSoftcoreForces2.cu_OpenMMFreeEnergyCuda_generated.cpp
Linking CXX shared library ../../../../../libOpenMMFreeEnergyCuda.dylib
cd /Users/alan/Downloads/build_openmm/plugins/freeEnergy/platforms/cuda/sharedTarget && /sw/bin/cmake -E cmake_link_script CMakeFiles/OpenMMFreeEnergyCuda.dir/link.txt --verbose=1
/usr/bin/c++ -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5 -O3 -DNDEBUG -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -dynamiclib -headerpad_max_install_names -o ../../../../../libOpenMMFreeEnergyCuda.dylib -install_name /Users/alan/Downloads/build_openmm/libOpenMMFreeEnergyCuda.dylib CMakeFiles/OpenMMFreeEnergyCuda.dir/__/src/CudaFreeEnergyKernelFactory.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/src/CudaFreeEnergyKernels.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/src/kernels/GpuGBVISoftcore.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/src/kernels/GpuLJ14Softcore.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/src/kernels/GpuNonbondedSoftcore.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/src/kernels/GpuObcGbsaSoftcore.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateCDLJObcGbsaSoftcoreForces1.cu_OpenMMFreeEnergyCuda_generated.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateGBVISoftcoreBornSum.cu_OpenMMFreeEnergyCuda_generated.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateGBVISoftcoreForces2.cu_OpenMMFreeEnergyCuda_generated.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateLocalSoftcoreForces.cu_OpenMMFreeEnergyCuda_generated.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateNonbondedSoftcore.cu_OpenMMFreeEnergyCuda_generated.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateObcGbsaSoftcoreBornSum.cu_OpenMMFreeEnergyCuda_generated.cpp.o CMakeFiles/OpenMMFreeEnergyCuda.dir/__/__/__/__/__/src/cuda/kCalculateObcGbsaSoftcoreForces2.cu_OpenMMFreeEnergyCuda_generated.cpp.o -L/usr/local/cuda/lib/libcuda.dylib -L/usr/local/cuda/lib/libcudart.dylib /usr/local/cuda/lib/libcuda.dylib /usr/local/cuda/lib/libcudart.dylib ../../../../../libOpenMM.dylib ../../../../../libOpenMMCuda.dylib ../../../../../libOpenMMFreeEnergy.dylib /usr/local/cuda/lib/libcuda.dylib /usr/local/cuda/lib/libcudart.dylib /usr/local/cuda/lib/libcufft.dylib ../../../../../libOpenMM.dylib /usr/lib/libdl.dylib
ld: warning: path '/usr/local/cuda/lib/libcuda.dylib' following -L not a directory
ld: warning: path '/usr/local/cuda/lib/libcudart.dylib' following -L not a directory
ld: warning: duplicate dylib /usr/local/cuda/lib/libcuda.dylib
ld: warning: path '/usr/local/cuda/lib/libcuda.dylib' following -L not a directory
ld: warning: path '/usr/local/cuda/lib/libcudart.dylib' following -L not a directory
ld: warning: duplicate dylib /usr/local/cuda/lib/libcuda.dylib
Undefined symbols:
"_fwrite$UNIX2003", referenced from:
kPrintGBVISoftcore(_gpuContext*, GpuGBVISoftcore*, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, int)in kCalculateGBVISoftcoreBornSum.cu_OpenMMFreeEnergyCuda_generated.cpp.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
lipo: can't open input file: /var/folders/+m/+meU9wCwHYKwh16Rr5-GsE+++TM/-Tmp-//cc8jPJB3.out (No such file or directory)
make[2]: *** [libOpenMMFreeEnergyCuda.dylib] Error 1
make[1]: *** [plugins/freeEnergy/platforms/cuda/sharedTarget/CMakeFiles/OpenMMFreeEnergyCuda.dir/all] Error 2
make: *** [all] Error 2


The files generated up here are universal i386 and x86_64, e.g.,

TestCudaAndersenThermostat: Mach-O universal binary with 2 architectures
TestCudaAndersenThermostat (for architecture i386): Mach-O executable i386
TestCudaAndersenThermostat (for architecture x86_64): Mach-O 64-bit executable x86_64


Any idea? Thanks,
Alan

[Peter Eastman]

It looks like all of the errors are related to the free energy plugin. Unless you're actually planning to use that plugin (and it isn't yet really ready for production use), you can just turn it off. That's the OPENMM_BUILD_FREE_ENERGY_PLUGIN and OPENMM_BUILD_FREE_ENERGY_CUDA_LIB options in CMake. In fact, those options *should* have been turned off by default. Were they not?

Peter

[Alan Wilter Sousa da Silva]

Thanks Peter!

Indeed, turning off only OPENMM_BUILD_FREE_ENERGY_CUDA_LIB and I could compiled in 64 bits.

Then doing make test I got "10% tests passed, 69 tests failed out of 77".

Confused with this result I did the compilation in 32 bits with lib above ON and got "12% tests passed, 69 tests failed out of 78". Essentially the same tests that failed in 32 bits failed in 64 as well. The only extra one in 32 bits was TestCudaGBVISoftcoreForce.

So, how do I interpret these so many failures (SEGFAULT, OTHER_FAULT or Failed)?

Thanks,

Alan

[Peter Eastman]

Which ones are failing? If, for example, all the CUDA test cases fail, it probably means you don't have the CUDA lib directory (typically /usr/local/cuda/lib) in your DYLD_LIBRARY_PATH.

Try running some individual test cases from the command line. They should print out error messages saying what happened.

> The only extra one in 32 bits was TestCudaGBVISoftcoreForce.

That's part of the free energy plugin, so it shouldn't exist if you have turned off OPENMM_BUILD_FREE_ENERGY_CUDA_LIB. Perhaps do a "make clean" and then rebuild from scratch?

Peter

[Alan Wilter Sousa da Silva]

OK, let's go.

echo $DYLD_LIBRARY_PATH
/opt/intel/Compiler/11.1/088/lib:/opt/intel/Compiler/11.1/088/lib/intel64:/opt/intel/Compiler/11.1/088/Frameworks/ipp/Libraries:/opt/intel/Compiler/11.1/088/Frameworks/mkl/lib/em64t:/opt/intel/Compiler/11.1/088/Frameworks/tbb/ia32/cc4.0.1_os10.5.4/lib:/usr/local/openmm/lib:/usr/local/cuda/lib:

If lib cuda was not properly installed I doubt I would even compile openmm. Besides, all Cuda SDK examples compile and work in 64 bits.

I can see:

file /usr/local/cuda/lib/*
/usr/local/cuda/lib/libcublas.dylib: Mach-O universal binary with 2 architectures
/usr/local/cuda/lib/libcublas.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
/usr/local/cuda/lib/libcublas.dylib (for architecture i386): Mach-O dynamically linked shared library i386
[etc...]

For the tests, is better say which worked (only 8):

5/77 Test #5: TestReferenceCustomBondForce .............. Passed 0.05 sec
6/77 Test #6: TestReferenceCustomExternalForce .......... Passed 0.01 sec
14/77 Test #14: TestReferenceHarmonicAngleForce ........... Passed 0.01 sec
15/77 Test #15: TestReferenceHarmonicBondForce ............ Passed 0.01 sec
21/77 Test #21: TestReferencePeriodicTorsionForce ......... Passed 0.01 sec
22/77 Test #22: TestReferenceRBTorsionForce ............... Passed 0.02 sec
23/77 Test #23: TestReferenceRandom ....................... Passed 0.46 sec
69/77 Test #69: TestOpenCLRandom .......................... Passed 0.34 sec

Yes, no cuda test worked, but even most of reference tests failed. I am completely puzzled. With openmm1.1 (and cuda 3.0), at least in 32 bits all tests worked, I remember.

And yes, I always do 'make clean' when not starting from scratch.

Could be my /usr/local/openmm/lib (still openmm1.1) that is mangling my test compilation? I will remove this and test all again.

Thanks,

Alan

[Alan Wilter Sousa da Silva]

Removing current /usr/local/openmm/lib and "make test" went much better. Perhaps it would mindful to have in the manual a warning for those who have openmm already installed to be aware.

In 32 bits, using default ccmake and everything went fine. all 78 tests passed.

In 64 bits, setting OPENMM_BUILD_FREE_ENERGY_CUDA_LIB = Off, from 77 tests, all tests passed except all cuda ones.

For example:

./TestCudaBrownianIntegrator
cudaMemcpyToSymbol: SetSim copy to cSim failed invalid argument

(BTW, all failed with the same error above)

Just to refresh, I need things in 64 bits because I want to work with pyopenmm and my python is in 64 bits (Fink).

I hope it may help you to help me.

Many thanks

Alan

[Alan Wilter Sousa da Silva]

Dear Peter,

You once told me you develop in Mac. Would it be possible to you to try to build openmm in 64 bits since now CUDA 3.1 in 64 bits is available for Mac?

Thanks,

Alan