Installing cuda Gromacs/Openmm from source

[Peter Eastman]

It's just a guess, but did you remember to run automake and autoconf to rebuild the makefiles after you copied in the OpenMM modifications? See this message:

https://simtk.org/forum/message.php?msg_id=2119

We're looking into creating binaries for 64 bit Linux, since there seems to be a lot of demand for them, and as you've discovered, the build process is nontrivial. Hopefully we'll have those posted soon.

Peter

[Huijun Zhu]

Thanks for the tips. After automake and autoconf, there are some problems of linking. The OpenMM is installed in /usr/local/openmm. The following are messages at the end of "make" command:

g++ -g -O2 -o mdrun glaasje.o gctio.o ionize.o do_gct.o repl_ex.o xutils.o md.o mdrun.o genalg.o /usr/local/openmm/lib/libOpenMM.so ./.libs/libgmxpreprocess.a -L/usr/lib64 ./.libs/libopenmminterface.a ../mdlib/.libs/libmd.a /root/gromacs/gromacs-4.0.5/src/gmxlib/.libs/libgmx.a ../gmxlib/.libs/libgmx.a -lxml2 -lnsl -lfftw3f -lm -lX11
./.libs/libopenmminterface.a(md_openmm.o): In function `openmm_init':
/root/gromacs/gromacs-4.0.5/src/kernel/md_openmm.cpp:122: undefined reference to `OpenMM::Platform::getDefaultPluginsDirectory()'
/root/gromacs/gromacs-4.0.5/src/kernel/md_openmm.cpp:122: undefined reference to `OpenMM::Platform::loadPluginsFromDirectory(std::basic_string<char, std::char_traits<char>, std::allocator<char> >)'
...

There are more similar messages about "undifined reference" afterwards. What additional options are needed for compiling?

Thanks!

[Peter Eastman]

Is it possible that you're compiling the PR3 version of Gromacs against the PR2 version of OpenMM? That's what this error message looks like to me.

It might be a good idea to completely delete your openmm install directory (/usr/local/openmm) and then reinstall it. That will make sure everything in it is correct.

Peter

[Huijun Zhu]

Hi Peter,

You were right, I had OpenMM-PR2 installed in /usr/local/openmm, and I thought the new installation of PR3 would cover old files. However, the files of both versions were mixed. I deleted the /usr/local/openmm and re-installed OpenMM-PR3 (cd OpenMMPreview3-Source/src, mkdir bin, cd bin, ccmake .., make install). The libcudpp.a and libcutil.a were recompiled with -fPIC as suggested before. When using "make test" to test the installation, test 28 TestCudaRBTorsionForce failed.

In the Gromacs-4.0.5 installation, it ends with the following messages:

Making all in kernel
make[3]: Entering directory `/root/gromacs/gromacs-4.0.5/src/kernel'
make[3]: *** No rule to make target `/usr/local/openmm/lib/libOpenMM.*', needed by `grompp'. Stop.
make[3]: Leaving directory `/root/gromacs/gromacs-4.0.5/src/kernel'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/root/gromacs/gromacs-4.0.5/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/root/gromacs/gromacs-4.0.5/src'
make: *** [all-recursive] Error 1

Any suggestions for these?

Regards,

Huijun

[Peter Eastman]

> make[3]: *** No rule to make target `/usr/local/openmm/lib/libOpenMM.*',
> needed by `grompp'. Stop.

That indicates that it isn't finding the main OpenMM library (libOpenMM.so). Take a look in /usr/local/openmm/lib. Is it there? Perhaps you compiled it in debug mode (if so, the library name will end in _d), or you only built a static library (the name will include _static in it)? Gromacs requires a dynamic library compiled in release mode.

Peter

[Huijun Zhu]

Hi Peter,

Thanks to all your diagnnoses and previous posts that the OpenMM and Gromacs are installed on our 64-bit Linux machine. Here is a summary of installation, hopefully it will be useful for other users.

1. Install Cuda-2.2 (following NVIDIA instructions)
2. Recompile cudpp and cutil for 64-bit machine
2.1 Download cudpp_1.0a package from http://gpgpu.org/developer/cudpp
2.2 Open the file linux_build/common.mk, find the line starting with "# Compilers", then modify the following lines as:
NVCC := nvcc -Xcompiler -fPIC
CXX := g++ -fPIC
CC := gcc -fPIC
LINK := g++ -fPIC
2.3 Enter the directory cudpp_1.0a/cudpp, compile it by "make", then libcudpp64.a is generated in cudpp_1.0a/lib
2.4 Enter the directory cudpp_1.0a/common, compile it by "make", then libcutil.a is generated in cudpp_1.0a/lib.
3. Install OpenMM-Preview3
3.1 Download OpenMM_Preview3-Source
3.2 Copy libcudpp64.a and libcudtil.a (from the last step) to OpenMMPreview3-Source/src/platforms/cuda/cudpp/linux, and make libcudpp.a link to libcudpp64.a
3.3 Enter OpenMMPreview-Source/src, mkdir bin, cd bin, ccmake ..; on the screen brought by ccmake, make sure "CMAKE_BUILD_TYPE" is "Release", "CUDA_BUILD_TYPE" is "Device", set "OPENMM_BUILD_CUDA_LIB" to "ON", then type "make install", it will be installed in /usr/local/openmm.
4. Install Gromacs-4.0.5 with OpenMM
4.1 Download Gromacs-4.0.5
4.2 Download GromacsOpenMMPreview3-Linux32 and merge the files provided in src directory to Gromacs-4.0.5 with the same directory structure
4.3 Enter the directory of Gromacs-4.0.5, automake, autoconf (automake should be updated to version 1.10, and autoconf should be updated to version 2.61), ./configure, make install. This will install Gromacs-4.0.5 to /usr/local/gromacs.

[Peter Eastman]

That's excellent! Thanks for all your work on this! I know a lot of people are interested in using OpenMM on 64 bit Linux.

Peter

[Dirk Colbry]

Using the hints from Huljun Zhu and others (thank you!), I have also managed to successfully install the latest version of gromacs (4.0.7) with openmm (1.0beta) on 64 bit linux (sles10).

What is interesting about my build is that I used the fft libraries provided with the Intel MKL (Math Kernel Library). I also used the intel compilers instead of gnu and none of the libraries where in standard install locations so I had to hard code the paths.

First I installed openmm using the standard cmake install. Then I copied the GromacsOpenMM1.0beta-Linux64 folder to the main directory of my gromacs build tree. Finally, I ran the helper script shown below.

This install procedure is probably more complex than it should be and the script I provide below will not work on other peoples system. However, I hope the script will provide some incite to others trying to install gromacs and openmm from scratch.

I hope you find this useful,

- Dirk

-----------------------------------------------
Dr. Dirk Joel Luchini Colbry
Research Specialist
Institute for Cyber-Enabled Research (iCER)
Michigan State University
web: http://www.dirk.colbry.com

----- Start build_gromacs_openmm.sh -----
#!/bin/bash --login
# Written by Dirk Colbry 01-06-10
# For Michigan State University High Performance Computing Lab

#Copy the OpenMM source changes to the current gromacs build tree
cp -r ./GromacsOpenMM1.0beta-Linux64/src .

rm Makefile

# Load in system settings (Specific for our cluster)
module load use.cus
module load intelcc
module load mkl
module load cuda
module load openmm/1.0beta

#rebuild the configure file
autoreconf -fvi
autoconf
automake
make clean

# Set local variables for custom file locations
ROOT_DIR=/opt/cus/
OpenMM_Path=$ROOT_DIR/openmm/OpenMM1.0beta-Source
Gromacs_Path=$ROOT_DIR/gromacs/gromacs-4.0.7
MKL_PATH=/opt/intel/mkl/10.1.2.024
CUDA_PATH=/opt/hpc/cuda/cuda-2.3

# Openmm Library locations
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$OpenMM_Path/lib/:$OpenMM_Path/lib/plugins/
export INCLUDE=$INCLUDE:$OpenMM_Path/include/
export INCLUDE=$INCLUDE:$OpenMM_Path/include/openmm
export OPENMM_PLUGIN_DIR=$OpenMM_Path/lib/plugins

# Cuda path
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CUDA_PATH/lib64
export INCLUDE=$INCLUDE:$CUDA_PATH/include/

# fft path
export INCLUDE=$INCLUDE:$MKL_PATH/include/:$MKL_PATH/include/fftw/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MKL_PATH/lib/em64t/

# Set build settings using intel compilers
export CC=iccexport CXX=icpc
export CFLAGS="-g -axW"
export CPPFLAGS="-I${CUDA_PATH}/include/ -I${OpenMM_Path}/include/ -I${OpenMM_Path}/include/internel -I${OpenMM_Path}/in
clude/openmm/ -I${MKL_PATH}/include/fftw -I${MKL_PATH}/include/"
export LDFLAGS="-L${CUDA_PATH}/lib64/ -L${OpenMM_Path}/lib/ -L${OpenMM_Path}/lib/plugins/ -L${MKL_PATH}/lib/em64t/ -lfft
w3xc_intel -lmkl_lapack -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lm -lOpenMM -lOpenMMCuda -lcudart"
export FFLAGS="-g -axW"

#Modify configure to work with MKL fft by removing extra library option
sed "s/\"-lfftw3f/\"/g" configure > test
sed "s/\"-lfftw3/\"/g" test > configure

#Configure the build
./configure \
--prefix=$Gromacs_Path \
--with-fft=fftw3 \
--enable-double \
--disable-software-sqrt \
--enable-prefetch-forces \
--program-suffix=_openmm \
--disable-cpu-optimization \
--with-external-blas \ --with-external-lapack

#Change path to non standard openmm directory (There is probably a setting to do this in configure)
sed "s/usr\/local\/openmm\/lib/opt\/cus\/openmm\/OpenMM1.0beta-Source\/lib/g" < src/kernel/Makefile > temp.makefile
sed "s/usr\/local\/openmm\/include/opt\/cus\/openmm\/OpenMM1.0beta-Source\/include/g" < temp.makefile > src/kernel/Makefile

#Copy the params file needed by the openmm version of gromacs
cp ./GromacsOpenMM1.0Beta-Linux64/params.agb ../share/gromacs/top/

#Continue with the make
make -j16

#install the files
make install


---- end script ----