simtk.org

Hi there,

I've just installed Cuda 3.0 on my Mac. It has finally cuda libs in 64 bits.

I am trying to compile openmm1.1.1 but first I don't know with gcc was used with CUDA. Still gcc 4.0 or finally gcc4.2? How to check that?

Thanks,

Alan

BTW, I did a try with gcc4.2

However, I found out that for CUDA, not all libs needed are in 64 bits yet.

amadeus[2397]:~/Downloads/build_openmm% for n in `ls /usr/local/cuda/lib/*`; do file $n; done
/usr/local/cuda/lib/libcublas.dylib: Mach-O dynamically linked shared library i386
/usr/local/cuda/lib/libcublasemu.dylib: Mach-O dynamically linked shared library i386
/usr/local/cuda/lib/libcuda.dylib: Mach-O universal binary with 2 architectures
/usr/local/cuda/lib/libcuda.dylib (for architecture i386): Mach-O dynamically linked shared library i386
/usr/local/cuda/lib/libcuda.dylib (for architecture x86_64): Mach-O 64-bit dynamically linked shared library x86_64
/usr/local/cuda/lib/libcudart.dylib: Mach-O dynamically linked shared library i386
/usr/local/cuda/lib/libcudartemu.dylib: Mach-O dynamically linked shared library i386
/usr/local/cuda/lib/libcufft.dylib: Mach-O dynamically linked shared library i386
/usr/local/cuda/lib/libcufftemu.dylib: Mach-O dynamically linked shared library i386
/usr/local/cuda/lib/libtlshook.dylib: Mach-O dynamically linked shared library i386

Compiling for CUDA only and in 32 bits worked fine, including 'make test' all passed.

However, trying to compile for OpenCL, I got this:

[ 84%] Building CXX object platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLCompact.cpp.o
In file included from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLContext.h:37,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLArray.h:30,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLCompact.h:28,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLCompact.cpp:27:
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:525:27: error: OpenGL/opengl.h: No such file or directory
In file included from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLContext.h:37,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLArray.h:30,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLCompact.h:28,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLCompact.cpp:27:

[snip]

/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7677: error: ‘cl_command_queue_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7679: error: expected initializer before ‘getInfo’
make[2]: *** [platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLCompact.cpp.o] Error 1
make[1]: *** [platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/all] Error 2
make: *** [all] Error 2

Thanks,

Alan

The precompiled binaries were built with gcc 4.0, since that was all that CUDA 2.3 supported. Using CUDA 3.0 and compiling from source with gcc 4.2 works fine, though. I just tested, and everything compiled and ran correctly, including OpenCL. How exactly are you compiling? The error message about OpenGL/opengl.h not being found looks very strange, since that's a standard system header.

Peter

Many thanks Peter.

So, let's see it well.

I have Mac OSX 10.6.3, Xcode 3.2.2, Cuda 3.0 and OpenMM 1.1.1.

I created a folder build_openmm, and did:

~/Downloads/build_openmm% ccmake -i ../OpenMM1.1.1-Source/src/

BUILD_TESTING ON
CMAKE_BACKWARDS_COMPATIBILITY 2.4
CMAKE_BUILD_TYPE Debug
CMAKE_INSTALL_PREFIX /usr/local/openmm
CMAKE_OSX_ARCHITECTURES
CMAKE_OSX_DEPLOYMENT_TARGET
CMAKE_OSX_SYSROOT /Developer/SDKs/MacOSX10.6.sdk
CUDA_BUILD_TYPE Device
CUDA_INSTALL_PREFIX /usr/local/cuda/bin
CUDA_SDK_ROOT_DIR /Developer/GPU Computing/C
CUDA_TOOLKIT_ROOT_DIR /usr/local/cuda
DL_LIBRARY /usr/lib/libdl.dylib
FOUND_CUBLAS /usr/local/cuda/lib/libcublas.dylib
FOUND_CUBLASEMU /usr/local/cuda/lib/libcublasemu.dylib
FOUND_CUFFT /usr/local/cuda/lib/libcufft.dylib
FOUND_CUFFTEMU /usr/local/cuda/lib/libcufftemu.dylib
FOUND_CUT FOUND_CUT-NOTFOUND
FOUND_CUT_INCLUDE FOUND_CUT_INCLUDE-NOTFOUND
GCCXML_PATH GCCXML_PATH-NOTFOUND
OPENCL_INCLUDE_DIR /System/Library/Frameworks/OpenCL.framework
OPENCL_LIBRARY /System/Library/Frameworks/OpenCL.framework
OPENMM_BUILD_API_WRAPPERS OFF
OPENMM_BUILD_CUDA_LIB OFF
OPENMM_BUILD_FREE_ENERGY_CUDA_ OFF
OPENMM_BUILD_FREE_ENERGY_PLUGI OFF
OPENMM_BUILD_OPENCL_LIB ON
OPENMM_SVN_REVISION exported
SVNVERSION_PROGRAM /sw/bin/svnversion

(if compiling only for CUDA, 32 bits) all went fine.)

I also did this in ~/Downloads/OpenMM1.1.1-Source/src/CMakeLists.txt:

# Build 32 bit binaries, since CUDA doesn't currently work with 64 bit
#IF (APPLE)
# SET (CMAKE_CXX_FLAGS "-arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5")
# SET (CMAKE_C_FLAGS "-arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -mmacosx-version-min=10.5")
#ENDIF (APPLE)

then, make, and got:

[snip]
[ 83%] Generating ../src/OpenCLKernelSources.cpp, ../src/OpenCLKernelSources.h
Scanning dependencies of target OpenMMOpenCL_d
[ 84%] Building CXX object platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLCompact.cpp.o
In file included from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLContext.h:37,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLArray.h:30,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLCompact.h:28,
from /Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/OpenCLCompact.cpp:27:
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:1653: error: expected identifier before numeric constant
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:1653: error: expected ‘,’ or ‘...’ before numeric constant
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:1655: error: expected constructor, destructor, or type conversion before ‘F’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:1903: error: ‘param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:1903: error: ‘param_name’ was not declared in this scope
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:1905: error: explicit specialization of non-template ‘cl::detail::param_traits’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:1907: error: ‘param_name’ was not declared in this scope
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:1909: error: ‘T’ does not name a type
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:2517: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:2517: error: ‘cl_device_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:2517: error: ‘cl_device_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:2519: error: expected initializer before ‘getInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:2699: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:2699: error: ‘cl_platform_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:2699: error: ‘cl_platform_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:2701: error: expected initializer before ‘getInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3239: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3239: error: ‘cl_context_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3239: error: ‘cl_context_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3241: error: expected initializer before ‘getInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3469: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3469: error: ‘cl_event_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3469: error: ‘cl_event_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3471: error: expected initializer before ‘getInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3603: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3603: error: ‘cl_profiling_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3603: error: ‘cl_profiling_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3605: error: expected initializer before ‘getProfilingInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3815: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3815: error: ‘cl_mem_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3815: error: ‘cl_mem_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:3817: error: expected initializer before ‘getInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:4071: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:4071: error: ‘cl_image_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:4071: error: ‘cl_image_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:4073: error: expected initializer before ‘getImageInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:4589: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:4589: error: ‘cl_sampler_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:4589: error: ‘cl_sampler_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:4591: error: expected initializer before ‘getInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:6185: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:6185: error: ‘cl_kernel_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:6185: error: ‘cl_kernel_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:6187: error: expected initializer before ‘getInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:6313: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:6313: error: ‘cl_kernel_work_group_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:6313: error: ‘cl_kernel_work_group_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:6315: error: expected initializer before ‘getWorkGroupInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7201: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7201: error: ‘cl_program_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7201: error: ‘cl_program_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7203: error: expected initializer before ‘getInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7323: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7323: error: ‘cl_program_build_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7323: error: ‘cl_program_build_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7325: error: expected initializer before ‘getBuildInfo’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7677: error: ‘cl::detail::param_traits’ is not a template
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7677: error: ‘cl_command_queue_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7677: error: ‘cl_command_queue_info’ is not a member of ‘cl::detail’
/Users/alan/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp:7679: error: expected initializer before ‘getInfo’
make[2]: *** [platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLCompact.cpp.o] Error 1
make[1]: *** [platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/all] Error 2
make: *** [all] Error 2

Note that when I updated to Cuda 3.0, the examples in /Developer/GPU\ Computing/OpenCL/ din't work by doing a simple 'make'.

I had to do this: http://forums.nvidia.com/index.php?show ... try1038451

The big point here for me is, are you Peter using Xcode 3.2.2? I frankly agree with you that OpenGL/opengl.h should be standard, but I remember debugging code and finding out problem with case. I have to say that my MacOSX filesystem is case sensitive.

But what's really strange is that I don't have problems with OpenGL/opengl.h, the error is different now. Any idea of how to check that?

Many thanks,

Alan

I have OS X 10.6.3 with Xcode 3.2.1 (though I don't actually use Xcode) and CMake 2.6-patch 4.

> I also did this in ~/Downloads/OpenMM1.1.1-Source/src/CMakeLists.txt:

I wouldn't do that. I haven't tried compiling in 64 bit mode, but as you said not all the CUDA libraries support 64 bit.

> I had to do this: http://forums.nvidia.com/index.php?show ... try1038451

I think that's something different: it sounds like Nvidia used the wrong include (CL/ instead of OpenCL/) on Macs for some of their files. But we're using the correct include for OpenGL, so that shouldn't be the problem.

Peter

Hi Peter, thanks.

More on this issue.

So I am doing OpenCL in 32 bits, but the whole point is I want the stuff in 64 bits and so far only OpenCL can give me this. In the end I don't even need CUDA libs.

I have pyopencl working fine in my laptop, with python and everything else in 64 bits. And I want to use pyopenmm.

So, that said, I forgot to mention that I needed to change file ~/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp, at line 274 from:

#include <OpenGL/opengl.h>

to

#include <OpenGL/OpenGL.h>

In Mac: /System/Library/Frameworks/OpenGL.framework/Versions/A/Headers/OpenGL.h

So you see, I have problems with case, which most people using vanilla osx filesystem wouldn't have, but I am not going to spend time here explaining why HFS+ should have to be case sensitive by default.

Besides, I found out that editing file ~/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/src/cl.hpp with TextWrangler simply mess the stuff (add a extra line for every <cr>), probably because this file comes from Windows. Fortunately, vi or Xcode worked fine.

That said, I finally got OpenMM with OpenCL in 32 bits.

Now it's bring back to the 64 bits quest. You may not remember, but I am following your advice here ([ openmm-Features-1028 ] OpenMM in 64bits using OpenCL in Mac : https://simtk.org/tracker/index.php?fun ... 1&atid=436)

So trying OpenCL 64 bits:

[ 87%] Building CXX object platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLParameterSet.cpp.o
[ 88%] Building CXX object platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLPlatform.cpp.o
[ 88%] Building C object platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/hilbert.c.o
[ 88%] Building CXX object platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLKernelSources.cpp.o
Linking CXX shared library ../../../libOpenMMOpenCL_d.dylib
ld: warning: in ../../../libOpenMM_d.dylib, file was built for unsupported file format which is not the architecture being linked (i386)
Undefined symbols:
"OpenMM::CustomBondForce::getEnergyFunction() const", referenced from:
OpenMM::OpenCLCalcCustomBondForceKernel::initialize(OpenMM::System const&, OpenMM::CustomBondForce const&)in OpenCLKernels.cpp.o
"OpenMM::PeriodicTorsionForce::getTorsionParameters(int, int&, int&, int&, int&, int&, double&, double&) const", referenced from:
OpenMM::OpenCLCalcPeriodicTorsionForceKernel::initialize(OpenMM::System const&, OpenMM::PeriodicTorsionForce const&)in OpenCLKernels.c
[snip]
OpenMM::OpenCLCalcCustomGBForceKernel::initialize(OpenMM::System const&, OpenMM::CustomGBForce const&)in OpenCLKernels.cpp.o
OpenMM::OpenCLCalcCustomGBForceKernel::initialize(OpenMM::System const&, OpenMM::CustomGBForce const&)in OpenCLKernels.cpp.o
ld: symbol(s) not found
collect2: ld returned 1 exit status
make[2]: *** [libOpenMMOpenCL_d.dylib] Error 1
make[1]: *** [platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/all] Error 2
make: *** [all] Error 2

Cheers,
Alan

> In Mac: /System/Library/Frameworks/OpenGL.framework/Versions/A/Headers/OpenGL.h

Thanks! I'll make sure that gets fixed.

> ld: warning: in ../../../libOpenMM_d.dylib, file was built for unsupported file format which is not the architecture being linked (i386)

Perhaps you just need to do a "make clean"? It's trying to link against a 32 bit version of the main OpenMM library. It's possible that's just left over from when you were doing 32 bit builds, and if you delete it, the build script will regenerate it in 64 bit mode.

Peter

Thanks Peter,

Some more progress.

Trying in 64 bits.

So, what is holding is what you said. It is in ~/Downloads/build_openmm/platforms/opencl/sharedTarget

For some reason, the libs there

./platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLCompact.cpp.o: Mach-O object i386
./platforms/opencl/sharedTarget/CMakeFiles/OpenMMOpenCL_d.dir/__/src/OpenCLContext.cpp.o: Mach-O object i386
etc.

are being built in 32 bits.

I don't know much cmake, but I need to put a flag like "-m64" somewhere to be sure things are compiled and linked in 64 bits.

Would you know where to tweak?

Many thanks,

Alan

I found the offending lines in ~/Downloads/OpenMM1.1.1-Source/src/platforms/opencl/CMakeLists.txt

IF (APPLE)
# SET (CMAKE_CXX_FLAGS "-arch i386")
# SET (CMAKE_C_FLAGS "-arch i386")
ENDIF (APPLE)

All 49 tests passed.

Many thanks Peter!
Alan