Using a Custom OpenSim Build

[Aaron Fox]

Hi OpenSim Team,

I have forked a copy of the OpenSim core code to create a series of new components, classes etc. for specific simulation purposes. I am wondering if there are any ideas or recommendations for how I could use these more readily in a batch processing framework to process a large amount of data/trials using the tools. Typically, with the usual OpenSim tools, I would do this by creating a Python script that I could run iterations of to process multiple participants, trials etc. I could achieve this by creating a plugin version of the new components from the custom build, but this still wouldn't work that well as I can't access the new components through Python bindings (or can I...?). Using the new components is nice and easy through C++, but I haven't figured out a great way to make this process efficient/usable in a batch processing manner - as all of the C++ examples are quite individualised. Further, I'd like to take advantage of our uni HPC cluster which runs via Python on Linux.

To summarise the question here, is there a way to make a custom OpenSim build user friendly for batch processing data (e.g. creating Python bindings, using C++) or is converting to a plugin approach the better option here?

Aaron

[Mohammadreza Rezaie]

Hi, the build instructions might be helpful: https://github.com/opensim-org/opensim- ... structions

You can also create or modify the workflow for the GitHub Actions of your forked repository. This will automatically build your modified opensim-core and upload the Python bindings with minimal effort whenever you push a commit: https://github.com/opensim-org/opensim- ... ration.yml

Hope this helps.

[Aaron Fox]

Thanks Mohammadreza.

I'm not sure the build script is going to solve my problems - I had some trouble running it a while back with the custom build + I don't have the permissions to run those types of scripts on our HPC Linux system unfortunately.

The workflows is interesting though if this can easily compile/build things like the Python bindings. I might need Ayman or Nick to explain a little bit more about how this workflow works - so hopefully they can chime in here. I also recall Ayman mentioning that there is a place you need to add details on your new classes so they get included in the Python bindings - but I can't remember where this is?

Aaron

[Nicholas Bianco]

Hi Aaron,

You'll need to modify a few files in the Bindings folder of opensim-core to include your custom classes in the bindings.

1. You need to update one of the OpenSimHeaders_*.h files to include header files for the classes you want to include in the bindings.

2. You need to update one of the *.i files to add a similar include statement for your class header file (e.g., %include <OpenSim/Actuators/CoordinateActuator.h>).

These are the bare minimum changes needed before rebuilding the bindings. You may find that certain methods might not work out of the box, which might require custom modifications to the bindings (as seen in some of the *.i files). Python- and Java-specific modifications to the bindings can be found in the subdirectories (e.g., Python/swig/actuators.i).

As for getting the build up on the HPC Linux server: can you not run a script that replicates your local build on the server? If not, you'll need to create a Linux build locally somehow and send it up to the server.

-Nick

[Aaron Fox]

Hi Nick,

Thanks for the advice on the bindings. Once I have everything working with these custom classes in C++ I'll check out if I can get them to work in Python.

I think I'm limited with what I can do permissions wise on the HPC cluster. For example, I did try and run the build script for the OpenSim core on it and it gave me a stern warning about how I couldn't do it and it had been reported (still waiting for the HPC police to come and get me...). The two ways in which it seems I can create an environment on the HPC is to use conda or a docker image via Singularity. I feel that the 2nd option there might be the only approach, which is unfortunate as I have no idea how to do it!

Aaron

[Nicholas Bianco]

Docker/Singularity is actually the best way to go!

You'll first need to create a Docker image and host it on Docker Hub. To do that, you'll first need to write a Dockerfile to create a Docker image that will contain your OpenSim build. Here is an example Dockerfile for building OpenSim. It is a bit outdated, but you can refer to the Linux build scripts on how to update the build commands. You will also need to change the "git clone https://github.com/opensim-org/opensim-core.git" command to whatever your cloned repo is (and change the BRANCH argument, if needed).

Once you create a Docker image locally, you can push it up to Docker Hub (refer to Docker's docs for this). Once the image is hosted on Docker Hub, you can create a Singularity container on the HPC by using the following command:

Code: Select all

singularity pull docker://user/image

where "user" and "image" will match your Docker Hub username and hosted image name. Note that if your HPC is using the open-source version of Singularity, which is called Apptainer, then the command might look like:

Code: Select all

apptainer pull docker://user/image

The whole process isn't too bad once you get it set up, and definitely worth the time investment to have access to your HPC.

[Aaron Fox]

Thanks Nick - you make it sound so easy! I'm sure I'll run into some hurdles when I give it a go though!

Aaron

[Nicholas Bianco]

I'm happy to help get over those hurdles whenever you encounter them!

It would be nice to update the Dockerfile on opensim-core, so it would be useful to know that the whole process is working for you.