Unising Matlab with OpenSim API in High Performance Computing clusters

[Pavlos Silvestros]

Dear all,

I am about to try running an optimisation process on Matlab that will be running OpenSim Forward Dynamic simulations in the cost function through the OpenSim 3.3 API on a HPC cluster. The Matlab code for the optimisation algorithm is suitable for HPC computing (parallelisation) but I am not sure how OpenSim can be set up for this use.

Has anyone got any experience or encountered any information about using OpenSim in such a way?

Thank you in advance for your help.

Best regards,
Pavlos

[Christopher Dembia]

Make sure you are not using the same OpenSim Model on multiple threads.

[Pavlos Silvestros]

Hi Christopher

Thank you for your response.

How can I ensure that the model isn't used over multiple threads? Do you mean when handling the model through the API or more generally such as when a model is being used during ForwardDynamics simulations?

Best regards,
Pavlos

[Christopher Dembia]

This depends on how you are parallelizing your code. If you are simply invoking OpenSim's command-line tools in parallel, there is little to worry about (make sure you're not overwriting the same output files).

If you are using models in MATLAB through the OpenSim MATLAB interface, then you should not use these models in parallel in MATLAB (e.g., parfor). Also, I'm not sure you can even use OpenSim models with parfor.

[Tom Augenstein]

If you are using models in MATLAB through the OpenSim MATLAB interface, then you should not use these models in parallel in MATLAB (e.g., parfor). Also, I'm not sure you can even use OpenSim models with parfor.

Does this mean that if you wanted to use MATLAB's parfor, there would have to be a unique model file for each loop iteration?

[Adrian Lai]

Hi Tom and Chris,

I might be able to help here.

When I've run parfor in MATLAB with the OpenSim API, a new instant of the model was needed for each loop iteration (see the code below). It means that each thread must have a unique model in which to use rather than one that has been previously initialised outside of the parfor loop.

Code: Select all

parfor i=1:Ncolloc
    import org.opensim.modeling.*
    osimModel = Model(model_name);
    osimState = osimModel.initSystem();
    % Rest of your code
    end

Good luck!

Adrian

[Ross Miller]

My brief experience in attempting this was the same as Adrian's: needed to load and initialize a new instance of the model within the loop.

Now I just run Direct Collocation simulations on a single computer =) But those calculations can (in theory) be parallelized too.

[Tom Augenstein]

Thanks for the replys!

I'm attempting to run it now but I'm getting the following error:

"An UndefinedFunction error was thrown on the workers for 'Model'. This might
be because the file containing 'Model' is not accessible on the workers. Use
addAttachedFiles(pool, files) to specify the required files to be attached.
See the documentation for 'parallel.Pool/addAttachedFiles' for more details.

Caused by:
Undefined function 'Model' for input arguments of type 'char'."

The code works perfectly fine without parallel (or at least I think, it takes 12 hours so I haven't seen the output yet).

Has anyone experienced a similar error, and if so, what are the files I should attach?

Also, here is the code that I am running:

tic
poolobj = parpool(48);
numForce = 3;

% declare set-up file
setup = 'C:\Users\NeuRRo Lab\Desktop\armModel\CMC_Matlab\reaching_CMC_setup.xml';

parfor i = 1:numForce

import org.opensim.modeling.*

model = Model('scaled_upper_arm_CMC.osim');
cmc = CMCTool(setup);

% Set model
state = model.initSystem();

dir = ['CMC_results' i];
cmc.setResultsDir(dir);

cmc.run()

disp('ran')

end
delete(poolobj);
toc

[Thomas Uchida]

Can you confirm that each thread is using a different working directory? If not, you may be encountering a sharing violation as multiple threads attempt to write to the same out.log and err.log files.

[Tom Augenstein]

Ok, I think that may be the issue. It seems like it is always writing these files to the .m file (the parallel file I'm trying to run) location for every thread. Is there a way to control where these files are output?