Same code, different machines, different number of iterations

[Carlos Gonçalves]

Hello everyone.

I'm trying to figure out I got (converged to) such different solutions with the same code in different machines. Below is an example of convergence simulation but with different outcomes (different jump heights in my case).

-Virtual Machine
- Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
- 18GB RAM
- 8 threads
- 314 iterations

- Laptop
- AMD Ryzen 7 5800H with Radeon Graphics 3.20 GHz
- 6GB RAM
- 10 threads (with solver.set_parallel(10))
- 955 iterations

Despite lowering the convergence threshold, I still get different numbers of iterations and results. In some cases, in one machine, the solution converges, and in the other, it fails (1000 iterations).

Any thoughts on that?

Best regards.

[Aaron Fox]

Hi Carlos,

My initial thought would be to check if the same initial guess is used? A different starting point can have an impact on where your solutions ends up (e.g. local minima vs. global minimum).

Aaron

[Carlos Gonçalves]

Hi Aaron,

All files and configuration are the same. I literally copied the folder and ran in the other machine.



Best regards.

[Pavlos Silvestros]

Hey Carlos,

From waht I understand depending on the architecture of the machine some low level changes to the setup of IPOPT can happen. Are they both Windows or Linux machines? Also check what solver IPOPT is selecting as default (e.g. mumps vs pardiso - see pic attached).

Screenshot 2023-04-28 130708.png (23.71 KiB) Viewed 1134 times

There are other possibilities also that might be causing your large discrepancies but this is a good place to start looking!

Cheers,
Pavlos

[Carlos Gonçalves]

Hello Pavlos,

During the simulations in both machines, MUMPS is used by IPOPT.

Since such possible differences exist in simulation results, is there a way to cope with it?

Working with higher convergence thresholds seems impractical because of the increase in simulation convergence time (Model with 19Dof and 92muscles).

Right now, I'm at the methodological crossroads in maybe changing the model to improve the convergence time. Or maybe changing the mesh from 100 to something lower.

Thanks in advance for any tips.

Best regards.

[Pasha van Bijlert]

Hi Carlos,

Presumably you've made sure that both computers are using the same version of opensim?

If you're using implicit tendon dynamics with compliant tendons (and also minimizing auxiliary derivatives), I've found that the weighting of this can affect how fast your optimizations converge. If I need fast convergence and my simulations don't require tendon compliance, turning it off substantially speeds up the search time. Using implicit multibody accelerations (and minimizing their derivatives) also speeds up the search time (but like aux. derivatives, the relative weighting can also impact the actual motion itself).

But I think you're familiar with these parameters in Moco already.

Cheers,
Pasha

[Carlos Gonçalves]

Hello Pasha; thanks for the feedback.

I tried "implicit dynamics" before, but it did not make a difference in my "gait" simulations. It will be nice to check how it will be in "jumping." Just need to update my script for the new guess file with more states.

About the tendons, I just read Rajagopal's model paper properly, and some tendons are set to "ignore_tendon_compliance". In my current model, all tendons are compliant.

I will get back here with that tested as well.

Best regards.

[Carlos Gonçalves]

Hello everyone.

So, I'm running the same code, different machines with the implicity body dynamics.

The simulation in my laptop without implicity body dynamics took 53939 s, and with took 130840 s. The performance (jump height) was similar (23cm jump).

Mr. Bones still needs strength training.

In the virtual machine, it is still running, but maybe the number of iterations can be the same.

Best regards.