Page 1 of 1

snapshot of solution every few iterations

Posted: Wed Sep 27, 2023 1:58 am
by deyvidpi
Hi all,

I am running Moco simulations, and sometimes the server crashes, or the IPOPT optimizer does not converge, etc. Regardless of the reasons, then Python is exited and I lose all variables, in particular my intermediate solution.

Is there any way to take a snapshot of my solution every few iterations and dump it in a sto file, like it is done with checkpoints in machine learning?

Re: snapshot of solution every few iterations

Posted: Mon Oct 02, 2023 9:45 am
by nbianco
Hi Pagnon,

This is possible through CasADi's intermediate callback feature, but it's currently broken in Moco. We have an issue tracking the bug here: https://github.com/opensim-org/opensim-core/issues/3265.

In the meantime, you could always set the maximum iterations limit on your problem to get an "intermediate" solution for debugging. Not ideal, I know, but better than nothing.

Best,
Nick

Re: snapshot of solution every few iterations

Posted: Mon Oct 02, 2023 3:44 pm
by deyvidpi
Well that's life! At least I know I can stop searching, and in the meantime I subscribed to the issue in order to be notified when it is fixed :)