IPOPT output to opensim.log

[Carlos Gonçalves]

Hello everyone,

Is there a way to save the IPOPT output (iteration per iteration) to opensim.log?

Best regards.

[Aaron Fox]

Hi Carlos,

There's a useful feature in newer versions of OpenSim (can't remember when it was added) that allows you to specify a file where the output writes to. I often split this up for different analyses steps (e.g. scaling, IK, Moco etc.) - and doing so should write the IPOPT output to file.

In python you would use:

Code: Select all

import opensim as osim

#Remove any existing file logger
osim.Logger.removeFileSink()

#Add a new file to log to
osim.Logger.addFileSink('fileName.log')
        

Hope that helps.

Aaron

[Carlos Gonçalves]

Thanks a lot Aaron.

I tried your code, and it worked, but I'm still without the IPOPT iteration messages in the log.

Here is what I got:

Code: Select all

[2023-05-12 07:20:33.937] [info] Number of threads: 10
[2023-05-12 07:26:03.851] [info] Set log level to Trace.
[2023-05-12 07:26:03.888] [info] ------------------------------------------------------------------------
[2023-05-12 07:26:03.888] [info] Elapsed real time: 331 second(s) (5 minute(s), 31 second(s)).
[2023-05-12 07:26:03.888] [info] Fri May 12 07:26:03 2023
[2023-05-12 07:26:03.888] [warning] MocoCasADiSolver did NOT succeed:
[2023-05-12 07:26:03.888] [warning]   User_Requested_Stop
[2023-05-12 07:26:03.888] [info] ========================================================================

So, after setting the number of threads, IPOPT kicks in and starts sending his messages in the console. All the information about the objective goal and so on. That's what I'm missing.

In the end, IPOPT finishes, and I get the answer about convergence succeeding or failing in the log.

Any suggestions?

Best regards.

[Nicholas Bianco]

Hi Carlos,

This is probably because the Logger controls all the output messages created by OpenSim, but not by the ones created by Matlab. To fix, I think we'd need to somehow pipe the IPOPT output into the output stream that's managed by OpenSim.

Unless...Aaron, have you gotten IPOPT output to save to a file using OpenSim::Logger()?

-Nick

[Pavlos Silvestros]

Hi Guys,

Carlos you could create a file called ipopt.opt (https://coin-or.github.io/Ipopt/OPTIONS.html) in the directory you are calling it from and specify a file IPOPT to print to after it is done. I have not done it for this exact problem but have used ipopt.opt file in the past as a work around where I can't get access to IPOPT through the OpenSim API.

Let me know if it works!

Cheers,
Pavlos

[Aaron Fox]

Hi Nick, Pavlos & Carlos,

I just double checked the outputs from some recent simulations we ran, and I have the same sort of output as Carlos - where the iterations are missing. I had assumed that the log was capturing everything in the MATLAB output window. I've checked some of my work in Python as well and it's the same thing in the logs.

Aaron

[Nicholas Bianco]

Thanks Pavlos!

That seems like the way to go until we figure out a way to get all output always piped to the Matlab console.

-Nick

[Pagnon David]

Thanks Pavlos!

Another way to do it is by redirecting everything like so:

Code: Select all

python script.py > full_log.txt

But you can't always do that, like if you want to define your log path in the python script. With your method, here is how I go about it:

Create ipopt.opt and add this single line:

Code: Select all

output_file last_log.txt

python script

Code: Select all

import opensim as osim
osim.Logger.removeFileSink()
osim.Logger.addFileSink('full_log.log')

<your moco code>

with open('full_log.txt', 'a') as opensim_log_f:
    with open('last_log.txt', 'r') as ipopt_log_f:
        opensim_log_f.write(ipopt_log_f.read())

It save almost everything, but the following lines. Are they the output of CasADi rather that of IPOPT? Would anyone have an idea of how to save them?

Code: Select all

         nlp  :   t_proc      (avg)   t_wall      (avg)    n_eval
callback_fun  |   1.00ms ( 52.63us)   1.53ms ( 80.44us)        19
       nlp_f  |        0 (       0) 277.00us ( 14.58us)        19
       nlp_g  |  82.00ms (  4.32ms)  80.71ms (  4.25ms)        19
  nlp_grad_f  |   2.00ms (100.00us)   3.65ms (182.64us)        20
   nlp_jac_g  | 148.00ms (  7.40ms) 147.23ms (  7.36ms)        20
       total  | 271.00ms (271.00ms) 271.68ms (271.68ms)         1

Breakdown of objective (including weights):
  goal: 0.402694

[Nicholas Bianco]

Hi David,

Yes, those lines are specific to CasADi/Moco. They should at least get printed to the opensim.log file for now, until we find a way to have everything piped to the same file.

-Nick