Page 1 of 1
Random seed multiple optimizations
Posted: Thu Jul 15, 2021 11:13 pm
by adkoele
Hi,
I would like to run multiple optimizations of the same scenario, each with a different random seed. However, they should use the same seed throughout the optimization. I have tried to program this using the signature (using HasSignature), but this did not work. Is there any way to program this in a controller? Or does this already happen automatically if I do not prescribe a random seed?
So what I want is:
filename.R1 uses random_seed 1
filename.R2 uses random_seed 2
etc.
And I tried to following in my controller:
Code: Select all
if scone.HasSignature == 'pendulum.SC.SM.D20.R1.' then
perturb_seed = 1
elseif scone.HasSignature == 'pendulum.SC.SM.D20.R2' then
perturb_seed = 2
...
end
but scone.HasSignature seems to be empty, so I am probably using the wrong variable.
Thanks for your help!
Kind regards,
Anne
Re: Random seed multiple optimizations
Posted: Fri Jul 16, 2021 1:07 am
by tgeijten
The best approach is to make
random_seed a parameter in your ScriptController, so that you can control it directly from your .scone file. See
this tutorial for an example on how to do this (it works for both ScriptMeasure and ScriptController). You can then specify the random seed in your ScriptController, without needing to change your script:
Code: Select all
ScriptController {
script_file = <your_script.lua>
random_seed = 1
}
If you only need to run a couple of different optimizations, it might be easiest to just change the value by hand for each optimization. You can also change the random_seed of the CmaOptimizer simultaneously, if you wish, by specifying it in the
CmaOptimizer.
If you need to run many optimizations and wish to automate the process, you can do so via the
command line interface. You can create a file called
optimize_with_different_seeds.bat, which would look something like:
Code: Select all
set SCONECMD="C:\Program Files\SCONE\bin\sconecmd.exe"
%SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=1
%SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=2
%SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=3
Please let me know if this helps!
Re: Random seed multiple optimizations
Posted: Mon Jul 19, 2021 4:38 am
by adkoele
Hi,
Not really. I guess the .bat approach could work, but it seems slower than using the "run multiple optimizations" in the GUI, because it does not run in parallel.
I can indeed set the random_seed in the controller file or in the scone file to a specific number, but then the same random_seed is used for all optimizations if I use the "run multiple optimizations" option. However, I am so far not able to make this dependent on the optimization instant, because I can find out how to access the file name that is given by SCONE in either the .lua controller file or the .scone file. Is there a way to do this?
Re: Random seed multiple optimizations
Posted: Mon Jul 19, 2021 8:26 am
by tgeijten
I understand your point, but unfortunately the Optimizer and the Controller operate independent from each other and do not share any data. This means the Controller has no way of knowing the random seed of the Optimizer.
Regarding your performance concern with the .bat file option, it is possible to run multiple optimizations in parallel -- simply by prefixing the command with "start", e.g.:
Code: Select all
set SCONECMD="C:\Program Files\SCONE\bin\sconecmd.exe"
start %SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=1
start %SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=2
start %SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=3
This will create a separate window and thread for each optimization. Please note though that in SCONE, the evaluations of each iteration are already executed in parallel, so this option might not make a lot of difference unless you use a machine with a lot of cores (8 or more).