Random seed multiple optimizations

[Anne Koelewijn]

Hi,

I would like to run multiple optimizations of the same scenario, each with a different random seed. However, they should use the same seed throughout the optimization. I have tried to program this using the signature (using HasSignature), but this did not work. Is there any way to program this in a controller? Or does this already happen automatically if I do not prescribe a random seed?

So what I want is:
filename.R1 uses random_seed 1
filename.R2 uses random_seed 2
etc.

And I tried to following in my controller:

Code: Select all

if scone.HasSignature == 'pendulum.SC.SM.D20.R1.' then
		perturb_seed = 1
	elseif scone.HasSignature == 'pendulum.SC.SM.D20.R2' then
		perturb_seed = 2
		...
end

but scone.HasSignature seems to be empty, so I am probably using the wrong variable.

Thanks for your help!
Kind regards,
Anne

[Thomas Geijtenbeek]

The best approach is to make random_seed a parameter in your ScriptController, so that you can control it directly from your .scone file. See this tutorial for an example on how to do this (it works for both ScriptMeasure and ScriptController). You can then specify the random seed in your ScriptController, without needing to change your script:

Code: Select all

ScriptController {
	script_file = <your_script.lua>
	random_seed = 1
}

If you only need to run a couple of different optimizations, it might be easiest to just change the value by hand for each optimization. You can also change the random_seed of the CmaOptimizer simultaneously, if you wish, by specifying it in the CmaOptimizer.

If you need to run many optimizations and wish to automate the process, you can do so via the command line interface. You can create a file called optimize_with_different_seeds.bat, which would look something like:

Code: Select all

set SCONECMD="C:\Program Files\SCONE\bin\sconecmd.exe"
%SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=1
%SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=2
%SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=3

Please let me know if this helps!

[Anne Koelewijn]

Hi,

Not really. I guess the .bat approach could work, but it seems slower than using the "run multiple optimizations" in the GUI, because it does not run in parallel.

I can indeed set the random_seed in the controller file or in the scone file to a specific number, but then the same random_seed is used for all optimizations if I use the "run multiple optimizations" option. However, I am so far not able to make this dependent on the optimization instant, because I can find out how to access the file name that is given by SCONE in either the .lua controller file or the .scone file. Is there a way to do this?

[Thomas Geijtenbeek]

I understand your point, but unfortunately the Optimizer and the Controller operate independent from each other and do not share any data. This means the Controller has no way of knowing the random seed of the Optimizer.

Regarding your performance concern with the .bat file option, it is possible to run multiple optimizations in parallel -- simply by prefixing the command with "start", e.g.:

Code: Select all

set SCONECMD="C:\Program Files\SCONE\bin\sconecmd.exe"
start %SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=1
start %SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=2
start %SCONECMD% -o "your_file.scone" CmaOptimizer.SimulationObjective.ScriptController.random_seed=3

This will create a separate window and thread for each optimization. Please note though that in SCONE, the evaluations of each iteration are already executed in parallel, so this option might not make a lot of difference unless you use a machine with a lot of cores (8 or more).