Fabricating and tracking IMU signals

[Grace M]

Hello again, And thank you for these great forums!

I am trying to run a simulation based off the squat to stand example with running kinematics. I want to generate IMU signals that can be tracked in subsequent simulations.

I have successfully added IMUs to my model and run a simulation using MocoInverse as far as I am aware. When I try to extract the accelerometer signals I get the error:

Code: Select all

predictSolution = MocoTrajectory('example3DRunning_MocoInverse_solution.sto')


Java exception occurred:
java.lang.RuntimeException: Datatype 'double
' is not supported.
	Thrown at STOFileAdapter.cpp:77 in createSTOFileAdapterForReading().

	at org.opensim.modeling.opensimMocoJNI.new_MocoTrajectory__SWIG_5(Native Method)

	at org.opensim.modeling.MocoTrajectory.<init>(MocoTrajectory.java:245)

I have attached a copy of my results file. I could not attach it as a .sto so have converted it to text. I do not get this error when I load the file generated with the squat to stand IMU example.

Thanks in advance

Grace

[Grace M]

I wanted to follow up on this analysis. I have managed to extract the accelerometer and gyroscope signals using the code below.

Code: Select all

outputPaths = StdVectorString();
outputPaths.add('.*accelerometer_signal');
accelerometerSignals = opensimSimulation.analyzeVec3(torqueModel, ...
    prevStatesTable, ...
    prevControlsTable, ...
    outputPaths);
imuFramePaths = StdVectorString();
imuFramePaths.add('/bodyset/torso/torso_imu_offset');
imuFramePaths.add('/bodyset/femur_r/femur_r_imu_offset');
imuFramePaths.add('/bodyset/tibia_r/tibia_r_imu_offset');
accelerometerSignals.setColumnLabels(imuFramePaths);
sto = STOFileAdapterVec3();
sto.write(accelerometerSignals,[outputPath filesep solname '_accel.sto']);

outputPaths = StdVectorString();
outputPaths.add('.*gyroscope_signal');
gyroSignals = opensimSimulation.analyzeVec3(torqueModel, ...
    prevStatesTable, ...
    prevControlsTable, ...
    outputPaths);
gyroSignals.setColumnLabels(imuFramePaths);
sto = STOFileAdapterVec3();
sto.write(gyroSignals,[outputPath filesep solname '_gyro.sto']);

I wasn't able to get the orientation signals. I have tried outputPaths.add('.*orientation_as_quaternion');
Does anyone know what the correct output path is for this?

I have also tried using the Analyzetool to extract these signals after I have run the simulation using the following code. "IMUs" is an IMUdatareporter.

Code: Select all

analyzeIMU=AnalyzeTool ;
analyzeIMU.setModel(torqueModel)
analyzeIMU.setStatesStorage(Storage([outputPath filesep solname '_states.sto']));
analyzeIMU.setStartTime(initialTime);
analyzeIMU.setFinalTime(finalTime);

analyzeIMU.updAnalysisSet().cloneAndAppend(IMUs);
analyzeIMU.print("analyzeReportIMUData.xml");
analyzeIMU.run();

This automatically outputs linear acceleration, angular velocity and orientation (as quaterions) files.
When I compare the acceleration files between the two methods the results appear different. Is this due to a different reference frame? Which result would be useable in a MOCO acceleration tracking goal?

Also, I would like to add in a angular velcocity tracking goal and orientation tracking goal (I assume this would assist with accurate IMUtracking?) I am running into errors with each:

When I run the code IMUAngVel = MocoAngularVelocityTrackingGoal('AngularVelocity_tracking',0.1)
I get the error: Unrecognized function or variable 'MocoAngularVelocityTrackingGoal'.

When I try to use the orientations from the analyzetool method in an orientation goal I don't seem to be able to set them as a reference. I have tried:

Code: Select all

IMUOrient = MocoOrientationTrackingGoal('Orientation_tracking',IMUTrackingWeight);
IMUOrient.setFramePaths(imuFramePaths);
orientSignals=TimeSeriesTableQuaternion([outputPath filesep 'Run_torqueDriven_' speed rotations=OpenSenseUtilities.convertQuaternionsToRotations(orientSignals);

IMUOrient.setRotationReference(rotations);

I get the error "Check for incorrect argument data type or missing argument in call to function 'setRotationReference'.

I have noticed the column labels for the frame paths are different and not grouped by IMU and was wondering if this was the issue and if so if there was a relatively simple method to reorder and rename these? Alternatively if I can get the orinetations using output paths that would probably be easier.

Thanks

[Aaron Fox]

Hi Grace,

Apologies for the post that isn't directed as answering your question - I think I'm interested in doing something similar to what you have proposed, in 'fabricating' or 'simulating' makeshift IMU signals from various parts of the body. What's your process been for adding what I assume are these 'fake' IMUs to the model, and how do you think the accuracy of the signals are?

Aaron

[Grace M]

Hi Aaron,

No problem! I have just used the code from the IMUtracking squat to stand example as with the MATLAB code below:

Code: Select all

% Create the base Model by passing in the model file.
modelP = ModelProcessor(OSModel);


% Part 1b: Add frames to the model that will represent our IMU locations. 
% The function addIMUFrame() adds a PhysicalOffsetFrame to a body at a 
% specified location and orientation. Each frame is added at the path:

model = modelP.process();

addIMUFrame(model, 'torso',   Vec3(0.08, 0.3, 0), Vec3(0, 0.5*pi, 0.5*pi));
addIMUFrame(model, 'femur_r', Vec3(0, -0.2, 0.05), Vec3(0, 0, 0.5*pi));
addIMUFrame(model, 'tibia_r', Vec3(0, -0.2, 0.05), Vec3(0, 0, 0.5*pi));

% Part 1c: Add IMU components to the model using the PhysicalOffsetFrames
% we just added to the model. We'll use the helper function addModelIMUs()
% included with OpenSenseUtilities.
imuFramePaths = StdVectorString();
imuFramePaths.add('/bodyset/torso/torso_imu_offset');
imuFramePaths.add('/bodyset/femur_r/femur_r_imu_offset');
imuFramePaths.add('/bodyset/tibia_r/tibia_r_imu_offset');
OpenSenseUtilities().addModelIMUs(model, imuFramePaths);
model.initSystem();

model.print(outputmodel);
fprintf('model created')

function addIMUFrame(model, bodyName, translation, orientation)

import org.opensim.modeling.*;
body = model.updBodySet().get(bodyName);
name = [char(body.getName()) '_imu_offset'];
bodyOffset = PhysicalOffsetFrame(name, body, Transform());
bodyOffset.set_translation(translation);
bodyOffset.set_orientation(orientation);
body.addComponent(bodyOffset);
model.finalizeConnections();

end

I haven't done any comparisons with real IMU signals yet but I would also be interested if anyone else has!

[Aaron Fox]

Thanks Grace.

Here I was thinking that I need some sort of creative method to do this, and there's one built in! This may be new in OpenSim 4.3 though so that could be my excuse - off to download the new version.

Aaron

[Nicholas Bianco]

Hi Grace,

Apologies for not responding to your original post, but glad that you can load the trajectory now.

You're not getting the orientation signals because orientations are of type Quaternion, but you're using the 'analyzeVec3' utility, which will only extract a TimeSeriesTableVec3 (a table of Vec3 values). We currently don't have an equivalent 'analyzeQuaternion', so you'll have to get them via the AnalyzeTool or create the table yourself.

Regarding the AnalyzeTool, if you don't specify the controls from your simulation to the model when computing accelerations, you will get different results. (This is because the model has to realize to Stage::Acceleration within the AnalyzeTool, which requires realizing to Stage::Dynamics first. In Stage::Dynamics, all the model forces are computed, include those associated with model controls.) However, you can enable the following flag to compute the correct accelerations when controls are unknown:

Code: Select all

IMUDataReporter IMUs;
// other calls to setup your IMUDataReporter here
IMUs.set_compute_accelerations_without_forces(true);
analyzeIMU.updAnalysisSet().cloneAndAppend(IMUs);

For some reason, MocoAngularVelocityTrackingGoal has not been added to scripting, so currently you won't be able to access it via Matlab or Python. Sorry about that, but thanks for pointing out the bug. We'll add it in for the next OpenSim release.

Lastly, for setting up MocoOrientationTrackingGoal, you have this line:

Code: Select all

IMUOrient.setRotationReference(rotations);

but I think you meant to use the TimeSeriesTableQuaternion 'orientSignals' here? I'm not sure what type 'rotations' is, but it's not the correct type for this goal (TimeSeriesTableQuaternion).

I have noticed the column labels for the frame paths are different and not grouped by IMU and was wondering if this was the issue and if so if there was a relatively simple method to reorder and rename these?

I'm not sure exactly what you mean here, but the order of the columns in the tracking reference doesn't matter as long as the labels are correct. But if you think you need to reorder/rename columns, you can use `table.getColumnLabels()`, `table.setColumnLabels(labels)`, `table.removeColumn(label)`, `table.appendColumn(label, columnData)`, etc.

Best,
-Nick

[Grace M]

Hi Nick,

Thanks for your help and the info!

I added in the line of code to the Analyze tool and it didn't seem to change the files/result. So I also exported the controls from the simulation and added them to the analysis and still got the same result - so hopefully what I have is correct!

Sorry I realised I was missing the line of code: rotations=OpenSenseUtilities.convertQuaternionsToRotations(orientSignals);
I was thinking based on the API docs here: https://opensim-org.github.io/opensim-m ... _goal.html
that setRotationReference needed a TimeSeriesTableRotation as the input. Can you please confirm this should be in Quaterions?

With the column headings, I figured out my issue was that the AnalyzeTool results had repeating columns/results for each IMU 3x and appending _0, _1 to the headers. I am not sure why but have managed to crop the file. I tried writing this Quateriontable using STOFileAdapterQuaternion and setting as a file reference using setRotationReferenceFile but still get an error with the OrientationTracking. I have attached the file I have been trying to use (as .txt instead of .sto)

Thanks!

[Nicholas Bianco]

Hi Grace,

Apologies, the correct input for setRotationReference is TimeSeriesTableRotation as you say, I should have double checked. However, it may be that this function is not wrapped correctly in the bindings. When you run 'methodsview MocoOrientationTrackingGoal', does 'setRotationReference()' look correct (i.e., no SWIG_type or similar looking arguments)?

The accelerations should match between these two methods. Can you show me how you modified the AnalyzeTool, both with the accelerations flag and by adding controls?

The repeating columns you are seeing are the result of a "flattened" table. The IMU outputs tables are of type TimeSeriesTableVec3, where each column contain Vec3 values. You can flatten these tables so they become type TimeSeriesTable, where every column now represents a scalar value. The suffixes "_0", "_1", etc., designate which element of the original Vec3 that column represents.

Best,
-Nick

[Grace M]

Hi Nick,

Yes MethodView showed SWIG_TYPE for setRotationReference. Is it possible to instead write the TimeSeriesTableRotation to a file and use that as the reference with setRotationReferenceFile?

This was my original code with the analyzetool

Code: Select all

predictSolution = MocoTrajectory(solnfile); 
prevStatesTable= predictSolution.exportToStatesTable(); % Extract previous solution's states
prevControlsTable = predictSolution.exportToControlsTable(); % Extract previous solution's controls

STOFileAdapter.write(prevStatesTable,[outputPath filesep solname '_states.sto']); % Write states and control table to storage file
STOFileAdapter.write(prevStatesTable,[outputPath filesep solname '_controls.sto']); 


analyzeIMU=AnalyzeTool ;
analyzeIMU.setName([solname '_analyzetool']);
analyzeIMU.setModel(torqueModel)
analyzeIMU.setStatesStorage(Storage([outputPath filesep solname '_states.sto']));
analyzeIMU.setStartTime(initialTime);
analyzeIMU.setFinalTime(finalTime);
%IMU reporter

imuDataReporter=IMUDataReporter ;
imuDataReporter.setName("IMU_DataReporter");
imuDataReporter.append_frame_paths('/bodyset/torso/torso_imu_offset');
imuDataReporter.append_frame_paths('/bodyset/femur_r/femur_r_imu_offset');
imuDataReporter.append_frame_paths('/bodyset/tibia_r/tibia_r_imu_offset');

analyzeIMU.updAnalysisSet().cloneAndAppend(imuDataReporter);%datareporter
analyzeIMU.print("analyzeReportIMUData.xml");
analyzeIMU.run();

I have then added in the line: imuDataReporter.set_compute_accelerations_without_forces(true);
and seperately: analyzeIMU.setControlsFileName([outputPath filesep solname '_controls.sto']) before updating the analysis set. Both do not seem have have changed the results which are still different from the analyzeVec3 results.

With the flattening of the table, this is not something I was doing - is it what analyzetool does? From what I can tell it is still outputing the files as a TimeseriesVec3 or Quaterions as indicated in the header, just repeating it 3x. The code below is what I have used to remove the duplicates for the orientations.

Code: Select all

%read in file
orientSignals=TimeSeriesTableQuaternion([outputPath filesep 'Run_torqueDriven_' speed 'ms_analyzetool_orientations.sto']);
%create a duplicate table
orientSignals2=TimeSeriesTableQuaternion(orientSignals.getIndependentColumn());

nRows= orientSignals.getNumRows();
nLabels = orientSignals.getNumColumns();
dataArray = NaN([nRows 3]);
labels={'torso_imu_offset_imu', 'femur_r_imu_offset_imu', 'tibia_r_imu_offset_imu'};

for iLabel = 0 : length(labels) -1 
     col_label  = char(orientSignals.getColumnLabels.get(iLabel));
    orientSignals2.appendColumn(col_label,orientSignals.getDependentColumnAtIndex(iLabel))
end
STOFileAdapterQuaternion.write(orientSignals2,[outputPath filesep 'Run_torqueDriven_' speed 'ms_analyzetool_orientations_short.sto']);

I realise I had an error in the code and file I attached previously so have attached the updated version as well as what was originally written from the analyze tool.

Thanks again!

Grace

[Grace M]

Just an update if others are interested!

With some help from Nick I have been able to add in the angular velocity tracking by copying the xml code from the OpensimXML browser into the .omoco study file and manually adding in the file/frames to track.
However loading the study in MATLAB results in this error with the contact tracking goal:

Encountered unrecognized Object typename MocoContactTrackingGoalGroup while reading property contact_groups. There is no registered Object of this type;

The solver still runs but the objective_contactGoal=0 in the solution file so I don't think it is being used.
I was able to add the contact tracking after I had loaded the study as a workaround.

With the MocoOrientationGoal, I have been trying to instead use setStateReference to track the orientation of the IMU frames as states.
I am a bit confused how to get the states of the IMU frames since they don't seem to be written in the solution. - is this using an OutputPath? I have tried various variations of things like
outputPaths.add('/bodyset/torso/torso_imu_offset|orientation' ) or '/bodyset/torso/torso_imu_offset|value' but this doesn't output anything.

Thanks