Page 1 of 1
slow processing speed
Posted: Sat Mar 28, 2020 5:22 pm
by kernalnet
Dear Experts
In MATLAB, two of your functions bother me due to the low processing pace.
1) "c3d.getAsStructs" is very slow especially for large c3d files. Moreover, although "c3d.getTable" is quite fast, again "osimTableToStruct" is very slow too. In these cases, I prefer to use BTK packages instead of osimC3D functions .
2) I use "osimTableToStruct" function after "TimeSeriesTable" for importing ['.sto' '.mot' '.trc'] files into MATLAB. But again this function suffers from a slow pace. In this case, I prefer to import those files as a text or ASCII format and this way is much faster.
I wonder if you would tell me whether I use the correct functions for those purposes.
Regards
Re: slow processing speed
Posted: Mon Mar 30, 2020 10:07 am
by aymanh
Hello Mohammad,
It is true that these Matlab utilities take a long time due to the fact that they need to manipulate every single entry in these large files to copy them into Matlab structures, doing the same processing entirely using the OpenSim API operations that act on full rows or columns or entirely on the OpenSim side would be more efficient. Please indicate what specific needs you use these utilities for so we can advise you of alternatives or give suggestions.
Best regards,
-Ayman
Re: slow processing speed
Posted: Mon Mar 30, 2020 1:06 pm
by kernalnet
Hi Aymen,
Thanks for your reply.
I just want to access to the data (c3d sto mot) and it's not matter what type it is (double, struct, cell, table). Indeed I want to do both pre and post data processing e.g filtering, interpolation, normalization, extracting different variables, plots and etc.
Would you please give me a guidance for converting the OpenSim tables (TimeSeriesTable and TimeSeriesTableVec3) to any accessible data in MATLAB in a faster way? (I prefer a MATLAB double and a cell for labels.)
I'm greatly grateful for your help
Regards
Re: slow processing speed
Posted: Tue Mar 31, 2020 1:22 am
by kernalnet
Dear Aymen,
I also tried to create a double. Although it is a little better than before,but it is still slow (especially compared to BTK package).
Code: Select all
mTable = c3d.getTable_markers();
mNum = mTable.getNumColumns();
mFrame = mTable.getNumRows();
for i = 0 : mNum - 1
for j = 0 : mFrame -1
mArray(j+1,3*(i+1)-2) = mTable.getDependentColumnAtIndex(i).get(j).get(0);
mArray(j+1,3*(i+1)-1) = mTable.getDependentColumnAtIndex(i).get(j).get(1);
mArray(j+1,3*(i+1)) = mTable.getDependentColumnAtIndex(i).get(j).get(2);
end
mLabel{i+1}= char(mTable.getColumnLabels.get(i));
end
I wonder if you would give me a faster way.
Regards
Re: slow processing speed
Posted: Tue Mar 31, 2020 9:37 am
by aymanh
Dear Mohammad,
You're doing exactly the same as the matlab utility we provide which makes a copy of all the data from huge data files to Matlab structures and that's the time consuming part.
My suggestion would be to list the operations that you want performed on the TimeSeriesTable and see if you can have OpenSim perform them for you, this will help us build up the most commonly needed functionality.
For example, if you want to average the TimeSeriesTable between two times, you can now call
which performs all the math on the OpenSim side without needing to make a copy. We may have more utilities already to operate on these tables but not as extensive as Matlab of course.
Also if you can operate on OpenSim Vec3s rather than doubles that could be 3-4 times faster as it avoid indexing calls altogether.
Hope this helps,
-Ayman