Inconsistent kinematics results between trials

[Jasper Verheul]

Hi OpenCap team,

We are currently collecting running kinematics data with OpenCap for treadmill running. We have noticed that when we collect multiple trials at the same running speed and for the same participant, using the exact same setup, there can be very different outcomes (either successful or unsuccessful kinematic results) between trials. We use v0.3 with the OpenPose model, sample at 240 Hz, and use three iPads. We also use the recommended arm punch.

A good example would be session "ef1f15e9-d0bd-4dd4-86fe-9c1ae6e99e3a" (account name: cmu-opencap) for which at each speed there is one good trial and one trial for which the results are very glitchy. Likewise, for other sessions we get a wide range of good and poor kinematic outcomes between trials, using the exact same setup.

Is there anything we can do to avoid this inconsistency between trials? Moreover, is there anything we can do to improve the trials that have resulted in poor kinematics?

Many thanks
Jasper

[Matt Petrucci]

Hi Jasper,

Here's a few things to try:

1. Use HRNet vs. OpenPose for the pose estimation. It seems to perform better for these more dynamics tasks. For these trials that you have already collected, you could reprocess with HRNet using this code: https://github.com/stanfordnmbl/opencap ... ons.py#L23. There is also a helpful YouTube tutorial here: https://www.youtube.com/watch?v=Mk3KPo79_eQ&t=509s
2. Although OpenCap will track the tallest person in the video, you want to avoid having people in the background if possible. I can't tell for sure, but the pose estimation may be getting a little bit mixed up when the runners feet are closer to the person sitting behind on the computer. You could even occlude that person with a box or curtain.
3. You could also consider moving your iPad to the right of the participant to reduce the occlusions of the legs from the cross bars. If you widen the camera angle a bit, it will probably help.
4. You can also move the front facing camera to be slightly off center (so not directly in from the person). This just helps with depth perception.

Hope these help!

[Jasper Verheul]

Hi Matt,

Thanks for your reply - that is very helpful!

We will try those additional suggestions for data collection. Following from point 1, can you confirm that it is possible to reprocess trials that have been collected with OpenPose settings, using HRNet locally? The code seems to suggest this is not possible?

I have one additional question on the scaling as this may or may not affect these results. When investigating the scaled models for each participant we noticed that some segments are scaled separately for the left/right side and others are not. For example, the scaled lengths of the foot and femur/thigh segments are identical between left and right, whereas the tibia/shank segment is scaled slightly different between both legs (see figure). Does OpenCap scale certain segments identical between the left and right side, and others separate? If so, is there a particular reason for this?

Many thanks
Jasper

[Antoine Falisse]

Hi Jasper,

1. It is not supported out of the box. I believe HRNet through MMPose is now supported on windows (it was not when we developed this code), but we have no instructions for using it. You could reprocess with OpenPose, but using a higher resolution: https://github.com/stanfordnmbl/opencap ... y#L84-L100

2. This is interesting. The scaling setup is here: https://github.com/stanfordnmbl/opencap ... rms_KA.xml. We use the same scaling for both feet, but not for both tibias and femurs. Do you get the exact same scale factors for left and right femurs? You can check that in the xml file of the scaled opensim model.

Some other thoughts:

3. A quick look at the data makes me think it is a camera synchronization problem. I see that some of the bad trials start with the hand in the air. I should double check but the arm punch method expects the participant to start the punch from below the shoulder. The idea is to have some velocity signal there, if you start at the top there is no signal. Check this video around 10min: https://www.youtube.com/watch?v=7Xk8-K7A3Zk

4. We pushed a new synchronization algorithm a couple of weeks ago. If you reprocess your trials locally with the latest code, you might get better results.

Best,
Antoine

[Jasper Verheul]

Hi Antoine,

Thanks for your suggestions.

1. I have tried to reprocess with OpenPose at a higher resolution. However, the attached error occurs. The session metadata has been changed and posted fine. For example, I tried this for session 'dc9982b5-ff06-4735-adcd-ea5bc9966fe9' and trial 'P22_16kph'. The trial was previously processed green with HRNet (which resulted in some severe glitches) but is now red in the web application. Do you have any suggestions for how to overcome the attached error?

2. The scale factors in the scaled models are different for the left/right femur and tibia. I have discovered an error in the definition of the femur length and noticed that left and right are indeed scaled independently so this issue seems resolved. Can you confirm that both feet should have identical scaled lengths? The scaling factors for left/right are the same in the scaled model file.

3. We indeed try to follow these instructions during data collections. Participants sometimes find it challenging to punch their arm when running on the treadmill at slightly faster speeds. We have, however, noticed that good synchronisation can occur even without an arm punch, and poor synchronisation still occurs when a 'proper' arm punch is used.

4. It would be great to try this out when the error in 1. is resolved.

Many thanks
Jasper

[Jasper Verheul]

Hi Antoine,

A different error occurs (see attached) when trying to reprocess trials that have originally been collected with OpenPose settings.

Regards
Jasper

[Antoine Falisse]

hi,

Do you have a machine with a good enough GPU to reprocess data at high res?
https://github.com/stanfordnmbl/opencap ... py#L97-L99

Antoine

[Brandon Schoenwether]

Hello,

I am also interested in trial reprocessing using HRnet, specifically making sure the most current marker augmenter model is being applied to the 3D triangulation after pose estimation per the most recent publication, https://doi.org/10.1101/2024.07.13.603382

Any code associated with the solution would be appreciated as the current resources do not detail how this is accomplished.

Brandon