OpenCap pipeline locally

[Carlos Gonçalves]

Hello everyone.

I will start studying OpenCap capabilities to evaluate jump performance. I was really interested in the option below, found in https://github.com/stanfordnmbl/opencap-core.

Run this pipeline locally using videos collected near-synchronously from another source (e.g., videos collected synchronously with marker-based motion capture). Easy-to-use utilities for this pipeline are under development and will be released soon.

Are there some guidelines for this type of use of OpenCap? Is there a piece of code with some steps like:

- Import videos for calibration (checked box)
- Import motion videos
- Subject calibration
- motion
- Run OpenPose to collect the skeleton
- Run OpenCap to estimate 3D markers
- Run OpenCap to solve the motion

Best regards.

[Scott Uhlrich]

Hi Carlos,

This is all possible, but requires some re-organization of paths, etc. We plan to release some nice code for this in the next few months.

Here's a start to what code you would need.

- Intrinsic calibration for whatever cameras you are using. https://github.com/stanfordnmbl/opencap ... rinsics.py
- All computations from extrinsic camera calibration through kinematics happen in https://github.com/stanfordnmbl/opencap ... in/main.py
- This is an example to show you how you'd do this with many subjects in batch. You'll need to organize your folders appropriately first. https://github.com/stanfordnmbl/opencap ... ematics.py

[Carlos Gonçalves]

Thanks a lot, Scott.

I will start my way on your post. This is really helpful.

Best regards and congratulations to you and the team on the project.

[Logan Wade]

Hi Carlos,

This is all possible, but requires some re-organization of paths, etc. We plan to release some nice code for this in the next few months.

Here's a start to what code you would need.

- Intrinsic calibration for whatever cameras you are using. https://github.com/stanfordnmbl/opencap ... rinsics.py
- All computations from extrinsic camera calibration through kinematics happen in https://github.com/stanfordnmbl/opencap ... in/main.py
- This is an example to show you how you'd do this with many subjects in batch. You'll need to organize your folders appropriately first. https://github.com/stanfordnmbl/opencap ... ematics.py

Hi Scott and Antoine,

I was just wondering if you did end up releasing some code for this as Scott mentioned above.

Much appreciated!!

Logan

[Antoine Falisse]

Hi Logan,

Not yet unfortunately.

Best,
Antoine

[Corey Np]

Hi Antoine,

We're really keen on testing this locally on our computer for longer/higher res recordings. I'm sure there's a lot of other developments that are keeping your team busy, but any ballpark idea for when this code will be released?

Cheers,
Corey

[Scott Uhlrich]

Hi Corey,

We don't have an estimated timeline unfortunately.

Thanks,
Scott

[Sasha Portnova]

Hi,

I just wanted to check if the code for running the OpenCap pipeline on non-OpenCap collected videos have been released in the last year.

[Matt Petrucci]

Hi Sasha,

Please see this post: viewtopicPhpbb.php?f=2385&t=19035&p=0&s ... 75b97e96c7

[Carlos Gonçalves]

Hello, everyone; I made some progress in trying to set up the pipeline to run locally.

I'm just focusing on using the already available "cameraIntrinsicsExtrinsics.pickle" in the downloaded data from OpenCap, and redoing a pose estimation (creating the TRC file with augmented markers).

So far I'm stopped at configuring tensorflow and the GPU and cudatoolkit. But I already have some pre-augmented maker file. But they are weird.

I don't know if this is the right place to discuss this subject, but I would like some help with it.

The markers in the file are in 3D format, but they behave like a 2D camera. When the person is far away, the markers are closer, and when the person is closer, the distance between the markers is greater.

dist1.png (45.66 KiB) Viewed 16 times

dist2.png (60.89 KiB) Viewed 16 times

I just want to know if this is the expected behavior, or if the extrinsics configurations are faulty. Any thoughts on that?

Best regards.