I have some inverse kinematics results from a markerless motion capture system and would like to overlay them on top of the original video . However, I am having some difficulty getting the images to line up correctly. I have been trying to use the extrinsic (R) matrix and translation vector (T) from the camera calibration along with the "osimModel.updVisualizer().updSimbodyVisualizer().setCameraTransform(transform)" command.
I defined the "transform" parameter as:
"transform=osim.Transform(rot,vec)"
where "vec" is the translation of the camera and "rot" is a rotation matrix describing the camera orientation.
I calculated vec as "vec= -inv(R) * T" (where the * is matrix multiplication)
In my case:
R=[[0.243, -0.00722,0.969],
[-0.583, -0.799, 0.140],
[0.774, -0.600, -0.198]]
T= [0.0638, 0.115, 0.681]
so "vec=osim.Vec3(-0.476453, 0.501751, 0.0574492)"
My main question is how should "rot" be defined? My understanding is that by default the camera points in the -z direction (0, 0, -1). So "rot" should be defined as the rotation matrix between (0, 0 , -1) and the vector describing the direction the camera is pointing?
Based on the answers on this post: https://math.stackexchange.com/question ... 4x4-matrix, I calculated the camera direction as the third column of inv(R).
so the camera direction (inv(R)[0, 0, 1]) would be: [0.774, -0.600, -0.198]
and "rot" would be the rotation matrix between [0, 0, -1] and [0.774, -0.600, -0.198]
rot=osim.Rotation(osim.Mat33(0.499, 0.387, -0.774
0.387, 0.699, 0.600255,
0.774, -0.600255, 0.198782))
When I used these values of "rot" and "vec" this was the resulting overlay:
- new0.png (578.64 KiB) Viewed 187 times
I have also just tried setting rot=R and rot=inv(R) and the fits were worse than the image attached
Thanks for your help!
Ty