import cv2 #Notes

triangulatePoints

X = cv2.triangulatePoints(P0, P1, x0, x1)

• P0.shape = (3,4): Camera projection from X to x0
• P1.shape = (3,4): Camera projection from X to x1
• x0.shape = (2,-1): 2D points
• x1.shape = (2,-1): 2D points
• X.shape = (4,-1): 3D points

Important: Divide X by X[3] to make it homogeneous.

X = cv2.triangulatePoints(P0, P1, x0, x1)
X /= X[3]  #(wx,wy,wz,w) -> (x,y,z,1)

———————————————————

Rodrigues

R3,J = cv2.Rodrigues(R3x3)

R3x3,J = cv2.Rodrigues(R3)

• R3.shape = (3,1) or (3,) or (1,3): Rotation vector (radians)
• R3x3.shape = (3,3): Rotation matrix
• J.shape = (9,3) or (3,9): Jacobian

Important: When applied twice (R3 -> R3x3 -> R3), the resulting rotation vector might look different if the values are greater than pi/2, but it represents the same rotation.

R = r_[1.,2.,1.]
R = cv2.Rodrigues(R)[0]  #I only use the first return value

———————————————————

decomposeProjectionMatrix

R,t = cv2.decomposeProjectionMatrix( P )[1:3]

• P.shape = (3,4): Camera projection matrix { x = dot(P,X) }
• R.shape = (3,3): Rotation matrix
• t.shape = (4,1): 3D Camera position in world coordinates

Important: Divide t by t[3] to make it homogeneous. This ‘t‘ is not the one in P = [R|t]. It is the one in P = [R | R(-t)]

R,t = cv2.decomposeProjectionMatrix( P )[1:3]
t /= t[3] #Position of camera. (wx,wy,wz,w) -> (x,y,z,1)

Recompose the projection matrix like this:

P = append(R, dot(R,-t[:3]),1)

The ‘t‘ is negative because world points are shifted in the opposite direction of the camera position.
———————————————————

correctMatches

x0c, x1c = cv2.correctMatches(F, x0, x1)

• F.shape = (3,3): Fundamental or essential matrix
• x0.shape = (1,-1,2): 2D points from 1st image
• x1.shape = (1,-1,2): 2D points from 2nd image
• x0c.shape = (1,-1,2): Optimized 2D points
• x1c.shape = (1,-1,2): Optimized 2D points

Important: The point format is different than other cv2 methods.

# x0 and x1 have shape 3xN. Convert them inline to 1xNx2
x0c, x1c = cv2.correctMatches( F, r_[[x0[:2].T]], r_[[x1[:2].T]] )
# Convert the result from 1xNx2 to 3xN.
x0c = append(x0c[0].T,ones((1,len(x0c[0]))),0)

———————————————————

solvePnP

r,t = cv2.solvePnP( X, x, K, distCoeffs)[1:3]

• X.shape = (-1,3) or (1,-1,3): 3D points (origin coordinates)
• x.shape = (-1,2) or (1,-1,2): 2D image location 
• K.shape = (3,3): Use eye(3) for normalized points
• distCoeffs = None: 'None' or array of 4, 5 or 8 values
• r.shape = (3,1): Rotation vector (radians)
• t.shape = (3,1): Translation vector, the 't' in [R|t]

Important: This ‘t‘ is not the camera position in world coordinates but the actual last column of the camera projection matrix. Just append ‘t‘ to the 3×3 R matrix.

r,t = cv2.solvePnP(X[:3].T, x[:2].T, eye(3), None)[1:3]
# Compose the camera projection. Check it with x = dot(P,X)
P = append( cv2.Rodrigues( r )[0], t, 1 )

———————————————————

convertPointsFromHomogeneous

x = cv2.convertPointsFromHomogeneous( xh )

• xh.shape = (-1,1,3) or (1,-1,3): (wx,wy,w) or (wx,wy,wz,w)
• x.shape = (-1,1,2): (x,y) or (x,y,z)

Important: This cv2 method is unnecessary but it was difficult to figure out how to use it, so maybe this will save you some frustration.

# xh is a 3xN or 4xN array of points, x is 2xN or 3xN respectively
x = cv2.convertPointsFromHomogeneous(r_[[xh.T]].astype(float32))
# Or simply do this instead
x = xh[:-1]/xh[-1]

———————————————————

convertPointsToHomogeneous

xh = cv2.convertPointsToHomogeneous( x )

• x.shape = (-1,1,2) or (1,-1,2): (x,y) or (x,y,z)
• x.shape = (-1,1,3): (x,y,1) or (x,y,z,1)

Important: This cv2 method is also unnecessary but it was difficult to figure out how to use it, so maybe this will save you some frustration.

# xh is a 3xN or 4xN array of points, x is 2xN or 3xN respectively
xh = cv2.convertPointsToHomogeneous(r_[[x.T]].astype(float32))
# Or do this instead
xh = append(x,ones((1,len(x[0]))),0)

Leave a comment