Coordinate Conventions, or: Why Your Transform Is Inverted

Almost every multi-sensor bug I have chased came down to one of three things: a transform applied in the wrong direction, a quaternion whose components were in the wrong order, or two libraries that disagreed about which way $+Y$ points. None of these are hard problems. They are bookkeeping problems, and they survive because the notation most people use is ambiguous.

This is the convention I use and the table I keep open.

Say what you mean: “pose of” versus “transform from”

The sentence “the camera transform” has two readings, and they are inverses of each other:

• the pose of the camera, expressed in the world frame
• the transform from world coordinates into camera coordinates

Both are described by the same six numbers, and they are not the same six numbers.

So don’t say “the camera transform.” Write $T_{W \leftarrow C}$ — read as “world from camera” — for the rigid transform that takes a point expressed in camera coordinates and re-expresses it in world coordinates:

The payoff is that chaining becomes mechanical — adjacent frames cancel:

In code I name the variable after the subscripts, T_world_cam, and then a chain is correct exactly when the inner names match:

T_world_cam = T_world_base @ T_base_cam    # base cancels -> world_cam. Correct.
T_world_cam = T_world_base @ T_cam_base    # base vs cam. Wrong, and it is now obvious.

That single naming rule has caught more bugs for me than any amount of careful thinking. Note also that $T_{W \leftarrow C}$ is the pose of the camera in the world — the two readings above are $T_{W \leftarrow C}$ and $T_{C \leftarrow W}$ respectively.

Written out, with the two operations you actually need:

import numpy as np
from scipy.spatial.transform import Rotation

def make_T(R, t):
    T = np.eye(4)
    T[:3, :3], T[:3, 3] = R, t
    return T

def invert_T(T):
    R, t = T[:3, :3], T[:3, 3]
    return make_T(R.T, -R.T @ t)          # NOT just R.T -- the -R^T t matters

# scipy's from_quat is scalar-LAST: (x, y, z, w)
R_world_base = Rotation.from_quat([0.0, 0.0, 0.3827, 0.9239]).as_matrix()   # 45 deg yaw
T_world_base = make_T(R_world_base, np.array([10.0, 4.0, 0.0]))
T_base_cam   = make_T(np.eye(3), np.array([1.5, 0.0, 1.2]))

T_world_cam = T_world_base @ T_base_cam   # base cancels
cam_position_in_world = T_world_cam[:3, 3]

And the inverse is not the transpose:

Only the rotation block transposes. Forgetting the $-R^{\top}t$ gives you a pose with the right orientation in a plausible-but-wrong location, which is the hardest kind of wrong to notice.

Axis conventions

All of these are right-handed. They still disagree.

Two traps here.

The OpenCV↔OpenGL flip. Going between a reconstruction pipeline and a renderer means flipping $Y$ and $Z$. If your point cloud renders upside down and inside out, this is why.

ROS has two camera frames on purpose. REP‑103 specifies that camera_link follows the body convention (x forward, y left, z up), while camera_link_optical — anything with the _optical_frame suffix — uses the vision convention (z forward, y down). They describe the same physical camera. Publishing your OpenCV-derived extrinsics onto the non-optical frame is a very common and very confusing mistake, because everything is almost right.

For world frames, REP‑103 specifies ENU (east-north-up), not the NED that aviation and many IMU datasheets use. That is another $Y$/$Z$ flip waiting to happen at the driver boundary.

Quaternion component order

Same four numbers, three orderings in wide use:

Eigen is its own worst enemy here: the constructor takes $w$ first, but .coeffs() returns it last. Passing q.coeffs().data() to something expecting scalar-first gives you a rotation that is wrong in a way that still looks like a valid rotation.

The cheap defence is a unit test with a known non-symmetric rotation — 90° about $Z$, say — carried through every serialization boundary in your stack. Symmetric test cases like 180° rotations will happily pass with the components scrambled.

Library-specific notes

OpenCV. solvePnP returns rvec, tvec satisfying $p^{C} = R\,p^{W} + t$. That is $T_{C \leftarrow W}$ — world from camera is the inverse of what you got back. If you want the camera’s position in world coordinates it is $-R^{\top}t$, not tvec. cv2.Rodrigues converts between the 3-vector rvec and the matrix; the vector’s direction is the axis and its norm is the angle in radians.

ROS 2 / tf2.

The tree is a strict tree: every frame has exactly one parent, no cycles. REP‑105 fixes the chain as earth → map → odom → base_link → sensor. The split between map and odom is the useful part of that design — odom is continuous but drifts, map is drift-free but jumps when localization corrects. Controllers consume odom because a discontinuity is worse than slow drift; planners consume map.

lookupTransform(target, source, time) returns $T_{\text{target} \leftarrow \text{source}}$, which composes in the order you would hope. Note the argument order is the opposite of the reading order in the name.

GTSAM. Pose3 is the pose of a body in the world, i.e. $T_{W \leftarrow B}$. pose.transformFrom(p) takes a body-frame point to world; pose.transformTo(p) goes the other way. A BetweenFactor<Pose3> between poses $i$ and $j$ measures $T_i^{-1} T_j$, the relative pose expressed in frame $i$ — so odometry goes in directly, but only if your odometry is also expressed in the previous body frame rather than the world frame. GTSAM’s PinholeCamera uses the OpenCV optical convention, which makes it easy to mix with cv2 output as long as you keep the pose direction straight.

A checklist

When a transform looks wrong, in order of how often it turns out to be the cause:

1. Direction. Try the inverse. It is right about half the time, and it costs one line.
2. Component order at a serialization boundary — quaternion, or Euler angle sequence.
3. Axis convention — optical versus body, ENU versus NED.
4. Point versus vector. A direction rotates but must not translate. Homogeneous $w=0$ for directions, $w=1$ for points; using $w=1$ on a velocity or a surface normal adds the translation to it.
5. Timestamps. On a moving platform, the right transform at the wrong time is the wrong transform. At 20 m/s, 50 ms is a metre.

The last one is the one that keeps working in the parking lot and fails on the highway, which makes it worth more scrutiny than its position on this list suggests.