Pinhole Camera

A pinhole camera has no lens: light passes through a single small aperture and forms an inverted image on a photographic plate behind it (PHD §2.1, fig. 2.1). Equivalently, and more conveniently, one can place a virtual retinal plane in front of the projection center and reason with the (non-inverted) image formed there. Ignoring lens effects — most notably radial distortion, which bends straight lines near the image border and is not modeled by PinholeCamera at all (see the aside below) — this projection is exactly linear once points are expressed in homogeneous coordinates, which is the whole reason projective geometry is the natural language for it.

Put the camera center at the origin, pointing down the Z axis, with the retinal plane at . A 3D point projects to (PHD eq. 2.1). In homogeneous coordinates this inhomogeneous division disappears and the mapping becomes linear:

which is exactly the canonical camera ] (PHD eq. 2.2-2.3; HZ §6.1, "the basic pinhole model"). Every other camera in this library — however it was constructed or estimated — is this same linear idea generalized with a focal length, a principal-point offset, a pixel/skew correction, and an arbitrary position and orientation in the world. That generalization is what the rest of this page unpacks.

A PinholeCamera is stored as one 3×4 projective matrix P (not separately as K, R, C). Starting from the canonical camera above, adding a focal length , a principal point offset , a pixel/skew correction (all folded into a single upper-triangular calibration matrix ), and a world-to-camera rigid motion (rotation , camera center ) gives the classic factorization

(PHD eq. 2.22-2.24; HZ eq. 6.7-6.8). Writing ] with the top-left 3×3 block, a camera is called finite when  — this is the case the rest of this page mostly addresses — and a camera at infinity otherwise, whose center lies on the plane at infinity (PHD §2.3; HZ §6.2.1/6.3).

The factorization above is recovered lazily, on demand, via decompose():

The decomposed PinholeCameraIntrinsicParameters, Rotation3D and Point3D (center) are cached and are only recomputed the next time they are requested after any setter (setInternalMatrix, setIntrinsicParameters, setCameraRotation, …​) invalidates the cache; areIntrinsicParametersAvailable(), isCameraRotationAvailable() and isCameraCenterAvailable() report whether a cached value already exists without forcing a decomposition.

PinholeCameraIntrinsicParameters wraps a 3×3 upper-triangular calibration matrix K:

This matrix is built up in exactly three steps, each adding one real-world effect (PHD §2.2.2):

getAspectRatio() returns fy/fx; setAspectRatioKeepingHorizontalFocalLength(double) / …​KeepingVerticalFocalLength(double) adjust it while pinning one focal length. getSkewnessAngle() / setSkewnessAngle(double) expose the skew as the angle from the formula above, rather than the raw coefficient s.

Three factories cover the common cases. createCanonicalIntrinsicParameters() returns the identity K (the bare canonical camera derived above). createTypicalIntrinsicParameters(width, height) centers the principal point on the image and picks a focal length equal to (width + height) / 2, roughly a 45° field of view — a reasonable guess when nothing else is known. create(focalLength, sensorWidth, sensorHeight, imageWidth, imageHeight) performs exactly the CCD-pixel conversion of step 3 above, in physical units (millimeters):

1 / pixelWidth and 1 / pixelHeight are precisely and above, so focalLength / pixelWidth  — this is PHD eq. 2.16 read directly off a datasheet’s focal length and sensor size. Skew and principal-point offset are assumed zero here, which is the common case for real CCD/CMOS sensors. getInverseInternalMatrix() computes with a hand-derived closed form from fx, fy, s and the principal point, rather than a generic matrix inversion, purely for efficiency: K is triangular and small enough that this is both faster and better conditioned than Utils.inverse.

Given only P, recovering K and R from the top-left 3×3 block Mp = K·R is exactly the problem an RQ decomposition solves: factor an arbitrary square matrix into an upper-triangular matrix times an orthogonal one, (note the unfortunate name clash — the "R" in "RQ" is unrelated to the camera’s rotation matrix, which happens to be the "Q" here). RQ decomposition is not unique up to sign/scale, so the extra constraints that K’s diagonal be positive and `R be a proper rotation ( , not just orthogonal) pin down a unique answer, up to the camera matrix’s own overall scale ambiguity (PHD §2.3.6, eq. 2.74-2.78).

com.irurueta.algebra.RQDecomposer (in the sibling module irurueta-algebra) implements this with Givens rotations, following HZ's Algorithm A4.1: three successive rotations about the coordinate axes, each chosen to zero one below-diagonal entry of the matrix without disturbing the entries already cleared, leaving an upper-triangular R times a rotation Q (HZ Appendix §A4.1.1, p. 579). This is the same family of factorizations as the more familiar QR decomposition via Householder reflections (Numerical Recipes §2.10, pp. 102-106) — RQDecomposer internally wraps a QRDecomposer rather than reimplementing Givens rotations directly.

PinholeCamera.computeIntrinsicsAndRotation() uses it exactly like this:

The rest of the method (not reproduced in full) multiplies r and q by a suitable so that r’s diagonal ends up strictly positive and `q ends up a proper rotation — exactly the sign-fixing recipe of PHD eq. 2.76-2.78 — before wrapping the results as PinholeCameraIntrinsicParameters and MatrixRotation3D.

The camera center C is the unique 4-vector (up to scale) with  — the right null-space of P. Geometrically, any 3D line through C and another point M is a ray of light: every point on it projects to the same image point (the projection of M), so the "extra" degree of freedom along the ray must vanish when multiplied by P, which is exactly (PHD §2.3.1, Proof 2.1; HZ §6.2.1, "camera centre"). Because P is rank 3 with 4 columns, this null-space is always exactly one-dimensional — the camera always has a well-defined center, even when it lies at infinity (an "affine" or degenerate camera, ).

That single geometric fact can be turned into a concrete number in three different ways, and PinholeCamera ships all three because they trade off cost against generality:

computeCameraCenterSVD() (the default used by decompose()). Any null-space is most robustly found via singular value decomposition: writing P’s SVD as , the column of `V corresponding to P’s zero singular value is exactly the null-space vector (PHD §2.3.1.1; HZ §6.2.4, "Finding the camera centre", citing its own Appendix §A4.4 for SVD; general SVD algorithm in Numerical Recipes §2.6, pp. 65-67). It remains correct even when the true center is at infinity (rank-deficient `Mp) since the SVD makes no finiteness assumption — at the cost of being the most expensive of the three:

computeCameraCenterDet(). The same null-space can be obtained algebraically, without a full SVD, from the determinants of `P’s four 3×3 minors (each formed by dropping one column):

This follows from the fact that the camera center lies on all three of the camera’s row-planes (see below), so appending any other plane through it to a 3×4 stack yields a rank-deficient, zero-determinant matrix; expanding that determinant by Laplace’s formula reproduces exactly this cofactor expression (PHD §2.3.1.2, Proof 2.2). It is cheaper than SVD (four 3×3 determinants via Utils.det, instead of a full decomposition of the whole 3×4 matrix) and, like the SVD method, remains valid even at infinity. The real method repeats the same four lines (build a 3×3 minor by copying three of the four columns, take its determinant) once per coordinate, with alternating sign (abbreviated here for brevity):

computeCameraCenterFiniteCamera(). If the camera is known to be finite ( invertible), the center follows directly from (PHD §2.3.1.3, eq. 2.45-2.46; HZ §6.2.4, closed form ). This is the cheapest of the three (one 3×3 matrix inverse instead of a determinant per column or a full SVD) but the only one of the three that requires a non-degenerate, finite camera — it will fail or be numerically unstable if Mp is singular or ill-conditioned:

In short: use the closed form when the camera is known to be finite and speed matters; fall back to the cofactor method for a cheaper alternative that still tolerates a center at infinity; use SVD (the default) when robustness matters more than speed, or when nothing is known about the camera’s degeneracy in advance.

Several classic projective-camera facts fall directly out of P’s own columns and rows, without needing a full `decompose():

Because P is only defined up to an arbitrary non-zero scale, and describe the exact same camera geometry — yet only one of the two sign choices consistently places 3D points "in front of" rather than "behind" the lens. Resolving that ambiguity is what camera sign and cheirality are for.

getCameraSign() / getCameraSign(double threshold) return the sign of . This works because scaling P by any scales by and the third row of Mp by k, so the vector is invariant to regardless of the sign of k — it always points toward the front of the camera (PHD §2.3.5, eq. 2.65-2.72; HZ §6.2.1, principal axis vector). fixCameraSign() multiplies the whole matrix by that sign once, and isCameraSignFixed() reports whether that has already happened (any setter resets the flag).

getDepth(Point3D point) computes the signed distance of a 3D point from the principal plane, along the principal axis:

(PHD §2.4.1, eq. 2.79-2.82; HZ Result 6.1, eq. 6.15). In code this is exactly the dot product of (point - cameraCenter) with the already sign-fixed, unit-length principalAxisArray():

getCheirality(Point3D point) answers the cheaper yes/no question — is the point in front of, or behind, the camera — without computing an actual distance. It reduces to the sign of the last homogeneous coordinate of the projected point times the last homogeneous coordinate of the 3D point, (PHD §2.4.2, eq. 2.83-2.96), corrected by the camera sign whenever it has not already been fixed:

Because lines/planes and points transform contragradiently (see Transformations), the camera’s project/back-project operations use different formulas depending on what is being mapped:

A camera’s images are unaffected by which projective/affine/metric stratum the world points and the camera are expressed in, as long as both are transformed consistently: if , then , i.e. any 3D transformation induces on the camera (PHD §2.8, eq. 2.153-2.154; see Transformations for how Transformation3D acts on a PinholeCamera). Composing this with the dual-quadric projection formula above and specializing to the dual absolute quadric gives the dual image of the absolute conic (DIAC), , which in the metric stratum reduces to  — i.e. K is recoverable from by Cholesky factorization (PHD §2.8, eq. 2.158-2.160; HZ §8.5, Result 8.17, ). This is the theoretical basis for camera self-calibration and for recovering the metric stratum from a purely projective reconstruction.

This library does not, however, ship a ready-made self-calibration/DIAC-estimator class — there is no Quadric.createCanonicalAbsoluteQuadric() or Kruppa-equation solver in com.irurueta.geometry.estimators or com.irurueta.geometry.refiners; only DualQuadric.createCanonicalDualAbsoluteQuadric() exists as a building block. The EPnPPointCorrespondencePinholeCameraEstimator (Lepetit, Moreno-Noguer and Fua) remains the only estimator this page cites directly. See Conics and Quadrics for the absolute conic/quadric themselves, and Transformations for the stratification of 3D transformations.

The classes exercised by the code examples above, with links to their source and Javadoc: