Overview

This documentation was generated with the assistance of AI. Please report any inaccuracies.

irurueta-ar is a Java library for augmented reality and 3D reconstruction. Given point correspondences observed across multiple images of a scene taken from a moving camera, it estimates the cameras' geometry and recovers the 3D structure of the scene, optionally fusing inertial sensor data (accelerometer/gyroscope) for scale and orientation estimation.

Why it exists

Reconstructing a 3D scene from ordinary photographs or video frames requires solving several interlocking geometric estimation problems: finding the epipolar geometry between image pairs, recovering camera calibration and pose, and triangulating 3D points, all robustly in the presence of noisy or outlier-ridden point matches. irurueta-ar packages these building blocks, along with a ready-to-use Structure-from-Motion (SfM) pipeline, so that consumers don’t have to implement multi-view geometry and robust estimation from scratch.

Core concepts

  • Estimation (Robust vs. non-robust estimation) — the shared robust/non-robust estimator pattern (RANSAC, LMedS, MSAC, PROSAC, PROMedS) used by every algorithm family below.

  • Epipolar geometry (com.irurueta.ar.epipolar) — the fundamental matrix (7-/8-point algorithms) and the essential matrix describing the geometric relationship between a pair of camera views, plus point correction (Sampson, Gold Standard) for accurate triangulation.

  • Calibration (com.irurueta.ar.calibration) — self-calibration via the image/dual absolute conic, dual absolute quadric, and Kruppa equations, plus pattern-based calibration and radial distortion estimation.

  • Structure from Motion (Structure from Motion, com.irurueta.ar.sfm) — the SparseReconstructor family (and its TwoViews, PairedViews, and SLAM-fused variants) drives the end-to-end pipeline from matched 2D points to reconstructed 3D points and cameras, upgrading from a projective to a metric stratum via self-calibration when camera intrinsics are unknown.

  • SLAM (com.irurueta.ar.slam) — Kalman-filter based estimators that predict a device’s position, velocity, acceleration, and orientation from accelerometer and gyroscope readings, which can be fused into the SfM pipeline to resolve absolute scale.

The reconstruction pipeline

At the core of the library is a Structure-from-Motion pipeline that turns matched 2D image points into reconstructed 3D points and camera poses:

flowchart LR A["Matched 2D point
correspondences"] --> B["Fundamental matrix
estimation (robust)"] B --> C["Point correction"] C --> D1["K known:
Essential matrix"] C --> D2["K unknown:
projective camera pair"] D1 --> F["Triangulation"] D2 --> E["Self-calibration
(DAQ / Kruppa)"] E --> F F --> G["Reconstructed 3D points
& cameras, up to scale"] H["Accelerometer /
gyroscope data"] -.SLAM fusion.-> G

See Robust vs. non-robust estimation, Fundamental matrix estimation, Essential matrix, Point correction, Self-calibration, and Structure from Motion for the full detail behind each stage.

The SparseReconstructor class (and its BaseSparseReconstructor base class) is the main entry point that drives this pipeline end to end.

Next steps

  • Installation — how to add irurueta-ar as a dependency, or build it from source.

  • Reference — links to the generated Javadoc, coverage, and code-quality reports.