Fixing the Scale and Shift in Monocular Depth For Camera Pose Estimation

Introduction

This repository hosts the solvers of the paper:
"Fundamental Matrix Estimation Using Relative Depths" ECCV 2024. [Paper][Supp]
"Fixing the Scale and Shift in Monocular Depth For Camera Pose Estimation" Arxiv 2025. [Paper]
We used [graph-cut ransac] for robust estimation. The evaluation script is based on [homography-benchmark]

Installation

Install in Conda environment

conda conda create -n posemono python=3.9
conda activate posemono
cd graph-cut-ransac
python -m pip install .
cd ..
pip install -r requirements.txt

Data

An example data can be downloaded [here]
More details come soon.

Run the test code by specifically typing the scene, solver and depth name, e.g.,

python test_calibrated.py --path 'pathtodata' --scene 'british_museum' --solver 'mono' --depth 'unidepth'

which solves the calibrated relative pose problem using 3 points with their monodepths.

Solvers in this repo

📁 graph-cut-ransac/src/pygcransac/include/estimators: contains the code of all the solvers.

Fundamental matrix estimation using relative depth with unknown relative scale:

solver_fundamental_matrix_4p4d - general fundamental matrix estimation
solver_fundamental_matrix_4p3d - varying focal lengths problem
solver_fundamental_matrix_3p3d - equal and unknown focal length

Camera pose estimation using monocular depth modeling scale and shift:

Calibrated case:
solver_essential_matrix_mono_suv - closed-form solution, fastest. - ${\rm 3PT}_{suv}$ (C)
solver_essential_matrix_mono_suv_GB - Gröbner basis solution. - ${\rm 3PT}_{suv}$ (GB)
solver_essential_matrix_mono_suv_Eigen - polynomial eigenvalue solution. - ${\rm 3PT}_{suv}$ (Eigen)
solver_essential_matrix_mono_suv_inverse - inverse depth model (not practical). - ${\rm 3PT}_{suv}$ (inverse)

Solver	G-J	Eigen	Poly	Time($\mu s$)
${\rm 3PT}_{suv}$ (GB)	12 × 16	4 × 4	-	4.45
${\rm 3PT}_{suv}$ (Eigen)	6 × 10	4 × 4	-	3.42
${\rm 3PT}_{suv}$ (C)	3 × 6	-	4	1.46
${\rm 3PT}_{suv}$ (inverse)	54 × 66	12 × 12	-	36.9

Equal and unknown focal length:
solver_fundamental_matrix_mono_suvf_GB - Gröbner basis solution. - ${\rm 4PT}_{suv}f$ (GB)
solver_fundamental_matrix_mono_suvf_Eigen - polynomial eigenvalue solution. - ${\rm 4PT}_{suv}f$ (Eigen)

Solver	G-J	Eigen	Time($\mu s$)
${\rm 4PT}_{suv}f$ (GB)	24 × 32	8 × 8	12.5
${\rm 4PT}_{suv}f$ (Eigen)	6 × 8	2 × 2	2.38

Varying focal lengths:
solver_fundamental_matrix_mono_suvfvar_GB - Gröbner basis solution. - ${\rm 4PT}_{suv}f_{1,2}$ (GB)
solver_fundamental_matrix_mono_suvfvar_Eigen - polynomial eigenvalue solution. - ${\rm 4PT}_{suv}f_{1,2}$ (Eigen)

Solver	G-J	Eigen	Time($\mu s$)
${\rm 4PT}_{suv}f_{1,2}$ (GB)	20 × 24	4 × 4	6.45
${\rm 4PT}_{suv}f_{1,2}$ (Eigen)	6 × 8	2 × 2	2.49

References

@inproceedings{ding2025fundamental,
  title={Fundamental matrix estimation using relative depths},
  author={Ding, Yaqing and V{\'a}vra, V{\'a}clav and Bhayani, Snehal and Wu, Qianliang and Yang, Jian and Kukelova, Zuzana},
  booktitle={European Conference on Computer Vision},
  pages={142--159},
  year={2025},
  organization={Springer}
}

@article{ding2025fixing,
  title={Fixing the Scale and Shift in Monocular Depth For Camera Pose Estimation},
  author={Ding, Yaqing and V{\'a}vra, V{\'a}clav and Kocur, Viktor and Yang, Jian and Sattler, Torsten and Kukelova, Zuzana},
  journal = {arXiv preprint arXiv:2501.07742},
  year={2025}
}