profile photo

Paul‑Edouard Sarlin

EMAIL | RÉSUMÉ | LINKEDIN | GITHUB | TWITTER | SCHOLAR

PhD student in the Computer Vision and Geometry group at ETH Zurich.

I started in February 2020 and am supervised by Prof. Marc Pollefeys. I am interested in solving 3D computer vision and geometry problems, especially large-scale visual mapping and localization, using Deep Learning. I have previously worked as a research intern at Google Research in Zurich, Facebook Reality Labs in London, the Microsoft Mixed Reality & AI lab in Zurich, and Magic Leap in San Francisco. I have also spent time at ASL, Traverse.ai, NUS, and EPFL.

News

07/2023: LightGlue, our new super-fast deep neural network for feature matching, got accepted to ICCV 2023.
06/2023: I spent 6 months at Google and worked on SNAP, a framework that infers rich neural maps from streetview and aerial images using self-supervision.
04/2023: OrienterNet, which I worked on during an internship at Meta Reality Labs, got accepted to CVPR 2023.
10/2022: We're releasing LaMAR, a benchmark for visual localization and mapping with Augmented Reality devices! Check out our ECCV 2022 paper and dedicated 4-hour tutorial.
07/2022: I attended ICVSS 2022 and received the Best Presentation Award, covering almost 3 years of research: SuperGlue, Back to the Feature, and Pixel-Perfect SfM.
10/2021: We received the Best Student Paper Award at ICCV 2021 for our work on Pixel-Perfect Structure-from-Motion! Learn more about it by watching our detailed video.
09/2021: I joined Facebook Reality Labs for a research internship.
08/2021: We released a preview of the upcoming ETH-Microsoft visual localization dataset as part of an ICCV 2021 challenge. It focuses on AR scenarios in challenging conditions.
06/2021: Our paper Back to the Feature will appear at CVPR 2021. Check out our cool visualizations 💥
05/2021: I received the Graduate Award 2020 of the SEW-Eurodrive Foundation for my Master's thesis SuperGlue.
04/2021: I joined the Microsoft Mixed Reality & AI lab in Zurich as a research intern.
08/2020: SuperGlue and my toolbox hloc won 4 ECCV 2020 competitions on visual localization. I gave two talks at two workshops (here and there), and released my slides below.
07/2020: I just released hloc, a new toolbox for visual localization and structure-from-motion with SuperGlue and other modern approaches.
06/2020: SuperGlue won 3 CVPR 2020 competitions: local features for visual localization, visual localization for handheld devices, and the image matching challenge. As a result, I gave 4 talks at various CVPR 2020 workshops, with recordings publicly available.
04/2020: My Master's thesis SuperGlue, conducted at Magic Leap, got accepted to CVPR 2020 for oral presentation! The code is now publicly available.
02/2020: I joined Prof. Marc Pollefeys’ group as a PhD student.
10/2019: Our Fishyscapes Benchmark paper was presented as an oral at the ICCV 2019 workshop on Autonomous Driving. The benchmark is up and running at fishyscapes.com.
06/2019: Our Hierarchical Localization paper won the Visual Localization Challenge at the CVPR 2019 workshop on Long-Term Visual Localization. The code is available on GitHub.
04/2019: I started my 6-month Master’s thesis with Tomasz Malisiewicz at Magic Leap in Sunnyvale, California.
04/2019: New preprint on uncertainty estimation: The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation.
04/2019: My second paper From Coarse to Fine: Robust Hierarchical Localization at Large Scale got accepted to CVPR 2019.
09/2018: My first paper Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization got accepted to CoRL 2018.
06/2018: I started a 6-month internship at Magic Leap, Zurich.

My Research

SNAP

SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding

Paul-Edouard Sarlin, Eduard Trulls, Marc Pollefeys, Jan Hosang, Simon Lynen

Preprint

Paper | Bibtex 
@article{sarlin2023snap,
  author    = {Paul-Edouard Sarlin and
               Eduard Trulls and
               Marc Pollefeys and
               Jan Hosang and
               Simon Lynen},
  title     = {{SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding}},
  journal = {ArXiv},
  year      = {2023}
}
LightGlue

LightGlue: Local Feature Matching at Light Speed

Philipp Lindenberger, Paul-Edouard Sarlin, Marc Pollefeys

ICCV 2023

Paper | Code | Bibtex 
@inproceedings{lindenberger2023lightglue,
  author    = {Philipp Lindenberger and
               Paul-Edouard Sarlin and
               Marc Pollefeys},
  title     = {{LightGlue: Local Feature Matching at Light Speed}},
  booktitle = {ICCV},
  year      = {2023}
}
OrienterNet

OrienterNet: Visual Localization in 2D Public Maps with Neural Matching

Paul-Edouard Sarlin, Daniel DeTone, Tsun-Yi Yang, Armen Avetisyan, Julian Straub, Tomasz Malisiewicz, Samuel Rota Bulo, Richard Newcombe, Peter Kontschieder, Vasileios Balntas

CVPR 2023

Paper | Code | Webpage | Bibtex 
@inproceedings{sarlin2023orienternet,
  author    = {Paul-Edouard Sarlin and
               Daniel DeTone and
               Tsun-Yi Yang and
               Armen Avetisyan and
               Julian Straub and
               Tomasz Malisiewicz and
               Samuel Rota Bulo and
               Richard Newcombe and
               Peter Kontschieder and
               Vasileios Balntas},
  title     = {{OrienterNet: Visual Localization in 2D Public Maps with Neural Matching}},
  booktitle = {CVPR},
  year      = {2023},
}
The LaMAR Benchmark

LaMAR: Benchmarking Localization and Mapping for Augmented Reality

Paul-Edouard Sarlin,* Mihai Dusmanu,* Johannes L. Schönberger, Pablo Speciale, Lukas Gruber, Viktor Larsson, Ondrej Miksik, Marc Pollefeys

ECCV 2022

Paper | Webpage | Video | Microsoft Research Blog | Bibtex 
@inproceedings{sarlin2022lamar,
  author    = {Paul-Edouard Sarlin and
               Mihai Dusmanu and
               Johannes L. Schönberger and
               Pablo Speciale and
               Lukas Gruber and
               Viktor Larsson and
               Ondrej Miksik and
               Marc Pollefeys},
  title     = {{LaMAR: Benchmarking Localization and Mapping for Augmented Reality}},
  booktitle = {ECCV},
  year      = {2022},
}
Pixel-Perfect SfM

Pixel-Perfect Structure-from-Motion with Featuremetric Refinement

Philipp Lindenberger,* Paul-Edouard Sarlin,* Viktor Larsson, Marc Pollefeys

ICCV 2021 - best student paper award

Paper | Webpage | Code | Bibtex 
@inproceedings{lindenberger2021pixsfm,
  title     = {{Pixel-Perfect Structure-from-Motion with Featuremetric Refinement}},
  author    = {Philipp Lindenberger and
               Paul-Edouard Sarlin and
               Viktor Larsson and
               Marc Pollefeys},
  booktitle = {ICCV},
  year      = {2021},
}
Back to the Feature

Back to the Feature: Learning Robust Camera Localization from Pixels to Pose

Paul-Edouard Sarlin,* Ajaykumar Unagar,* Måns Larsson, Hugo Germain, Carl Toft, Viktor Larsson, Marc Pollefeys, Vincent Lepetit, Lars Hammarstrand, Fredrik Kahl, Torsten Sattler

CVPR 2021

Paper | Webpage | Code | Bibtex 
@inproceedings{sarlin2021pixloc,
  title     = {{Back to the Feature: Learning Robust Camera Localization from Pixels to Pose}},
  author    = {Paul-Edouard Sarlin and
               Ajaykumar Unagar and
               Måns Larsson and
               Hugo Germain and
               Carl Toft and
               Viktor Larsson and
               Marc Pollefeys and
               Vincent Lepetit and
               Lars Hammarstrand and
               Fredrik Kahl and
               Torsten Sattler},
  booktitle = {CVPR},
  year      = {2021},
}
SuperGlue

SuperGlue: Learning Feature Matching with Graph Neural Networks

Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, Andrew Rabinovich

CVPR 2020 - oral presentation and winner of 7 competitions on localization and matching

Paper | Webpage | Code | Bibtex 
@inproceedings{sarlin2020superglue,
  title     = {{SuperGlue: Learning Feature Matching with Graph Neural Networks}},
  author    = {Paul-Edouard Sarlin and
               Daniel DeTone and
               Tomasz Malisiewicz and
               Andrew Rabinovich},
  booktitle = {CVPR},
  year      = {2020},
}
Fishyscapes

The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation

Hermann Blum,* Paul-Edouard Sarlin,* Juan Nieto, Roland Siegwart, Cesar Cadena

ICCV Workshops 2019

Paper | Webpage | Bibtex 
@inproceedings{blum2019fishyscapes,
  title     = {{The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation}},
  author    = {Hermann Blum and
               Paul-Edouard Sarlin and
               Juan Nieto and
               Roland Siegwart and
               Cesar Cadena},
  booktitle = {ICCV Workshops},
  year      = {2019}
}
CVPR 2019 teaser

From Coarse to Fine: Robust Hierarchical Localization at Large Scale

Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, Marcin Dymczyk

CVPR 2019 - winner of the visual localization challenge

Paper | Code | Bibtex 
@inproceedings{sarlin2019coarse,
  title     = {{From Coarse to Fine: Robust Hierarchical Localization at Large Scale}},
  author    = {Paul-Edouard Sarlin and
               Cesar Cadena and
               Roland Siegwart and
               Marcin Dymczyk},
  booktitle = {CVPR},
  year      = {2019}
}
CoRL 2018 teaser

Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization

Paul-Edouard Sarlin, Frédéric Debraine, Marcin Dymczyk, Roland Siegwart, Cesar Cadena

CoRL 2018

Paper | Video | Code | Bibtex 
@inproceedings{hloc2018,
  title     = {{Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization}},
  author    = {Paul-Edouard Sarlin and
               Fr{\'{e}}d{\'{e}}ric Debraine and
               Marcin Dymczyk and
               Roland Siegwart and
               Cesar Cadena},
  booktitle = {Conference on Robot Learning (CoRL)},
  year      = {2018}
}

Talks

09/2023: Localization and Mapping with 2D Neural Maps. Grab Singapore.
07/2023: Learning to Map the World. Zurich Computer Vision Meetup on Building a digital world. Zurich, Switzerland.
04/2023: OrienterNet: Visual Localization in 2D Public Maps with Neural Matching. Google Research. Zurich, Switzerland.
10/2022: Tutorial on Localization and Mapping for Augmented Reality. ECCV 2022. Tel-Aviv, Israel.
09/2022: Learning Priors for Visual Localization and Mapping. Daniel Cremers' Computer Vision Group, TU Munich.
09/2022: Learning Priors for Visual Localization and Mapping. Wayve.
07/2022: Learning Priors for Visual Localization. ICVSS 2022. Sicily, Italy. [Poster]
04/2022: Learning Priors for Large-scale Visual Localization in all Conditions. Noah Snavely's Group, Cornell Tech.
11/2021: Pixel-Perfect Structure-from-Motion with Featuremetric Refinement. Mixed Reality Seminar, Microsoft.
11/2021: Featuremetric Optimization as a Substitute for Good Keypoints. MatchLab, Imperial College London.
08/2020: Hierarchical Localization with hloc and SuperGlue. ECCV 2020 Workshop on Long-Term Visual Localization under Changing Conditions. Glasgow, UK. [Slides, Video]
08/2020: Hierarchical Localization with hloc and SuperGlue. ECCV 2020 Workshop on Map-based Localization for Autonomous Driving. Glasgow, UK. [Slides, Video]
06/2020: SuperGlue: Winner of the Image Matching Challenge. CVPR 2020 Workshop on Image Matching: Local Features & Beyond. Seattle WA, USA. [Video]
06/2020: SuperGlue: Learning Feature Matching with Graph Neural Networks. CVPR 2020 Workshop on 3D Scene Understanding for Vision, Graphics, and Robotics. Seattle WA, USA. [Video]
06/2020: SuperGlue for Indoor and Outdoor Visual Localization. CVPR 2020 Workshop on Long-Term Visual Localization, Visual Odometry and Geometric and Learning-based SLAM. Seattle WA, USA. [Video]
06/2020: SuperGlue for Day-Night Local Feature Matching. CVPR 2020 Workshop on Long-Term Visual Localization, Visual Odometry and Geometric and Learning-based SLAM. Seattle WA, USA. [Video]
06/2019: From Coarse to Fine: Robust Hierarchical Localization at Large Scale. CVPR 2019 Workshop on Long-Term Visual Localization under Changing Conditions. Long beach CA, USA.

Awards