orbslam

ORB-SLAM: a Versatile and Accurate Monocular SLAM System

Introduction

• Use ORB features
• Real time operation, using covisibility graph
• real time loop closing based on essential graph
• real time relocalization
• initialization & model selection
• survival of the fittest approach to the map point & keyframe selection

System Overview

Feature Choice

• ORB

Three Threads: Tracking, Local Mapping & Loop Closing

• Tracking
  localizing camera and deciding when to insert a keyframe

  • initial feature matching & motion-only BA
  • IF tracking is lost
    • place recognition > global relocalization
  • initial camera pose & feature matchings
    • local visible map using covisibility graph of keyframes
  • matches found by reprojection & camera pose opt again
  • decide whether to insert keyframe

• Local Mapping

  • process new keyframes and local BA
  • unmatched ORB searched in connected keyframes in the covisibility graph to triangulate new points
  • exigent point culling

• Loop Closing

  • search for loops with every new keyframe
  • IF loop is detected
    • compute similarity transformation
    • both sides of the loop is aligned and duplicated points fused
    • pose graph opt > global consistency
      • MAIN NOVELTY: opt on Essential Graph

MapPoints, KeyFrames, Selection

MapPoint p_i

• 3D position: X_w,i
• The viewing direction: n_i
• ORB descriptor: D_i
• d_max,d_min: maximum distance & minimum distance the point can be observed

KeyFrame K_i

• T_iw, tranformation world2camera
• Camera intrinsics, including focal length & principal point
• Undistorted coordinates of ORB features

Covisibility Graph & Essential Graph**

• Covisibility Graph: Undirected Weighted Graph

  • node: keyframe
  • edge: shared observations of the same MapPoints

• Essential Graph

  • spanning tree
    • INSERTED: new keyframe linked tokeyframe which shares most point observations
    • ERASED: update links affected by that KeyFrame

BoW Place Recognition

• DBoW2: loop detection & relocalization

Automatic Map Initialization

GOAL

compute relative pose between 2 frames & triangulate an initial set of MapPoints.

STEPS

• Find intial correspondences
  • ORB

• Parallel computation of two models

  • homography H_cr

  • fundamental matrix F_cr

    each interation, compute score S_m

• Model selection
  

  • IF R_H > 0.45
    • choose homography
  • ELSE
    • fundamental matrix

• Motion and SfM

  • check if there is one solution with most points seen with parallax, in front of both cameras and with low reprojection error

• BA

  • full BA to refine initial reconstruction

Tracking

1. ORB Extraction

   • FAST corners at 8 scale levels
   • compute ORB descriptors & orientation

2. Initial Pose Estimation from Previous Frame

   • IF tracking is SUCCESSFUL
     • constant velocity model
     • wider search

3. Initial Pose Estimation via Global Relocalization

   • IF tracking LOST
     • convert frame into BoW
     • query for keyframe candidates for global relocalization
   • alternatively RANSAC iterations for each keyframe, PnP
   • camera pose with enough liniers
   • camera pose opt

4. Track Local Map

   once get initial estimation of camera pose & feature matches, we can project map into frame and search more map point correspondence

   The local map also has a reference keyframe K_ref, which shares most map points with current frame

   • compute mapPoints projection & discard outliers
   • compute the angle between current viewing ray v and mapPoint mean viewing direction n. Discard if v * n<cos(60°)
   • compute distance, discard if out of [dmin, dmax]
   • compute scale
   • compare D

5. New KeyFrame Decision

   To insert a frame:

   • More than 20 frames passed global reloc
   • Local mapping idle OR >20 frames pass last keyframe insertion
   • current frame tracks >=50 points
   • current frame tracks <=90% of K_ref

Local Mapping

1. KeyFrame insertion

2. Recent MapPoints culling

   in order to be retained in the map, pass a restrictive test during te firsr 3 keyframes after creation

   • 25% the frames visible

   • observed from >= last 3 key frames

3. New MapPoint Creation

   Created by triangulating ORB from connected keyframes K_c

4. Local BA

   • current processed KeyFrame K_i
   • all keyframes connected to it
   • all MapPoints seen bt those
     NOTE all other KFs not connected but see,remain fixed in opt

5. Local KF culling

   GOAL detect redundant KFs & delete

Loop Closing

1. Loop Candidates Detection
   • compute similarity
   • discard below lowest score in DB
2. Compute Similarity Transformation
   • ORB
   • RANSAC iterations, find ST using Horn
     • IF enough inliers found
       • opt & guided search for more correspondences4
       • opt again & check
3. Loop Fusion
   • fuse duplicated MapPoints
   • insert new edges

4. Essential Graph Optimization

   effectively correct drifts