一种基于优化的多传感器状态估计器

vins_logo.png
kitti.png
VINS-Fusion是一种基于优化的多传感器状态估计器，可实现自主应用 (无人机、汽车和AR/VR) 的精确自定位。VINS-Fusion is an extension of VINS-Mono, which supports multiple visual-inertial sensor types (mono camera + IMU, stereo cameras + IMU, even stereo cameras only). 我们还展示了VINS与GPS融合的玩具示例。
特征:

  • 支持多传感器 (stereo cameras / mono camera+IMU / stereo cameras+IMU)
  • 在线空间校准(transformation between camera and IMU)
  • 在线时间校准 (time offset between camera and IMU)
  • 可视环扣

kitti_rank.png
We are the top open-sourced stereo algorithm on KITTI Odometry Benchmark (12.Jan.2019).

Authors: Tong Qin, Shaozu Cao, Jie Pan, Peiliang Li, and Shaojie Shen from the Aerial Robotics Group, HKUST

Videos:

相关论文 :( 论文与代码不完全相同)

  • Online Temporal Calibration for Monocular Visual-Inertial Systems, Tong Qin, Shaojie Shen, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS, 2018), best student paper award pdf
  • VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator, Tong Qin, Peiliang Li, Shaojie Shen, IEEE Transactions on Robotics pdf

如果您使用VINS-Fusion进行学术研究，请引用我们的相关论文. bib

1. 先决条件

1.1 Ubuntu and ROS

Ubuntu 64-bit 16.04 or 18.04.
ROS Kinetic or Melodic. ROS Installation

1.2. Ceres Solver

Follow Ceres Installation.

2. 构建 VINS-Fusion

克隆存储库和catkin_make:

  1.     cd ~/catkin_ws/src
  2.     git clone https://github.com/HKUST-Aerial-Robotics/VINS-Fusion.git
  3.     cd ../
  4.     catkin_make
  5.     source ~/catkin_ws/devel/setup.bash

(如果在此步骤中失败，请尝试查找具有干净系统的另一台计算机，或重新安装Ubuntu和ROS)

3. EuRoC Example

Download EuRoC MAV Dataset to YOUR_DATASET_FOLDER. Take MH_01 for example, 您可以使用三种传感器类型运行VINS-Fusion (monocular camera + IMU, stereo cameras + IMU and stereo cameras).
打开四个终端，运行vins里程计，可视环闭合 (可选), rviz and play the bag file respectively.
绿色路径是VIO里程计; 红色路径是视环闭合下的里程计。

3.1 Monocualr camera + IMU

  1.     roslaunch vins vins_rviz.launch
  2.     rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml 
  3.     (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml 
  4.     rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag

3.2 Stereo cameras + IMU

  1.     roslaunch vins vins_rviz.launch
  2.     rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_imu_config.yaml 
  3.     (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_imu_config.yaml 
  4.     rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag

3.3 Stereo cameras

  1.     roslaunch vins vins_rviz.launch
  2.     rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_config.yaml 
  3.     (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_config.yaml 
  4.     rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag

euroc.gif

4. KITTI Example

4.1 KITTI Odometry (Stereo)

Download KITTI Odometry dataset to YOUR_DATASET_FOLDER. Take sequences 00 for example,
Open two terminals, run vins and rviz respectively.
(我们在没有环闭合功能的KITTI基准上评估了里程计)

  1.     roslaunch vins vins_rviz.launch
  2.     (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml
  3.     rosrun vins kitti_odom_test ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/

4.2 KITTI GPS Fusion (Stereo + GPS)

Download KITTI raw dataset to YOUR_DATASET_FOLDER. Take 2011_10_03_drive_0027_synced for example.
打开三个终端，分别运行vins、global fusion和rviz。
绿色路径是VIO里程计; 蓝色路径是GPS全球融合下的里程计。

  1.     roslaunch vins vins_rviz.launch
  2.     rosrun vins kitti_gps_test ~/catkin_ws/src/VINS-Fusion/config/kitti_raw/kitti_10_03_config.yaml YOUR_DATASET_FOLDER/2011_10_03_drive_0027_sync/ 
  3.     rosrun global_fusion global_fusion_node

kitti.gif

5. VINS-汽车融合示范

Download car bag to YOUR_DATASET_FOLDER.
Open four terminals, run vins odometry, visual loop closure(optional), rviz and play the bag file respectively.
绿色路径是VIO里程计; 红色路径是视环闭合下的里程计。

  1.     roslaunch vins vins_rviz.launch
  2.     rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/vi_car/vi_car.yaml 
  3.     (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/vi_car/vi_car.yaml 
  4.     rosbag play YOUR_DATASET_FOLDER/car.bag

car_gif.gif

6. 使用您的设备运行

VIO不仅是一种软件算法，它在很大程度上依赖于硬件质量。对于初学者，我们建议您使用专业设备运行VIO，其中包含全局快门相机和硬件同步。

6.1 配置文件

为您的设备编写配置文件. You can take config files of EuRoC and KITTI as the example.

6.2 Camera calibration

VINS-Fusion support several camera models (针孔, mei, 等距). You can use camera model to calibrate your cameras. We put some example data under /camera_models/calibrationdata to tell you how to calibrate.

  1. cd ~/catkin_ws/src/VINS-Fusion/camera_models/camera_calib_example/
  2. rosrun camera_models Calibrations -w 12 -h 8 -s 80 -i calibrationdata --camera-model pinhole

7. Docker支持

To further facilitate the building process, we add docker in our code. Docker environment is like a sandbox, thus makes our code environment-independent. To run with docker, first make sure ros and docker are installed on your machine. Then add your account to docker group by sudo usermod -aG docker $YOUR_USER_NAME. Relaunch the terminal or logout and re-login if you get Permission denied error, type:

  1. cd ~/catkin_ws/src/VINS-Fusion/docker
  2. make build

请注意，docker构建过程可能需要一段时间，具体取决于您的网络和计算机。VINS-Fusion成功构建后，您可以使用脚本run.sh运行vins评估程序。
Script run.sh can take several flags and arguments. Flag -k means KITTI, -l represents loop fusion, and -g stands for global fusion. You can get the usage details by ./run.sh -h. Here are some examples with this script:

  1. # Euroc Monocualr camera + IMU
  2. ./run.sh ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml
  4. # Euroc Stereo cameras + IMU with loop fusion
  5. ./run.sh -l ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml
  7. # KITTI Odometry (Stereo)
  8. ./run.sh -k ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/
  10. # KITTI Odometry (Stereo) with loop fusion
  11. ./run.sh -kl ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/
  13. #  KITTI GPS Fusion (Stereo + GPS)
  14. ./run.sh -kg ~/catkin_ws/src/VINS-Fusion/config/kitti_raw/kitti_10_03_config.yaml YOUR_DATASET_FOLDER/2011_10_03_drive_0027_sync/

In Euroc cases, you need open another terminal and play your bag file. If you need modify the code, simply re-run ./run.sh with proper auguments after your changes.

8. 确认

We use ceres solver for non-linear optimization and DBoW2 for loop detection, a generic camera model and GeographicLib.

9. License

The source code is released under GPLv3 license.

We are still working on improving the code reliability. For any technical issues, please contact Tong Qin .
For commercial inquiries, please contact Shaojie Shen .