这是示教-重复-重新计划系统的实验验证。要在现实世界中运行此系统，请按照本页中的教程操作。

要在模拟中测试示教-重复-重新计划系统，请转到主分支。

教学-重复-重新计划 (自主无人机比赛)

示教-重复-重新计划: 一个完整而强大的复杂环境中积极飞行的系统
示教-重复-重新计划是一个完整而强大的系统，支持自主无人机竞赛。它包含了复杂环境中无人机攻击性飞行的所有组件。它建立在经典的机器人教学和重复框架的基础上，该框架被广泛应用于基础设施检查、空中运输和搜救。我们的系统可以捕捉用户的飞行任务意图，将任意不稳定的教学轨迹转换成保证平稳安全的重复轨迹，并生成安全的本地重新计划，以避免飞行中未映射或移动的障碍物。

Architecture:

点击查看【processon】

相关论文

• Teach-Repeat-Replan: A Complete and Robust System for Aggressive Flight in Complex Environments, Fei Gao, Luqi Wang, Boyu Zhou, Luxin Han, Jie Pan, shaojie Shen, IEEE Transactions on Robotics (T-RO), to appear, 2020.
• Optimal Trajectory Generation for Quadrotor Teach-and-Repeat, Fei Gao, Luqi Wang, Kaixuan Wang, William Wu, Boyu Zhou, Luxin Han, Shaojie Shen, IEEE Robotics and Automation Letters (RA-L), 2019.

如果您在应用或研究中使用示教-重复-重新计划或其子模块，请引用我们的相关论文。 bib

Hardware

机载计算机 (由无人机携带) 通过Wifi或以太网与地面站通信。所示下图。记得在机载计算机上启动roscore，以减少控制无人机时的数据传输延迟。

教程

1. 经过测试的硬件/软件

通过以下测试的硬件/软件组合是dji N3飞行控制器, dji Manifold 2-C 机载计算机, dji Lightbridge2 遥控器, intel realsense D435i camera and Ubuntu 16.04.

2. 安装驱动程序和依赖项

• Run ./install_tools.sh script 在src文件夹中安装必要的依赖项。如果出现任何问题，请打开此脚本并逐个手动安装。
• Follow the instruction1 to install the drivers for N3 flight controller on onboard computer. version 3.7 of Onboard-SDK is recomended.
• Follow the instruction2 to install the drivers for intel realsense on onboard computer.
• Install necessary solvers Mosek, OOQP, Ceres and NLopt. You can refer to simulation for more details.

3. 在机载计算机上编译代码

在此目录中的板载计算机和src文件夹上创建ros工作区， 将onboardcomputer中的所有文件移动到新创建的src文件夹, and compile the project in the workspace with catkin_make command. 如果系统提示您缺少库，请用谷歌搜索并安装丢失的库。Finally, add the script source <your_catkin_ws>/devel/setup.bash to ~/.bashrc_.

4. 配置并运行VIO模块 — vins-fusion

1. Run djiros to get IMU data from N3

Use the USB cable to connect the N3 flight controller to DJI Assistant2 software, check the Enable API box and select baud rate to 921600, 然后拔下并插入电源线以重新启动飞行控制器。

Use CP2102 module to connect Manifold-2C to N3 flight controller. Fill out the APP ID and APP KEY parameter to /src/djictrl/djiros/launch/djiros.launch. For more detials, please refer to dji developer website and N3 flight controller datasheet.

Run roslaunch djiros djiros.launch, and then run rostopic hz /djiros/imu. It can be seen that the frequency of data output of imu is 400HZ, which means success.

1. 运行realsense获取深度和灰度图像

Just run roslaunch realsense2_camera rs_camera.launch. If an error is reported, try replugging the USB port. After that, run rqt_image_viewer and select /camera/depth/image_rect_raw or /camera/infra1/image_rect_raw to check if the display is correct.

1. 配置摄像机参数

Enter /src/vio/vins-fusin/config/realsense/ folder, where lies the main configuration files for vins-fusin. Our hardware/software configuration matches realsense_stereo_imu_config.yaml where you must properly reconfigure the extrinsics of the camera and IMU (body_T_cam0, body_T_cam1, center_T_imu).

然后在同一目录中，打开left.yaml，这是左侧相机的参数配置文件。 Configure camera resolution, distortion factor, intrinsic parameters, etc.同样地， open the right.yaml file and fill in the parameters for the right camera.

1. 运行VINS-Fusion

现在是时候运行整个VIO系统了。Enter /src/ directory, and then open sensing_estimation.sh, change the path to your own realsense_stereo_imu_config.yaml file at the last two lines and save the file.
Make sure that all the ros nodes are killed and run ./sensing_estimation.sh in /src/ directory. It will take about ten seconds to finish the initialization, and after that accurate and fast camera pose and trajectory can be seen in rviz as show in the fig.1 that if all the parameters are configured properly.

5. 地面计算机和机载计算机之间的通信

1. 将地面计算机和机载计算机连接到同一LAN网络，以便它们可以相互ping通。Enter the ifconfig command to get the IP addresses at both computers. If the IP of the onboard computer is 192.168.1.100 and the IP of the ground computer is 192.168.1.101, than at the end of the ~/.bashrc on onboard computer, add the code below:

export ROS_HOSTNAME = 192.168.1.100
export ROS_MASTER_URI = http://192.168.1.100:11311

  在地面计算机上，将以下代码添加到同一文件中:

export ROS_HOSTNAME = 192.168.1.101
export ROS_MASTER_URI = http://192.168.1.100:11311

    Then `source ~/.bashrc` on both computers.

1. 通信测试: 运行机载计算机上的任何节点，然后在地面计算机上输入rostopic列表，主题应显示在地面侧，这意味着通信成功。

6. 建图

• 在地面计算机的ROS工作区的地面站文件夹中编译代码。
• Create a new folder ~/output/pose_graph/ on the onboard computer to store the pose graph generated during mapping. Run ./sensing_estimation.sh. (It is highly recommended that the rviz statement be manually commented out to save computing resources.)
• Open /src/global_mapping/surfel_fusion/launch/global_map_fusion.launch on the ground computer and correctly modify the intrinsics and extrinsics of the camera.
• Modify the path of the .rviz file in mapping.sh, and then run ./mapping.sh. Move the quadrotor around and you will see Fig.2 below if everything goes well.

• 建图阶段完成后, 地面计算机可以按Ctrl + C关闭。程序将在退出前自动保存点云数据，文件名为model.pcd。机载计算机不能在建图阶段后使用Ctrl + C直接关闭，因为它需要保存建图过程中得到的姿势图，并且该过程必须通过输入 “s” 和手动按 “Enter”。然后机载计算机将记录的姿势图保存到~/output/pose_graph folder.

P.S.

• 确保机载计算机和地面计算机之间的网络速度至少可以达到20 mb/s。可以在系统监控软件中查看通信速度。如果无线网络的速度不够，建议在映射过程中使用有线连接。
• 如果要重建地图，则需要删除所有现有文件 in the ~/output/pose_graph/ folder.

7. 测试四旋翼是否正常运行

1. Modify the path of the realsense_stereo_imu_config.yaml file related to VINS in the script file ctrl_estimation_replanning.sh, and then run the ./ctrl_estimation_replanning.sh script on the onboard computer.
2. 试着飞。 Firstly, put the small switch(see the left picture of Fig.3) lies in the left upper side of the remote controller to stay at position “A”, and move the “H” switch(see the right picture of Fig.3) that in right bottom side of the remote controller to hit up. 然后取下四旋翼, in the air, move the left upper switch from A to P and the aircraft should perform fixed-point immediately that stablely hovering in the air, 这意味着控制器运行正常。

8. 执行教学-重复-重新计划

1. 修改相机参数in /src/local_replanner/local_replanner/launch/local_replanner.launch, and then run ctrl_estimation_replanning.sh on the onboard computer, waiting for pose graph to be added (possibly more than 1 minute), which will be prompted in terminal after loading.
2. Modify the name of point cloud file during mapping in the teaching_planning.sh script file, and then run the ./teaching_planning.sh. The rviz will be like Fig.4.

1. 然后使用Xbox操纵杆或键盘在rviz (教短语) 中绘制轨迹。
2. Start the quadrotor with a remote control while keeping the upper left switch be selected in A and the lower right “H” switch in the position up. At an altitude of about one meter, move the upper left switch from A to P, the quadrotor will enter a stable hovering state, and then move the lower right “H” switch down. If nothing goes wrong, the quadrotor will automatically start to execute the desired trajectory.

到目前为止，整个系统已经成功运行。

9.故障排除

1. 如果出现莫名其妙的错误，您可以尝试重新启动机载计算机和地面计算机。
2. 尽量不要在机载计算机上运行rviz，这会占用太多资源并对性能产生负面影响。
3. If the quadrotor can not be armed, then we need to unplug the USB/serial module and restart the flight controller, finally plug the USB/serial module. Make sure the USB/serial module is disconnected when the flight controller is powering on.
4. 如果realsense无法启动，可以通过再次拔下/插入来解决。
5. 始终注意通信质量、高延迟和低网络速度，这将导致意想不到的问题。
6. 如果在教学和重复阶段无法生成轨迹，建议缓慢教学，并确保在rviz中生成的所有多面体不间断。
7. 由于本地重新规划器在现实世界中是不稳定的，它在代码中被临时阻止。请等待更新。当地飞机的缺乏不会影响全球轨迹的产生和飞行，但无法避免飞行过程中出现的意外障碍。