badslam

BAD SLAM

Overview

BAD SLAM is a real-time approach for Simultaneous Localization and Mapping (SLAM) for RGB-D cameras. Supported platforms are Linux and Windows. The software requires an NVidia graphics card with CUDA compute capability 5.3 or later (however, it would be easy to lower this requirement).

This repository contains the BAD SLAM application and the library it is based on, libvis. The library is work-in-progress and it is not recommended to use it for other projects at this point.

The application and library code is licensed under the BSD license, but please also notice the licenses of the included or externally used third-party components.

If you use the provided code for research, please cite the paper describing the approach:

Thomas Schöps, Torsten Sattler, Marc Pollefeys, "BAD SLAM: Bundle Adjusted Direct RGB-D SLAM", CVPR 2019.

The Windows port and Kinect-for-Azure (K4A) integration has been contributed by Silvano Galliani (Microsoft AI & Vision Zurich).

Screenshots & Videos

• Oral presentation of the conference paper by Torsten Sattler at CVPR 2019
• Short demonstration of some of the SLAM GUI features

Camera requirements

Please keep in mind that BAD SLAM has been designed for high-quality RGB-D videos and is likely to perform badly (no pun intended) on lower-quality RGB-D videos. For more details, see the documentation on camera compatibility.

Pre-built binaries

Binaries are available for download as GitHub releases.

Windows

For Windows, an executable compiled with Visual Studio 2019 is provided. It is also required to download the loop closure resource files as described below in this ReadMe, or loop closures will be disabled. In addition, performing CUDA block-size autotuning as also described below is recommended.

If the executable fails to start due to missing DLLs, try installing the latest Visual C++ redistributable files for Visual Studio 2019.

Linux

For Linux, an AppImage is provided. Please note that it is also required to download the loop closure resource files as described below in this ReadMe, or loop closures will be disabled. In addition, performing CUDA block-size autotuning as also described below is recommended.

In case you encounter an error like

./badslam: relocation error: [...]/libQt5DBus.so.5: symbol dbus_message_get_allow_interactive_authorization, version LIBDBUS_1_3 not defined in file libdbus-1.so.3 with link time reference

then your dbus library is too old. This can be fixed by downloading a recent version and setting LD_LIBRARY_PATH to the directory containing these files before starting the AppImage.

Building

Building has been tested on Ubuntu 14.04 and Ubuntu 18.04 (with gcc), and on Windows (with Visual Studio 2019 and 2017).

The following external dependencies are required.

Notice that OpenCV is only required as a dependency for loop detection by DLib. Also notice that there are (at least) two different libraries with the name DLib, so be sure to install the correct one.

The following external dependencies are optional.

Build instructions for Linux

Since OpenGV (at the time of writing) always uses the -march=native flag, both BAD SLAM and g2o must use this as well. (For g2o, check that the BUILD_WITH_MARCH_NATIVE CMake option is set to ON.) If there are inconsistencies, the program will likely crash when OpenGV or g2o functionality is used (i.e., at loop closures).

After obtaining all dependencies, the application can be built with CMake, for example as follows:

mkdir build_RelWithDebInfo
cd build_RelWithDebInfo
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_CUDA_FLAGS="-arch=sm_61" ..
make -j badslam  # Reduce the number of threads if running out of memory, e.g., -j3

Make sure to specify suitable CUDA architecture(s) in CMAKE_CUDA_FLAGS. Common settings would either be the CUDA architecture of your graphics card only (in case you only intend to run the compiled application on the system it was compiled on), or a range of virtual architectures (in case the compiled application is intended for distribution). See the corresponding CUDA documentation.

Optionally, after building, the unit tests can be run, which test some of the bundle adjustment functionality. To do so, build and run the following executable:

make -j badslam_test
./build_RelWithDebInfo/applications/badslam/badslam_test

All tests should pass. Troubleshooting:

• If you get a CUDA error like "too many resources requested for launch", probably a default CUDA kernel block size does not work for your GPU. See below for block-size tuning. The application has been tested on GTX 1080 and GTX 1070 GPUs.
• If the Optimization.PoseGraphOptimizer test crashes, for example with an error message like "Cholesky failure", then please verify that your build of g2o has the BUILD_WITH_MARCH_NATIVE CMake option set to ON, and that BAD SLAM actually uses this install of g2o.

Build instructions for Windows

The application can be built by creating a Visual Studio 2019 solution for it with CMake, then compiling the "badslam" project in this solution.

It seemed that a workaround was required to prevent some unresolved external symbols in g2o_csparse_extension (for example, duplicating the problematic functions into g2o_solver_csparse).

Make sure to specify suitable CUDA architecture(s) in CMAKE_CUDA_FLAGS. Common settings would either be the CUDA architecture of your graphics card only (in case you only intend to run the compiled application on the system it was compiled on), or a range of virtual architectures (in case the compiled application is intended for distribution). See the corresponding CUDA documentation.

Dataset format

For CUDA block-size tuning (see below), at least one dataset should be obtained, even if one intends to run the program with live input.

The program supports datasets in the format of the ETH3D SLAM Benchmark for RGB-D videos. This is an extension of the format introduced by the TUM RGB-D benchmark, containing two small additions:

• The original format does not specify the intrinsic camera calibration. BAD SLAM thus additionally expects a file calibration.txt in the dataset directory, consisting of a single line of text structured as follows:
  复制

  fx fy cx cy

  These values specify the parameters for the pinhole projection (fx * x + cx, fy * y + cy). The coordinate system convention for cx and cy is that the origin (0, 0) of pixel coordinates is at the center of the top-left pixel in the image.
• The associate.py tool from the benchmark must be run as follows to associate the color and depth images:
  bash复制

  python associate.py rgb.txt depth.txt > associated.txt

Initial setup

After building the executable and obtaining a dataset, there are two more steps to be done before running the program.

First, the resource files for loop closure handling should be set up (unless the parameter --no_loop_detection is used to disable loop detection). Download the resource files of the DLoopDetector demo. The two relevant files from this archive, brief_k10L6.voc and brief_pattern.yml, must be extracted into a directory named "resources" in the application executable's directory (or an analogous symlink must be created), for example:

- build_RelWithDebInfo
  - applications
    - badslam
      - badslam (executable file)
      - resources
        - brief_k10L6.voc (notice that this is compressed in the archive and needs to be extracted separately)
        - brief_pattern.yml

Second, the CUDA kernel block size auto-tuning should be run. This is not strictly required in case the default sizes work for your GPU, but strongly recommended. This step serves two purposes:

• Sometimes, CUDA kernels won't launch with a given thread block size since this would require too many resources. Block size auto-tuning determines and avoids those problematic configurations.
• The best block sizes to call CUDA kernels may vary between different graphics cards, and the best way to figure them out is to benchmark it, which the tuning does.

To test your GPU, run the badslam executable with the provided tuning script on any dataset in sequential mode:

python scripts/auto_tune_parameters.py   --sequential_ba --sequential_loop_detection

The script will run the program multiple times using different parameters and measure the runtime, i.e., do not run another computing task at the same time to not influence the measurements. It should output a file auto_tuning_result.txt and intermediate files auto_tuning_iteration_X.txt. Move the result file into the resources directory used by BAD SLAM (where the loop detector resources are also stored in). The file will be loaded automatically if it exists in this directory. The intermediate files can be deleted.

Since the program runs multiple times, you may want to limit the number of frames it runs on to speed it up with --end_frame. Also, please notice that tuning data will only be gathered for CUDA kernels that run during the tuning. If later other kernels run during the actual program invocation, they will still use the default block size. So, if for example you want to tune the PCG-related kernels instead