OpenTracker

What is OpenTracker?

OpenTracker is an open sourced repository for Visual Tracking. It's written in C++, high speed, easy to use, and easy to be implemented in embedded system.

- AND this is not only boring Codes, 
+ It also has Maths and Implement Notes!

If you don't exactly know what this means:

Don't worry, it will be explained fully in the Notes. All the maths details of the Not-that-easy algorithms are explaned fully from the very beginning. If you have headache of reading the papers(as most of us have), this is a good tutorial. (Check Notes(draft now)).

Or, if you have problems with the implementation of a complicate cutting-edge algorithms, check this! You will get something!

Attention! OpenTracker is NOT designed just for tracking human beings as the demo images, it can track everything, even some special points!

For Multiple Object Tracker, check: OpenMultiTracker.

2018/11/06 -- New features add CMake compile support for ECO tracker. (Thanks to ou-zhi-hui)

2018/09/19 -- New features Performance tested on VOT2017 dataset!

2018/09/13 -- New features CN feature added!

2018/08/30 -- New features Support Initialize by Object Detection using Darknet and track.

2018/08/27 -- New features Support ECO API.

2018/08/24 -- New features Now ECO runs "almost" real-time on Raspberry Pi 3!

2018/08/24 -- New features Support FFTW.

2018/08/13 -- New features Speed up by multi-thread.

2018/08/09 -- New features Now it supports Raspberry Pi 3, and speed up with NEON!

2018/08/08 -- New features Speed up with NEON, speed up from ~32FPS to ~42FPS on Jetson TX2 with scale one.

2018/08/06 -- New features Speed up with SSE, speed up from ~86FPS to ~102FPS(quicker than matlab version) with scale one.

2018/07/07 -- New features OpenTracker Implement Notes draft published! Check notes/OpenTrackerNotes.pdf. Complete version is comming!

2018/07/06 -- New features Now it supports Nvidia Jetson TX1/2!

2018/07/05 -- New features Now it supports macOS!

2018/06/28 -- New features Now it supports automatic initialization with Web camera using OpenPose!

Supported tracker (more in progressing):

Supported Dataset (more in progressing):

Supported Autodetection with Web Camera

Tested Operating Systems / Platform

Performance Analysis

"ECOHCMATLAB" is the original matlab full version ECO-HC.

"ECOHCMATLABHOGCN" is the matlab version ECO-HC without fDSST scale filter.

"ECOHCMATLABHOG" is the matlab version ECO-HC without fDSST scale filter and CN feature.

"ECOCPPHOGCN" is the c++ ECO tracker in OpenTracker without fDSST scale filter.

"ECOCPPHOG" is the c++ ECO tracker in OpenTracker without CN feature and fDSST scale filter.

"KCFCPP" is the c++ KCF tracker in OpenTracker.

"NCC" is a demo tracker in vot-toolkit.

The test is on dataset VOT2017, and parameters are set exactly the same as "VOT2016_HC_settings" in matlab version. This is just for proof of validation of c++ version code, thus the parameters are not tuned for VOT2017.

You can see from the plot that, full-featured "ECOHCMATLAB" has the highest performance, "ECOCPPHOGCN" has almost the same performance with "ECOHCMATLABHOGCN", and "ECOCPPHOG" quite similar to "ECOHCMATLABHOG". And "KCFCPP" perform even better than the HOG-only ECO version, so it seems that CN feature matters.

Speed-up(without CN feature)

Speed Analysis(without CN feature)

Quick start

With quick start, you can have a quick first taste of this repository, without any panic. No need to install Caffe, CUDA etc. (But of course you have to install OpenCV 3.0 first).

OpenCV 3.0 Install on Ubuntu check this [Tutorial].

Quick Run ECO Tracker:

In eco/runecotracker.cc, make sure to choose the dataset Demo:

    string databaseType = databaseTypes[0];

Quick start -- Ubuntu

git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/eco
make -j`nproc`
sudo make install
./runecotracker.bin

Quick start -- macOS

brew install tesseract
git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/eco
make  -j`nproc`
sudo make install
./runecotracker.bin

Quick Run KCF and DSST Tracker:

In file kcf/runkcftracker.cc, make sure to choose the dataset Demo:

    string databaseType = databaseTypes[0];

Quick start -- Ubuntu

git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/kcf
make 
./runkcftracker.bin

Quick start -- macOS

brew install tesseract
git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker/kcf
make
./runkcftracker.bin

Quick Run (almost) all the tracker:

git clone https://github.com/rockkingjy/OpenTracker
cd OpenTracker
make 
sudo make install
./trackerscompare.bin

Compile and Run

For the environment settings and detailed procedures (with all the packages from the very beginning), refer to: [My DeeplearningSettings].

The only extra-package is: Opencv3.x (already installed if you follow the environment settings above).

Of course, for trackers that use Deep features, you need to install [caffe] (maybe I will use Darknet with C in the future, I like Darknet :lips: ), and change the makefile according to your path. Compile of caffe refer to : [Install caffe by makefile].

If you want to autodetection the people with web camera, you need to install [OpenPose].

Parameters setting

If you want to use Openpose, in ./makefile, set OPENPOSE=1, else set OPENPOSE=0.

Change the datasets, in inputs/readdatasets.hpp, change the number of string databaseType = databaseTypes[1];

Change the path of datasets, in inputs/readdatasets.cc, change the path to your path of data.

To use web camera with openpose

By raising your two arms higher than your nose, it will atomatically detect the person and start the tracking programme.

Run to compare all the trackers at the same time

make all
sudo make install
./trackerscompare.bin

Run ECO

Compile without Caffe

If you don't want to compile with Caffe, that means you cannot use Deep features, set in eco/makefile: USE_CAFFE=0.

If you don't want to compile with CUDA, that means you cannot use Deep features, set in eco/makefile: USE_CUDA=0.

Compile with Caffe

If you want to compile with Caffe, set in makefile and eco/makefile: USE_CAFFE=1 USE_CUDA=1, and set the according caffe path of your system in eco/makefile:

CAFFE_PATH=

Download a pretrained [VGG_CNN_M_2048.caffemodel (370 MB)], put it into folder: eco/model

If you could not download through the link above (especially for the people from Mainland China), check this [link] and download.

In eco/parameters.hpp, change the path to your path:

struct CnnParameters
{
	string proto = "/OpenTracker/eco/model/imagenet-vgg-m-2048.prototxt";
	string model = "/OpenTracker/eco/model/VGG_CNN_M_2048.caffemodel";
	string mean_file = "/OpenTracker/eco/model/VGG_mean.binaryproto";

Use CN feature

In eco/runecotracker.cc, change the path:

    parameters.useCnFeature = true;
    parameters.cn_features.fparams.tablename = "/OpenTracker/eco/look_tables/CNnorm.txt"

Speed-up with SIMD

If you are using Intel computer, in eco\makefile, set:

USE_SIMD=1

If you are using ARM like Jetson TX1/2, in eco\makefile, set:

USE_SIMD=2

If you are using ARM like Rasberrypi 3, in eco\makefile, set:

USE_SIMD=3

Speed-up with multi-thread

In eco\makefile, set:

USE_MULTI_THREAD=1

Speed-up with GPU (not yet implemented)

If you have a GPU, it can speed-up with gpu.

First don't forget to install Opencv with CUDA supported:

cmake -D OPENCV_EXTRA_MODULE_PATH=/media/elab/sdd/Amy/opencv_contrib/modules \
    -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D WITH_CUDA=ON \
    -D ENABLE_FAST_MATH=1 \
    -D CUDA_FAST_MATH=1 \
    -D WITH_CUBLAS=1 \
    ..
make -j`nproc` 
sudo make install

in eco/makefile, set:

USE_CUDA=1

Datasets settings

Change the path of your test images in eco/runecotracker.cc.

Change the datasets, in eco/runecotracker.cc, change the number of string databaseType = databaseTypes[1];.

Show heatmap

If you want to show the heatmap of the tracking, in eco/parameters.cc, change to #define DEBUG 1.

Compile and Run:

cd eco
make -j`nproc`
./runecotracker.bin

Run Opencv trackers

Change the path of your test images in kcf/opencvtrackers.cc.

cd opencv