百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
D

DanbooRegion

> 编程语言
开源

DanbooRegion: 一组插图区域数据集 (ECCV 2020)

400 stars0 点赞2 次浏览
访问官网GitHub

工具介绍

DanbooRegion: 一组插图区域数据集 (ECCV 2020)

The DanbooRegion 2020 Dataset

DanbooRegion is a project conducted by ToS2P (the Team of Style2Paints), aiming at finding a solution to extract regions from illustrations and cartoon images, so that many region-based image processing algrithoms can be applied to in-the-wild illustration and digital paintings. The main uniqueness of this project is that our dataset is created by real artists in a semi-automatic manner.

This project begins at 2019 Jan and a techinical paper, namely "DanbooRegion: An Illustration Region Dataset", is accepted to the European Conference on Computer Vision (ECCV) 2020.

- Some example illustration and the corresponding region annotations.

[Project Page] [Paper] [Zhihu]

Table of Contents

In this page, we provide the following contents:

0. Preparation.
1. Downloding the region annotation dataset.
2. Converting regions to learnable skeleton maps.
3. Converting the entire dataset to learnable skeleton maps.
4. Training a neural network to predict the skeleton maps.
5. Converting a skeleton map to regions.
6. Developing your own framework and train it on this dataset.
7. Benchmarking the performace of your model.

However, if you are busy dating your girlfriend, or in a great hurry to write a paper, you can skip the 01234567 above and directly jump to the 89 below.

8. Downloding the pre-trained model.
9. Testing the pre-trained model with an image.

We also provide some additional playful contents.

10. Playing with some applications.

0. Preparation.

Whether you want to download the dataset or the pretrained model, you will need a Python 3.6 environment with appropriate CUDA installed, and then clone the repo:

bash
git clone https://github.com/lllyasviel/DanbooRegion.git

and then install the pip environments via

bash
pip install tensorflow_gpu==1.5.0
pip install keras==2.2.4
pip install opencv-python==3.4.2
pip install numpy==1.15.4
pip install numba==0.39.0
pip install scipy==1.1.0
pip install scikit-image==0.13.0
pip install scikit-learn==0.22.2

Other versions may also works but the above versions are tested by myself.

1. Downloding the region annotation dataset.

You can download this dataset from

Google Drive:
https://drive.google.com/drive/folders/1ihLt6P7UQRlaFtZUEclXkWC9grmEXEUK?usp=sharing

For linux, you may run: (Windows does not need this step.)

cat DanbooRegion2020.zip.* > DanbooRegion2020.zip
zip -FF DanbooRegion2020.zip --out DanbooRegion2020_FF.zip
unzip DanbooRegion2020_FF.zip

or if you do not have access to google

Baidu Drive (百度网盘):
https://pan.baidu.com/s/12qGzKat48n8PyMeaYJPgOA
提取码: ts2p

After the downloading and decompression, you may see images like this:

The file structure is

../code/DanbooRegion2020
../code/DanbooRegion2020/DanbooRegion2020UNet.net
../code/DanbooRegion2020/train
../code/DanbooRegion2020/train/X.image.png
../code/DanbooRegion2020/train/X.region.png
../code/DanbooRegion2020/val
../code/DanbooRegion2020/val/X.image.png
../code/DanbooRegion2020/val/X.region.png
../code/DanbooRegion2020/test
../code/DanbooRegion2020/test/X.image.png

Below is an example of "X.image.png"

and the example of "X.region.png".

We also provide a script for you to visualize the regions by filling the original colors to all regions, you can use the script by:

bash
cd ./code/
python visualize.py ./X.image.png ./X.region.png

and you will see the visualization like this:

Something about the dataset quality

In fact we have an user study to visualize the quality of this dataset.

The x-axis is the quantity of regions in each region map, while the y-axis is the user scoring with the above scoring standard.

We can see that about 50% annotations only have normal (or below) quality and about 30% annotations are even unfinished. Besides, 70% annotations still need corrections. Only about 20% annotations look good or beautiful. This is because annotating these regions is very difficult and time-consuming. Thanks a lot to all 12 artists involved in this data annotating projects! The involved artists are:

Some artists do not want to be named and got mosaics.

Although the quality of this dataset is not perfect, we can still do lots of things that are very useful for cartoon image processing applications.

We are continuously improving the quality of this dataset. If you check back in a few months, the quality of the dataset may have improved many times.

2. Converting regions to learnable skeleton maps.

Firstly, let's say you have an image like this:

Then, you have asked some users or artists to annotate the image and get the region map like this:

The problem is that how we can train neural networks to predict the region maps? It is obvious that we cannot train a pix2pix to predict the region map because the colors in the region maps are random, and L1 loss is meaningless. Therefore, we will need some additional steps.

We first detect the edge of all regions using OpenCV and get this edge map:

After that, we use the skeletonize function to extract the region skeletons. The function can be found at

from skimage.morphology import thin as skeletonize

and the result is like this:

After that, we randomly initialize an image with skeletons and edges like this:

Then, we smooth the above image to get the final skeleton map like this:

Note that this skeleton map is learnable and you can use anything to learn it. For example, you can train pix2pix or pix2pixHD to predict the skeleton map.

Also, the skeleton map can be translated to normal maps like this:

Or you can thresold the skeleton map to get some watershed markers like this:

Note that we provide full codes for you to do all the above things! Just run

bash
cd ./code/
python skeletonize.py ./region_test.png

Have fun with the codes.

3. Converting the entire dataset to learnable skeleton maps.

Using the above transform, you can translate the full training data into skeleton maps. Just run

bash
cd ./code/
python skeletonize_all.py

This will automatically generate all training region skeletons, and the logs are like this:

Writing   ./DanbooRegion2020/train/0.skeleton.png   1/3377
Writing   ./DanbooRegion2020/train/1.skeleton.png   2/3377
Writing   ./DanbooRegion2020/train/2.skeleton.png   3/3377
Writing   ./DanbooRegion2020/train/3.skeleton.png   4/3377
Writing   ./DanbooRegion2020/train/4.skeleton.png   5/3377
Writing   ./DanbooRegion2020/train/5.skeleton.png   6/3377
...
Writing   ./DanbooRegion2020/train/3376.skeleton.png   3377/3377

Before running this script, the file structures are

../code/DanbooRegion2020
../code/DanbooRegion2020/train
../code/DanbooRegion2020/train/X.image.png
../code/DanbooRegion2020/train/X.region.png

After this script, the file structures become

../code/DanbooRegion2020
../code/DanbooRegion2020/train
../code/DanbooRegion2020/train/X.image.png
../code/DanbooRegion2020/train/X.region.png
../code/DanbooRegion2020/train/X.skeleton.png

And this is a screenshot after all skeletons are generated:

Note that the selected files in the above screenshot are the generated skeleton maps. Now you have prepared everything.

4. Training a neural network to predict the skeleton maps.

Before training the model, make sure that you have followed the instructions in 3. Converting the entire dataset to learnable skeleton maps.

We provide full codes for you to train a model on this dataset:

bash
cd ./code/
python train.py

We note that:

  • If you run this script in the first time, it will generate a new model and train it directly.
  • During training, the model will be saved in each 200 iterations.
  • If you run this script when a saved model is avaliable, it will read the previous model and continue training.
  • If you do not want to continue the training, please delete the "saved_model.net" and "saved_discriminator.net" file.
  • This code will train a U-net with adversarial and perceptural L1 loss, which has similar performance with pix2pixHD.
  • The script uses many data augmentation methods. Please see the code for details.
  • The training is not very expensive. You may even train it on a laptop with GTX980M.

During the training, the script will save the neural network estimated skeleton maps for each 10 iterations. The saved estimations are in the path:

../code/results
../code/results/0.jpg
../code/results/1.jpg
../code/results/2.jpg
...

The entire training will take 50000 iterations.

On my very old laptop (GTX980M+I7CPU), the successful logs are

Using TensorFlow backend.
2020-07-19 20:44:39.820506: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
2020-07-19 20:44:40.423216: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1030] Found device 0 with properties: 
name: GeForce GTX 980M major: 5 minor: 2 memoryClockRate(GHz): 1.1265
pciBusID: 0000:01:00.0
totalMemory: 8.00GiB freeMemory: 6.71GiB
2020-07-19 20:44:40.423930: I C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\36\tensorflow\core\common_runtime\gpu\gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 980M, pci bus id: 0000:01:00.0, compute capability: 5.2)
saved_model.net--created
saved_discriminator.net--created
0/50000 Currently 13.778525352478027 second for each iteration.
1/50000 Currently 2.0006484985351562 second for each iteration.
2/50000 Currently 1.6455974578857422 second for each iteration.
3/50000 Currently 1.8700015544891357 second for each iteration.
4/50000 Currently 1.8167574405670166 second for each iteration.
5/50000 Currently 1.5877196788787842 second for each iteration.
6/50000 Currently 1.6236474514007568 second for each iteration.
7/50000 Currently 1.7862193584442139 second for each iteration.

If you have better GPU than me, your speed should be much faster.

5. Converting a skeleton map to regions.

Given the skeleton map like this:

We can translate them to regions using the function of cv2.watershed as

watershed(image, markers) -> markers
.   @brief Performs a marker-based image segmentation using the watershed algorithm.
.   
.   The function implements one of the variants of watershed, non-parametric marker-based segmentation
.   algorithm, described in @cite Meyer92 .
.   
.   Before passing the image to the function, you have to roughly outline the desired regions in the
.   image markers with positive (\>0) indices. So, every region is represented as one or more connected
.   components with the pixel values 1, 2, 3, and so on. Such markers can be retrieved from a binary
.   mask using #findContours and #drawContours (see the watershed.cpp demo). The markers are "seeds" of
.   the future image regions. All the other pixels in markers , whose relation to the outlined regions
.   is not known and should be defined by the algorithm, should be set to 0's. In the function output,
.   each pixel in markers is set to a value of the "seed" components or to -1 at boundaries between the
.   regions.
.   
.   @note Any two neighbor connected components are not necessarily separated by a watershed boundary
.   (-1's pixels); for example, they can touch each other in the initial marker image passed to the
.   function.
.   
.   @param image Input 8-bit 3-channel image.
.   @param markers Input/output 32-bit single-

Issues· 3 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

Python

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言