chia-plotter

chia-plotter (pipelined multi-threaded)

This is a new implementation of a chia plotter which is designed as a processing pipeline, similar to how GPUs work, only the "cores" are normal software CPU threads.

As a result this plotter is able to fully max out any storage device's bandwidth, simply by increasing the number of "cores", ie. threads.

Sponsored by Flexpool.io - Check them out if you're looking for a secure and scalable Chia pool.

Usage

Join the Discord for support: https://discord.gg/BswFhNkMzY

…

Make sure to crank up <threads> if you have plenty of cores, the default is 4. Depending on the phase more threads will be launched, the setting is just a multiplier.

RAM usage depends on <threads> and <buckets>. With the new default of 256 buckets it's about 0.5 GB per thread at most.

-G option will alternate the temp dirs used while plotting to give each one, tmpdir and tmpdir2, equal usage. The first plot creation will use tmpdir and tmpdir2 as expected. The next run, if -n equals 2 or more, will swap the order to tmpdir2 and tmpdir. The next run swaps again to tmpdir and tmpdir2. This will occur until the number of plots created is reached or until stopped.

RAM disk setup on Linux

sudo mount -t tmpfs -o size=110G tmpfs /mnt/ram/

Note: 128 GiB System RAM minimum required for RAM disk.

How to Support

XCH: xch1w5c2vv5ak08pczeph7tp5xmkl5762pdf3pyjkg9z4ks4ed55j3psgay0zh

XFX: xfx1succfn2z3uwmq50ukztjanrvs9kw294mqn4lv22rk6tzmcu7e2xsyxyaa5

XCC: xcc16j65y35fs8u289nq6krcyehsmp5eqd4we493rxf36pg7eymcqrqqltsrat

Results

On a dual Xeon® E5-2650v2@2.60GHz R720 with 256GB RAM and a 3x800GB SATA SSD RAID0, using a 110G tmpfs for <tmpdir2>:

…

How to Verify

To make sure the plots are valid you can use the ProofOfSpace tool from chiapos:

git clone https://github.com/Chia-Network/chiapos.git
cd chiapos && mkdir build && cd build && cmake .. && make -j8
./ProofOfSpace check -f plot-k32-???.plot [num_iterations]

How to update to latest version

cd chia-plotter
git checkout master
git pull
git submodule update --init
./make_devel.sh

Future Plans

I do have some history with GPU mining, back in 2014 I was the first to open source a XPM GPU miner, which was about 40x more efficient than the CPU miner. See my other repos.

As such, it's only a matter of time until I add OpenCL support to speed up the plotter even more, keeping most of the load off the CPUs.

Dependencies

• cmake (>=3.14)
• libsodium-dev

Install

Binaries built by stotiks can be found here: https://github.com/stotiks/chia-plotter/releases

First, install dependencies from pacman:

sudo pacman -S cmake libsodium gmp gcc11

Then, clone and compile the project:

# Checkout the source and install
git clone https://github.com/madMAx43v3r/chia-plotter.git
cd chia-plotter

git submodule update --init
./make_devel.sh
./build/chia_plot --help

…

sudo swupd update
sudo swupd bundle-add c-basic devpkg-libsodium git wget

echo PATH=$PATH:/usr/local/bin/ # for statically compiled cmake if not already in your PATH

# Install libsodium
cd /tmp
wget https://download.libsodium.org/libsodium/releases/LATEST.tar.gz
tar -xvf LATEST.tar.gz
cd libsodium-stable
./configure
make && make check
sudo make install
# Checkout the source and install
cd ~/
git clone https://github.com/madMAx43v3r/chia-plotter.git 
cd ~/chia-plotter
git submodule update --init
./make_devel.sh
./build/chia_plot --help

sudo apt install -y libsodium-dev cmake g++ git build-essential
# Checkout the source and install
git clone https://github.com/madMAx43v3r/chia-plotter.git 
cd chia-plotter

git submodule update --init
./make_devel.sh
./build/chia_plot --help

The binaries will end up in build/, you can copy them elsewhere freely (on the same machine, or similar OS).

Make sure to add buster-backports to your sources.list otherwise the installation will fail because an older cmake version. See the debian backport documentation for reference.

# Install cmake 3.16 from buster-backports
sudo apt install -t buster-backports cmake
sudo apt install -y libsodium-dev g++ git
# Checkout the source and install
git clone https://github.com/madMAx43v3r/chia-plotter.git 
cd chia-plotter

git submodule update --init
./make_devel.sh
./build/chia_plot --help

The binaries will end up in build/, you can copy them elsewhere freely (on the same machine, or similar OS).

First you need to install the Brew package manager and Xcode OR Xcode Command Line Tools.

# Download Xcode Command Line Tools (skip if you already have Xcode)
xcode-select --install

# Now download chia-plotter's dependencies
brew install libsodium cmake git autoconf automake libtool wget
brew link cmake
git clone https://github.com/madMAx43v3r/chia-plotter.git 
cd chia-plotter
git submodule update --init
./make_devel.sh
./build/chia_plot --help

Linking libsodium should be performed automatically, but in case of failure please follow these instructions:

# If you downloaded Xcode run these:
sudo ln -s /usr/local/include/sodium.h /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/
sudo ln -s /usr/local/include/sodium /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/

# If you downloaded CommandLineTools run these:
sudo ln -s /usr/local/include/sodium.h /Library/Developer/CommandLineTools/usr/include
sudo ln -s /usr/local/include/sodium /Library/Developer/CommandLineTools/usr/include

Confirm which directory you have on YOUR Mac before applying following commands

# For x86_64 Macs
wget https://raw.githubusercontent.com/facebookincubator/fizz/master/build/fbcode_builder/CMake/FindSodium.cmake -O /usr/local/opt/cmake/share/cmake/Modules/FindSodium.cmake

or

# For ARM64 (M1) Macs
wget https://raw.githubusercontent.com/facebookincubator/fizz/master/build/fbcode_builder/CMake/FindSodium.cmake -O /opt/homebrew/Cellar/cmake/*/share/cmake/Modules/FindSodium.cmake

If a maximum open file limit error occurs (as default OS setting is 256, which is too low for default bucket size of 256), run this before starting the plotter

ulimit -n 3000

This file limit change will only affect the current session.

In some setups and scenarios, it could be useful to run your plotter inside a Docker container. This could be potentially useful while running chia-plotter in Windows.

To do so, install Docker on your computer and them run the following command:

docker run \
  -v <path-to-your-tmp-dir>:/mnt/harvester \
  -v <path-to-your-final-dir>:/mnt/farm \
  odelucca/chia-plotter \
    -t /mnt/harvester/ \
    -d /mnt/farm/ \
    -p <pool-key> \
    -f <farm-key> \
    -r <number-of-CPU-cores>

You can provide any of the plotter arguments after the image name (odelucca/chia-plotter)

In a Linux benchmark, we were able to find that running in Docker is only 5% slower than running in native OS.

For Windows users, you should check if your Docker configuration has any RAM or CPU limits. Since Docker runs inside HyperV, that could potentially constrain your hardware usage. In any case, you can set the RAM limits with the -m flag (after the docker run command).

Regarding multithread in Docker

While running in Windows, you may need to proper configure your Docker to allow multi CPUs. You can do so by following this article

In a nutshell, you could also pass the --cpus flag to your docker run command in order to achieve the same result.

So, for example, the following command...

docker run \
  -v <path-to-your-tmp-dir>:/mnt/harvester \
  -v <path-to-your-final-dir>:/mnt/farm \
  -m 8G \
  --cpus 8 \
  odelucca/chia-plotter \
    -t /mnt/harvester/ \
    -d /mnt/farm/ \
    -p <pool-key> \
    -f <farm-key> \
    -r 8

...would run your plotter with 8 CPU cores and 8GB of RAM.

Building a Docker container

Make sure your submodules are up-to-date by running git submodule update --init, then simply build with docker build .

Known Issues

• Needs at least cmake 3.14 (because of bls-signatures)

How to install latest cmake on Ubuntu 18.04