XKCP

What is the XKCP?

The eXtended Keccak Code Package (or the Xoodoo and Keccak Code Package, in both cases abbreviated as XKCP) is a repository that gathers different free and open-source implementations of the cryptographic schemes defined by the Keccak team. This includes the Keccak sponge function family and closely related variants, such as

• the SHAKE extendable-output functions and SHA-3 hash functions from [FIPS 202][fips202_standard],
• the cSHAKE, KMAC, ParallelHash and TupleHash functions from [NIST SP 800-185][sp800_185_standard],
• the fast [TurboSHAKE][turboshake] and [KangarooTwelve][k12] extendable-output functions,
• the Kravatte pseudo-random function and its modes,
• the SHAKE- and TurboSHAKE-Wrap and -BO authenticated encryption schemes,

as well as the Xoodoo permutation and

• the Xoofff pseudo-random function and its modes (experimental),
• the Xoodyak scheme (submission to the NIST lightweight crypto standardization process).

The code in this repository can be built as a library called libXKCP.

Note that we decided to remove Ketje and Keyak from the XKCP.

What is libXKCP?

libXKCP is a library that contains all the Keccak and Xoodoo-based cryptographic schemes mentioned above.

Before building, please make sure that the submodules have been initialized and fetched using git submodule update --init. Then, to build libXKCP, the quick answer is to launch:

make /libXKCP.so

where `` is to be replaced with the actual target (e.g., x86-64 or ARMv6M), and where .so can be replaced with .a for a static library or with .dylib for a dynamic library on macOS. More details, and in particular the list of targets, can be found in the section on how to build the XKCP below.

If your compiler supports it, you may add EXTRA_CFLAGS="-march=native -mtune=native" at the end of the command line so that the code is further optimized for the platform on which it is compiled.

More precisely, what does the XKCP contain?

First, the services available in this package are divided into high-level and low-level services. In a nutshell, the low level corresponds to Keccak-f[1600] and basic state manipulation, while the high level contains the constructions and the modes for, e.g., sponge functions, hashing or authenticated encryption. For more details, please see the section "How is the code organized?" below.

Second, these high-level and low-level services can be compiled as the libXKCP library.

Then, the XKCP also contains some utilities for testing, benchmarking and illustration purposes.

Finally, the repository contains some standalone implementations.

High-level services

When used as a library or directly from the sources, the XKCP offers the high-level services documented in the following header files:

• SimpleFIPS202, the six approved FIPS 202 instances (SHAKE128, SHAKE256 and the SHA-3 hash functions) through simple functions.
• KeccakHash, the six approved FIPS 202 instances, as well as any Keccak instance based on Keccak-f[1600]. This more advanced interface proposes a message queue (init-update-final) and supports bit-level inputs if needed.
• SP800-185, the functions (cSHAKE, KMAC, ParallelHash and TupleHash) in the official NIST SP 800-185 standard.
• KeccakSponge, all Keccak sponge functions, with or without a message queue.
• KeccakDuplex, all Keccak duplex objects.
• KeccakOD, all Keccak overwrite duplex (OD) objects.
• KeccakPRG, a pseudo-random number generator based on Keccak duplex objects.
• TurboSHAKE, the fast twelve-round variant to Keccak.
• KangarooTwelve, the fast and parallelizable hashing mode based on TurboSHAKE and Sakura coding.
• ShakingUpAE, the SHAKE- and TurboSHAKE-Wrap and -BO authenticated encryption schemes.
• Kravatte and KravatteModes, the pseudo-random function Kravatte, as well as the modes on top of it (SANE, SANSE, WBC and WBC-AE).
• Xoofff and XoofffModes, the pseudo-random function Xoofff, as well as the modes on top of it (SANE, SANSE, WBC and WBC-AE).
• Xoodyak, the lightweight cryptographic scheme Xoodyak that can be used for hashing, encryption, MAC computation and authenticated encryption.

Low-level services

The low-level services implement the different permutations Keccak-f[1600], Keccak-p[1600, 12 rounds] and Xoodoo.

The low-level services provide an opaque representation of the state together with functions to add data into and extract data from the state. Together with the permutations themselves, the low-level services implement what we call the state and permutation interface (abbreviated SnP). For parallelized implementation, we similarly use the parallel state and permutation interface or PlSnP.

• In lib/low/, one can find implementations of the following permutations for different platforms.

  • lib/low/KeccakP-1600/, for Keccak-p[1600]. This is the one used in the six approved FIPS 202 instances.
  • lib/low/Xoodoo/, for Xoodoo.

• In addition, one can find the implementation of parallelized permutations using SIMD instructions.

In both cases, the hierarchy first selects a permutation (or a permutation and a degree of parallelism) and then a given implementation. E.g., one finds in lib/low/KeccakP-1600-times4/ the implementations of 4 parallel instances of Keccak-p[1600] and in lib/low/KeccakP-1600-times4/AVX2/ a 256-bit SIMD implementation for AVX2.

The documentation of the low-level services can be found in SnP-documentation.h and PlSnP-documentation.h.

Utilities

The package contains:

• The libXKCP library;
• Self-tests that ensure that the implementation is working properly;
• A benchmarking tool to measure the timing of the various schemes;
• KeccakSum that computes a hash of the file (or multiple files) given in parameter.

Note that, to run the benchmarks on ARM processors, you may need to include the Kernel-PMU module.

Standalone implementations

The XKCP also provides some standalone implementations, including:

• a very compact C code of the FIPS 202 (SHA-3) standard in Standalone/CompactFIPS202/C/;
• a compact implementation in Python in Standalone/CompactFIPS202/Python/.

Is there example code?

Yes, there is example code for using many cryptographic functions of the XKCP. You can find them in the usage-example.md file.

Under which license is the XKCP distributed?

Most of the source and header files in the XKCP are released to the public domain and associated to the CC0 deed, but there are exceptions. Please refer to the LICENSE file for more information.

How can I build the XKCP?

To build on Linux or macOS, the following tools are needed:

• GCC or clang
• GNU make
• xsltproc

The different targets are defined in Makefile.build. This file is expanded into a regular makefile using xsltproc. To use it, simply type, e.g.,

make generic64/UnitTests

or

make x86-64/Benchmarks

to build UnitTests using plain 64-bit code or to build the Benchmarks tool with x86-64 code. The name before the slash indicates the target, i.e., the platform or instruction set used, while the part after the slash is the executable or library to build. As another example, the static (resp. dynamic) library is built by typing make ARMv7M/libXKCP.a (resp. .so) or similarly with ARMv7M replaced with the appropriate platform or instruction set name. An alternate C compiler can be specified via the CC environment variable.

At the time of this writing, the possible target names before the slash are:

• x86-64: automatic runtime selection among 64-bit plain C and SSSE3, AVX2 and AVX-512 instruction sets (recommended for x86-64 platforms);
• compact: plain C compact implementations;
• generic32: plain C implementation, generically optimized for 32-bit platforms;
• generic32lc: same as generic32 but featuring the lane complementing technique for platforms without a "and not" instruction;
• generic64: plain C implementation, generically optimized for 64-bit platforms;
• generic64lc: same as generic64 but featuring the lane complementing technique for platforms without a "and not" instruction;
• SSSE3: implementations selected for the processors that support the SSSE3 instruction set;
• AVX: implementations selected for processors that support the AVX instruction set (e.g., Sandy Bridge microarchitectures);
• XOP: implementations selected for processors that support the XOP instruction set (e.g., Bulldozer microarchitecture);
• AVX2: implementations selected for processors that support the AVX2 instruction set (e.g., Haswell and Skylake microarchitectures);
• AVX512: implementations selected for the processors that support the AVX-512 instruction set (e.g., SkylakeX microarchitecture);
• ARMv6: implementations selected for processors with the ARMv6 architecture;
• ARMv6M: implementations selected for processors with the ARMv6-M architecture;
• ARMv7M: implementations selected for processors with the ARMv7-M architecture;
• ARMv7A: implementations selected for processors with the ARMv7-A architecture;
• ARMv8A: implementations selected for processors with the ARMv8-A architecture;
• AVR8: implementations selected for processors with the 8-bit AVR architecture.

If your compiler supports it, you may add EXTRA_CFLAGS="-march=native -mtune=native" at the end of the command line so that the code is further optimized for the platform on which it is compiled.

Instead of building an executable with GCC, one can choose to select the files needed and make a package. For this, simply append .pack to the target name, e.g.,

make x86-64/UnitTests.pack

This creates a .tar.gz archive with all the necessary files to build the given target.

The list of targets can be found at the end of Makefile.build or by running make without parameters.

Microsoft Visual Studio support

The XKCP can be compiled with Microsoft Visual Studio (MSVC). The XKCP build system offers support for the creation of project files. To get a project file for a given target, simply append .vcxproj to the target name, e.g.,

make AVX512noAsm/KeccakSum.vcxproj

As of today, please note the current limitations:

• The assembly code, as used in some targets, follows the GCC syntax and at this point cannot be used directly with MSVC. Note that the AVX2noAsm and AVX512noAsm targets provide alternatives to AVX2 and AVX512, respectively, without assembly implementations.
• There is no support yet to build a dynamic library like libXKCP.dll. However, we