zbox

ZboxFS

ZboxFS is a zero-details, privacy-focused in-app file system. Its goal is to help application store files securely, privately and reliably. By encapsulating files and directories into an encrypted repository, it provides a virtual file system and exclusive access to authorised application.

Unlike other system-level file systems, such as [ext4], [XFS] and [Btrfs], which provide shared access to multiple processes, ZboxFS is a file system that runs in the same memory space as the application. It provides access to only one process at a time.

By abstracting IO access, ZboxFS supports a variety of underlying storage layers, including memory, OS file system, RDBMS and key-value object store.

Disclaimer

ZboxFS is under active development, we are not responsible for any data loss or leak caused by using it. Always back up your files and use at your own risk!

Features

• Everything is encrypted :lock:, including metadata and directory structure, no knowledge can be leaked to underlying storage
• State-of-the-art cryptography: AES-256-GCM (hardware), XChaCha20-Poly1305, Argon2 password hashing and etc., powered by [libsodium]
• Support varieties of underlying storages, including memory, OS file system, RDBMS, Key-value object store and more
• Files and directories are packed into same-sized blocks to eliminate metadata leakage
• Content-based data chunk deduplication and file-based deduplication
• Data compression using [LZ4] in fast mode, optional
• Data integrity is guaranteed by authenticated encryption primitives (AEAD crypto)
• File contents versioning
• Copy-on-write (COW :cow:) semantics
• ACID transactional operations
• Built with [Rust] :hearts:

Comparison

Many OS-level file systems support encryption, such as [EncFS], [APFS] and [ZFS]. Some disk encryption tools also provide virtual file system, such as [TrueCrypt], [LUKS] and [VeraCrypt].

This diagram shows the difference between ZboxFS and them.

Below is the feature comparison list.

ZboxFS OS-level File Systems Disk Encryption Tools Encrypts file contents :heavy_check_mark: partial :heavy_check_mark: Encrypts file metadata :heavy_check_mark: partial :heavy_check_mark: Encrypts directory :heavy_check_mark: partial :heavy_check_mark: Data integrity :heavy_check_mark: partial :heavy_multiplication_x: Shared access for processes :heavy_multiplication_x: :heavy_check_mark: :heavy_check_mark: Deduplication :heavy_check_mark: :heavy_multiplication_x: :heavy_multiplication_x: Compression :heavy_check_mark: partial :heavy_multiplication_x: Content versioning :heavy_check_mark: :heavy_multiplication_x: :heavy_multiplication_x: COW semantics :heavy_check_mark: partial :heavy_multiplication_x: ACID Transaction :heavy_check_mark: :heavy_multiplication_x: :heavy_multiplication_x: Varieties of storages :heavy_check_mark: :heavy_multiplication_x: :heavy_multiplication_x: API access :heavy_check_mark: through VFS through VFS Symbolic links :heavy_multiplication_x: :heavy_check_mark: depends on inner FS Users and permissions :heavy_multiplication_x: :heavy_check_mark: :heavy_check_mark: FUSE support :heavy_multiplication_x: :heavy_check_mark: :heavy_check_mark: Linux and macOS support :heavy_check_mark: :heavy_check_mark: :heavy_check_mark: Windows support :heavy_check_mark: partial :heavy_check_mark:

Supported Storage

ZboxFS supports a variety of underlying storages. Memory storage is enabled by default. All the other storages can be enabled individually by specifying its corresponding Cargo feature when building ZboxFS.

Storage URI identifier Cargo Feature Memory "mem://" N/A OS file system "file://" storage-file SQLite "sqlite://" storage-sqlite Redis "redis://" storage-redis Zbox Cloud Storage "zbox://" storage-zbox-native

* Visit zbox.io to learn more about Zbox Cloud Storage.

Specs

Algorithm and data structure Value Authenticated encryption AES-256-GCM or XChaCha20-Poly1305 Password hashing Argon2 Key derivation BLAKE2B Content dedup Rabin rolling hash File dedup Merkle tree Index structure Log-structured merge-tree Compression LZ4 in fast mode

Limits

Limit Value Data block size 8 KiB Maximum encryption frame size 128 KiB Super block size 8 KiB Maximum filename length No limit Allowable characters in directory entries Any UTF-8 character except / Maximum pathname length No limit Maximum file size 16 EiB Maximum repo size 16 EiB Max number of files No limit

Metadata

Metadata Value Stores file owner No POSIX file permissions No Creation timestamps Yes Last access / read timestamps No Last change timestamps Yes Access control lists No Security Integrated with crypto Extended attributes No

Capabilities

Capability Value Hard links No Symbolic links No Case-sensitive Yes Case-preserving Yes File Change Log By content versioning Filesystem-level encryption Yes Data deduplication Yes Data checksums Integrated with crypto Offline grow No Online grow Auto Offline shrink No Online shrink Auto

Allocation and layout policies

Feature Value Address allocation scheme Append-only, linear address space Sparse files No Transparent compression Yes Extents No Copy on write Yes

Storage fragmentation

Fragmentation Value Memory storage No File storage fragment unit size < 32 MiB RDBMS storage No Key-value storage No Zbox cloud storage fragment unit size < 128 KiB

How to use

For reference documentation, please visit documentation.

Requirements

• [Rust] stable >= 1.38
• [libsodium] >= 1.0.17

Supported Platforms

• 64-bit Debian-based Linux, such as Ubuntu
• 64-bit macOS
• 64-bit Windows
• 64-bit Android, API level >= 21

32-bit and other OS are NOT supported yet.

Usage

Add the following dependency to your Cargo.toml:

[dependencies]
zbox = "0.9.2"

If you don't want to install libsodium by yourself, simply specify libsodium-bundled feature in dependency, which will automatically download, verify and build libsodium.

[dependencies]
zbox = { version = "0.9.2", features = ["libsodium-bundled"] }

Example

…

Build with Docker

ZboxFS comes with [Docker] support, which made building ZboxFS easier. Check each repo for more details.

• [zboxfs/base] Base image for building ZboxFS on Linux

• [zboxfs/wasm] Docker image for building WebAssembly binding

• [zboxfs/nodejs] Docker image for building Node.js binding

• [zboxfs/android] Docker image for building Android Java binding

Static linking with libsodium

By default, ZboxFS uses dynamic linking when it is linked with libsodium. If you want to change this behavior and use static linking, you can enable below two environment variables.

On Linux/macOS,

export SODIUM_LIB_DIR=/path/to/your/libsodium/lib
export SODIUM_STATIC=true

On Windows,

set SODIUM_LIB_DIR=C:\path\to\your\libsodium\lib
set SODIUM_STATIC=true

And then re-build the code.

cargo build

Performance

The performance test is run on a Macbook Pro 2017 laptop with spec as below.

Spec Value Processor Name: Intel Core i7 Processor Speed: 3.5 GHz Number of Processors: 1 Total Number of Cores: 2 L2 Cache (per Core): 256 KB L3 Cache: 4 MB Memory: 16 GB OS Version: macOS High Sierra 10.13.6

Test result:

Read Write TPS Baseline (memcpy): 3658.23 MB/s 3658.23 MB/s N/A Baseline (file): 1307.97 MB/s 2206.30 MB/s N/A Memory