parquet-dotnet

Apache Parquet for .NET

home + documentation
Fully managed, safe, extremely fast .NET library to read and ✍️write Apache Parquet files designed for .NET world (not a wrapper). Targets only modern .NET runtimes such as .NET 10 and .NET 8.

[!IMPORTANT] This project does not use LLMs for writing code. It will not accept LLMs for issues. No pull requests written by LLMs will be accepted. No LLMs for comments anywhere. Attempts to circumvent this may prevent your future contributions to this repository.

Features at a glance

• 0️⃣ Has zero dependencies - pure library that just works anywhere .NET works i.e. desktops, servers, phones, watches and so on.
• Really fast. Faster than Python and Java, and alternative C# implementations out there. It's even faster than native C++ implementations. Hardware acceleration is added in more and more places.
• NET native. Designed to utilise .NET and made for .NET developers, not the other way around.
• ❤️‍Not a "wrapper" that forces you to fit in. It's the other way around — forces Parquet to fit into .NET.
• Unique Features:
  • The only library that supports dynamic schemas.
  • Supports all parquet types, encodings and compressions.
  • Fully supports C# class serialization, for all simple and complex Parquet types.
  • Provides low-level, high-level, and untyped API.
  • Access to file and column metadata.
  • Fine-tune encodings per column.
  • Integration with DataFrames (Microsoft.Data.Analysis).

Used by

• Azure Cosmos DB Desktop Data Migration Tool
• RavenDB - An ACID NoSQL Document Database
• Cinchoo ETL - An ETL framework for .NET
• ParquetViewer - Simple Windows desktop application for viewing & querying Apache Parquet files
• ML.NET - Machine Learning for .NET
• PSParquet - PowerShell Module for Parquet
• Omni Loader - Self-tuning Database Migration Accelerator
• Contoso Data Generator V2 - Sample data generator
• Recfuence - An analysis of YouTube's political influence through recommendations
• Kusto-loco - C# KQL query engine with flexible I/O layers and visualization
• DeltaIO - Delta Lake implementation in pure .NET
• Personal Data Warehouse - Import(Excel/Parquet/SQL/Fabric)-Transform(C#/Python)-Report(SSRS)
• FastBCP - Export to parquet files in parallel from Oracle, SQL Server, MySQL, PostgreSQL, ODBC, Teradata, Netezza, SAP HANA, ClickHouse in one command line (Windows & Linux)
• Parquet.FSharp - Adds first-class support for F# types such as records, options, lists and discriminated unions

...raise a PR to appear here...