simdjson

开源

分析千兆字节的 JSON 每秒 : 由 Facebook/Meta Velox 使用, Node.js 运行时间 : ClickHouse, 西瓜DB, Apache Doris, Milvus, StarRocks

24.1K stars0 点赞0 次浏览

工具介绍

分析千兆字节的 JSON 每秒 : 由 Facebook/Meta Velox 使用, Node.js 运行时间 : ClickHouse, 西瓜DB, Apache Doris, Milvus, StarRocks

[!][许可证] [许可证] [许可证] [许可证] [许可证] [许可证] simdjson: 剖析千兆字节 JSON每秒. JSON在互联网上到处都是. 服务器花了很多时间分析它。我们需要一个新的方法。 simdjson库使用常用的SIMD指令和微平行算法来解析JSON 4x比RapidJSON快,为现代C++解析出25x比JSON快. Fast:比常用的生产级JSON 剖析器快了超过4x. 记录破解地物:将JSON在6GB/s被忽略,将UTF-8在13GB/s被验证,NDJSON在3.5GB/s被验证. Easy:一等,易于使用并仔细记录了API. 严格:全JSON和UTF-8验证,无损失解析. 性能没有妥协自动:在运行时选择一个 CPU 指定的解析器。不需要配置。可靠:从内存分配到错误处理,Simdjson的设计..

核心特点

•Fast: Over 4x faster than commonly used production-grade JSON parsers.
•Record Breaking Features: Minify JSON at 6 GB/s, validate UTF-8 at 13 GB/s, NDJSON at 3.5 GB/s.
•Easy: First-class, easy to use and carefully documented APIs.
•Strict: Full JSON and UTF-8 validation, lossless parsing. Performance with no compromises.
•Automatic: Selects a CPU-tailored parser at runtime. No configuration needed.
•Reliable: From memory allocation to error handling, simdjson's design avoids surprises.
•Peer Reviewed: Our research appears in venues like VLDB Journal, Software: Practice and Experience.
•Real-world usage
•Quick Start
•Documentation