百科.dev
全部条目在线工具AI 编程趋势榜开源项目技术资讯提交条目
登录
< 返回工具列表
U

UTF-unknown

> 编程语言
开源

字符集检测器 - .NET 标准与 .NET

382 stars0 点赞0 次浏览
访问官网GitHub

工具介绍

字符集检测器 - .NET 标准与 .NET

UTF Unknown

Detect character set for files, streams and other bytes.

Detection of character sets with a simple and redesigned interface.

This package is based on Ude and since version 2 also on uchardet, which are ports of the Mozilla Universal Charset Detector.

The interface and other classes have been redesigned so it's easier to use and has a better object oriented design (OOD). Unit tests and CI have been added.

Features:

  • New API
  • Moved to .NET Standard
  • Added more unit tests
  • Builds on CI (GitHub Actions)
  • Strong named
  • Documentation added
  • Multiple bugs from Ude fixed

Supported Platforms

  • .NET 8, .NET 9 and .NET 10
  • .NET Standard 2.0

Note: .NET 6 support was removed in version 2.7.0.

Remarks: You can still register your EncodingProvider so that the Encoding.GetEncoding(...) method first tries to find the encoding using it.

Usage

Use the static detectX methods from CharsetDetector.

Synchronous Methods

…

Asynchronous Methods

csharp
// Detect from File asynchronously
DetectionResult result = await CharsetDetector.DetectFromFileAsync("path/to/file.txt", cancellationToken); // or pass FileInfo

// Detect from Stream asynchronously
result = await CharsetDetector.DetectFromStreamAsync(stream, cancellationToken);

Docs

The article "A composite approach to language/encoding detection" describes the charsets detection algorithms implemented by the library.

The following charsets are supported

Encodings with BOM: utf-7, utf-8, utf-16be/utf-16le, utf-32be/utf-32le, X-ISO-10646-UCS-4-34121/X-ISO-10646-UCS-4-21431, gb18030.

Encodings without BOM are presented in the table, separated by languages:

Language Encodings
International (Unicode) utf-8
Arabic iso-8859-6, windows-1256
Bulgarian iso-8859-5, windows-1251
Chinese iso-2022-cn, big5, euc-tw, gb18030, hz-gb-2312
Croatian iso-8859-2, iso-8859-13, iso-8859-16, windows-1250, ibm852, x-mac-ce
Czech windows-1250, iso-8859-2, ibm852, x-mac-ce
Danish iso-8859-1, iso-8859-15, windows-1252
English ascii
Esperanto iso-8859-3
Estonian iso-8859-4, iso-8859-13, iso-8859-13, windows-1252, windows-1257
Finnish iso-8859-1, iso-8859-4, iso-8859-9, iso-8859-13, iso-8859-15, windows-1252
French iso-8859-1, iso-8859-15, windows-1252
German iso-8859-1, windows-1252, CP 850/IBM 00850
Greek iso-8859-7, windows-1253
Hebrew iso-8859-8, windows-1255
Hungarian iso-8859-2, windows-1250
Irish Gaelic iso-8859-1, iso-8859-9, iso-8859-15, windows-1252
Italian iso-8859-1, iso-8859-3, iso-8859-9, iso-8859-15, windows-1252
Japanese iso-2022-jp, shift-jis, euc-jp
Korean iso-2022-kr, euc-kr/uhc, cp949
Lithuanian iso-8859-4, iso-8859-10, iso-8859-13
Latvian iso-8859-4, iso-8859-10, iso-8859-13
Maltese iso-8859-3
Polish iso-8859-2, iso-8859-13, iso-8859-16, windows-1250, ibm852, x-mac-ce
Portuguese iso-8859-1, iso-8859-9, iso-8859-15, windows-1252
Romanian iso-8859-2, iso-8859-16, windows-1250, ibm852
Russian iso-8859-5, koi8-r, windows-1251, x-mac-cyrillic, ibm855, ibm866
Slovak windows-1250, iso-8859-2, ibm852, x-mac-ce
Slovene iso-8859-2, iso-8859-16, windows-1250, ibm852, x-mac-ce
Spanish iso-8859-1, iso-8859-15, windows-1252
Swedish iso-8859-1, iso-8859-4, iso-8859-9, iso-8859-15, windows-1252
Thai tis-620, iso-8859-11
Turkish iso-8859-3, iso-8859-9
Vietnamese viscii, windows-1258
Others windows-1252

Remarks: For some encodings no alias is available: cp949, iso-2022-cn, euc-tw, iso-8859-10, iso-8859-16, viscii, X-ISO-10646-UCS-4-34121/X-ISO-10646-UCS-4-21431. Some of them have been offered a suitable replacement for the return result by DetectionDetail.Encoding:

  • cp949: use ks_c_5601-1987
  • iso-2022-cn: use x-cp50227

License

The library is subject to the Mozilla Public License Version 1.1 (the "License"). Alternatively, it may be used under the terms of either the GNU General Public License Version 2 or later (the "GPL"), or the GNU Lesser General Public License Version 2.1 or later (the "LGPL").

Test data has been extracted from Wikipedia and The Project Gutenberg books and is subject to their licenses.

Issues· 0 开放

查看全部 Issues在 GitHub 打开

暂无开放 Issues,或尚未同步最近议题。

> 标签

C#character-recognitioncharacter-setscharsetcharset-detector

暂无评论,来聊聊你的看法吧

> 工具信息

发布日期2026年8月1日
最后更新2026年9月17日
分类编程语言
定价开源

> 相关工具

T
TypeScript
JavaScript 的超集,为前端与全栈提供静态类型
P
Python
通用编程语言,广泛用于 Web、数据与 AI
G
Go
Google 推出的简洁高效系统语言