字符集检测器 - .NET 标准与 .NET
Detect character set for files, streams and other bytes.
Detection of character sets with a simple and redesigned interface.
This package is based on Ude and since version 2 also on uchardet, which are ports of the Mozilla Universal Charset Detector.
The interface and other classes have been redesigned so it's easier to use and has a better object oriented design (OOD). Unit tests and CI have been added.
Features:
Note: .NET 6 support was removed in version 2.7.0.
Remarks:
You can still register your EncodingProvider
so that the Encoding.GetEncoding(...) method first tries to find the encoding using it.
Use the static detectX methods from CharsetDetector.
…// Detect from File asynchronously
DetectionResult result = await CharsetDetector.DetectFromFileAsync("path/to/file.txt", cancellationToken); // or pass FileInfo
// Detect from Stream asynchronously
result = await CharsetDetector.DetectFromStreamAsync(stream, cancellationToken);
The article "A composite approach to language/encoding detection" describes the charsets detection algorithms implemented by the library.
Encodings with BOM: utf-7, utf-8, utf-16be/utf-16le, utf-32be/utf-32le, X-ISO-10646-UCS-4-34121/X-ISO-10646-UCS-4-21431, gb18030.
Encodings without BOM are presented in the table, separated by languages:
| Language | Encodings |
|---|---|
| International (Unicode) | utf-8 |
| Arabic | iso-8859-6, windows-1256 |
| Bulgarian | iso-8859-5, windows-1251 |
| Chinese | iso-2022-cn, big5, euc-tw, gb18030, hz-gb-2312 |
| Croatian | iso-8859-2, iso-8859-13, iso-8859-16, windows-1250, ibm852, x-mac-ce |
| Czech | windows-1250, iso-8859-2, ibm852, x-mac-ce |
| Danish | iso-8859-1, iso-8859-15, windows-1252 |
| English | ascii |
| Esperanto | iso-8859-3 |
| Estonian | iso-8859-4, iso-8859-13, iso-8859-13, windows-1252, windows-1257 |
| Finnish | iso-8859-1, iso-8859-4, iso-8859-9, iso-8859-13, iso-8859-15, windows-1252 |
| French | iso-8859-1, iso-8859-15, windows-1252 |
| German | iso-8859-1, windows-1252, CP 850/IBM 00850 |
| Greek | iso-8859-7, windows-1253 |
| Hebrew | iso-8859-8, windows-1255 |
| Hungarian | iso-8859-2, windows-1250 |
| Irish Gaelic | iso-8859-1, iso-8859-9, iso-8859-15, windows-1252 |
| Italian | iso-8859-1, iso-8859-3, iso-8859-9, iso-8859-15, windows-1252 |
| Japanese | iso-2022-jp, shift-jis, euc-jp |
| Korean | iso-2022-kr, euc-kr/uhc, cp949 |
| Lithuanian | iso-8859-4, iso-8859-10, iso-8859-13 |
| Latvian | iso-8859-4, iso-8859-10, iso-8859-13 |
| Maltese | iso-8859-3 |
| Polish | iso-8859-2, iso-8859-13, iso-8859-16, windows-1250, ibm852, x-mac-ce |
| Portuguese | iso-8859-1, iso-8859-9, iso-8859-15, windows-1252 |
| Romanian | iso-8859-2, iso-8859-16, windows-1250, ibm852 |
| Russian | iso-8859-5, koi8-r, windows-1251, x-mac-cyrillic, ibm855, ibm866 |
| Slovak | windows-1250, iso-8859-2, ibm852, x-mac-ce |
| Slovene | iso-8859-2, iso-8859-16, windows-1250, ibm852, x-mac-ce |
| Spanish | iso-8859-1, iso-8859-15, windows-1252 |
| Swedish | iso-8859-1, iso-8859-4, iso-8859-9, iso-8859-15, windows-1252 |
| Thai | tis-620, iso-8859-11 |
| Turkish | iso-8859-3, iso-8859-9 |
| Vietnamese | viscii, windows-1258 |
| Others | windows-1252 |
Remarks:
For some encodings no alias is available: cp949, iso-2022-cn, euc-tw, iso-8859-10, iso-8859-16, viscii, X-ISO-10646-UCS-4-34121/X-ISO-10646-UCS-4-21431. Some of them have been offered a suitable replacement for the return result by DetectionDetail.Encoding:
cp949: use ks_c_5601-1987iso-2022-cn: use x-cp50227The library is subject to the Mozilla Public License Version 1.1 (the "License"). Alternatively, it may be used under the terms of either the GNU General Public License Version 2 or later (the "GPL"), or the GNU Lesser General Public License Version 2.1 or later (the "LGPL").
Test data has been extracted from Wikipedia and The Project Gutenberg books and is subject to their licenses.
暂无开放 Issues,或尚未同步最近议题。