PHP 类,用于从任何自由文本中检测语言
PHP Class to detect languages from any free text.
It follows the approach described in the paper, a given text is tokenized into N-Grams (we cleanup whitespaces before doing this step). Then we sort the tokens and we compare against a language model.
The first thing we need is a language model (which looks like this file) that is used to compare the texts against at classification time. This process must done before anything, and it can be generated with an script similar to this file.
…Once we have our language model file (in this case language.php) we're ready to classify texts by their language.
…And that's it.
The project is designed to work with modules, which means you can provide your own algorithm for sorting and comparing the N-Grams. By default the library implements the PageRank as sorting algorithm, and out of place (described in the paper) as comparing.
In order to supply your own algorithms, you must change the $config at learning stage to load your own classes (which by the way should implement some interaces).
暂无开放 Issues,或尚未同步最近议题。