Search in sources :

Example 1 with ViterbiSegment

use of com.hankcs.hanlp.seg.Viterbi.ViterbiSegment in project HanLP by hankcs.

the class TestSegment method testJP.

public void testJP() throws Exception {
    String text = "明天8.9你好abc对了";
    Segment segment = new ViterbiSegment().enableCustomDictionary(false).enableAllNamedEntityRecognize(false);
    System.out.println(segment.seg(text));
}
Also used : ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment) Segment(com.hankcs.hanlp.seg.Segment) DoubleArrayTrieSegment(com.hankcs.hanlp.seg.Other.DoubleArrayTrieSegment) CRFSegment(com.hankcs.hanlp.seg.CRF.CRFSegment) DijkstraSegment(com.hankcs.hanlp.seg.Dijkstra.DijkstraSegment) ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment)

Example 2 with ViterbiSegment

use of com.hankcs.hanlp.seg.Viterbi.ViterbiSegment in project HanLP by hankcs.

the class TestSegment method testIssue290.

public void testIssue290() throws Exception {
    //        HanLP.Config.enableDebug();
    String txt = "而其他肢解出去的七个贝尔公司如西南贝尔、太平洋贝尔、大西洋贝尔。";
    Segment seg_viterbi = new ViterbiSegment().enablePartOfSpeechTagging(true).enableOffset(true).enableNameRecognize(true).enablePlaceRecognize(true).enableOrganizationRecognize(true).enableNumberQuantifierRecognize(true);
    System.out.println(seg_viterbi.seg(txt));
}
Also used : ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment) Segment(com.hankcs.hanlp.seg.Segment) DoubleArrayTrieSegment(com.hankcs.hanlp.seg.Other.DoubleArrayTrieSegment) CRFSegment(com.hankcs.hanlp.seg.CRF.CRFSegment) DijkstraSegment(com.hankcs.hanlp.seg.Dijkstra.DijkstraSegment) ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment)

Example 3 with ViterbiSegment

use of com.hankcs.hanlp.seg.Viterbi.ViterbiSegment in project HanLP by hankcs.

the class TestSegment method testTryToCrashSegment.

public void testTryToCrashSegment() throws Exception {
    String text = "尝试玩坏分词器";
    Segment segment = new ViterbiSegment().enableMultithreading(100);
    System.out.println(segment.seg(text));
}
Also used : ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment) Segment(com.hankcs.hanlp.seg.Segment) DoubleArrayTrieSegment(com.hankcs.hanlp.seg.Other.DoubleArrayTrieSegment) CRFSegment(com.hankcs.hanlp.seg.CRF.CRFSegment) DijkstraSegment(com.hankcs.hanlp.seg.Dijkstra.DijkstraSegment) ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment)

Example 4 with ViterbiSegment

use of com.hankcs.hanlp.seg.Viterbi.ViterbiSegment in project HanLP by hankcs.

the class TestSegment method testSpeedOfSecondViterbi.

public void testSpeedOfSecondViterbi() throws Exception {
    String text = "王总和小丽结婚了";
    Segment segment = new ViterbiSegment().enableAllNamedEntityRecognize(false).enableNameRecognize(// 人名识别需要二次维特比,比较慢
    false).enableCustomDictionary(false);
    System.out.println(segment.seg(text));
    long start = System.currentTimeMillis();
    int pressure = 1000000;
    for (int i = 0; i < pressure; ++i) {
        segment.seg(text);
    }
    double costTime = (System.currentTimeMillis() - start) / (double) 1000;
    System.out.printf("分词速度:%.2f字每秒", text.length() * pressure / costTime);
}
Also used : ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment) Segment(com.hankcs.hanlp.seg.Segment) DoubleArrayTrieSegment(com.hankcs.hanlp.seg.Other.DoubleArrayTrieSegment) CRFSegment(com.hankcs.hanlp.seg.CRF.CRFSegment) DijkstraSegment(com.hankcs.hanlp.seg.Dijkstra.DijkstraSegment) ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment)

Example 5 with ViterbiSegment

use of com.hankcs.hanlp.seg.Viterbi.ViterbiSegment in project HanLP by hankcs.

the class TestSegment method testCustomDictionary.

public void testCustomDictionary() throws Exception {
    CustomDictionary.insert("肯德基", "ns 1000");
    Segment segment = new ViterbiSegment();
    System.out.println(segment.seg("肯德基"));
}
Also used : ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment) Segment(com.hankcs.hanlp.seg.Segment) DoubleArrayTrieSegment(com.hankcs.hanlp.seg.Other.DoubleArrayTrieSegment) CRFSegment(com.hankcs.hanlp.seg.CRF.CRFSegment) DijkstraSegment(com.hankcs.hanlp.seg.Dijkstra.DijkstraSegment) ViterbiSegment(com.hankcs.hanlp.seg.Viterbi.ViterbiSegment)

Aggregations

Segment (com.hankcs.hanlp.seg.Segment)8 ViterbiSegment (com.hankcs.hanlp.seg.Viterbi.ViterbiSegment)8 CRFSegment (com.hankcs.hanlp.seg.CRF.CRFSegment)7 DijkstraSegment (com.hankcs.hanlp.seg.Dijkstra.DijkstraSegment)7 DoubleArrayTrieSegment (com.hankcs.hanlp.seg.Other.DoubleArrayTrieSegment)7 NShortSegment (com.hankcs.hanlp.seg.NShort.NShortSegment)1