Search in sources :

Example 6 with TurkishMorphology

use of zemberek.morphology.analysis.tr.TurkishMorphology in project zemberek-nlp by ahmetaa.

the class UnidentifiedTokenAnalyzerTest method shouldCreateUnidentifiedTokenParserSuccessfully.

@Test
public void shouldCreateUnidentifiedTokenParserSuccessfully() throws IOException {
    TurkishMorphology parser = TurkishMorphology.createWithDefaults();
    UnidentifiedTokenAnalyzer uiParser = new UnidentifiedTokenAnalyzer(parser);
    List<WordAnalysis> results = uiParser.analyze("Ankara'ya");
    for (WordAnalysis result : results) {
        System.out.println(result);
    }
}
Also used : WordAnalysis(zemberek.morphology.analysis.WordAnalysis) TurkishMorphology(zemberek.morphology.analysis.tr.TurkishMorphology) UnidentifiedTokenAnalyzer(zemberek.morphology.analysis.tr.UnidentifiedTokenAnalyzer) Test(org.junit.Test)

Example 7 with TurkishMorphology

use of zemberek.morphology.analysis.tr.TurkishMorphology in project zemberek-nlp by ahmetaa.

the class ZemberekNlpScripts method testWordAnalysis.

@Test
@Ignore("Not a Test.")
public void testWordAnalysis() throws IOException {
    TurkishMorphology morphology = TurkishMorphology.createWithDefaults();
    List<WordAnalysis> results = morphology.analyze("phpye");
    for (WordAnalysis result : results) {
        Log.info(result.formatLong());
        Log.info("\tStems = " + result.getStems());
        Log.info("\tLemmas = " + result.getLemmas());
    }
}
Also used : WordAnalysis(zemberek.morphology.analysis.WordAnalysis) TurkishMorphology(zemberek.morphology.analysis.tr.TurkishMorphology) Ignore(org.junit.Ignore) Test(org.junit.Test)

Example 8 with TurkishMorphology

use of zemberek.morphology.analysis.tr.TurkishMorphology in project zemberek-nlp by ahmetaa.

the class ZemberekNlpScripts method testSentenceAnalysis.

@Test
@Ignore("Not a Test.")
public void testSentenceAnalysis() throws IOException {
    TurkishMorphology morphology = TurkishMorphology.createWithDefaults();
    Z3MarkovModelDisambiguator disambiguator = new Z3MarkovModelDisambiguator();
    TurkishSentenceAnalyzer analyzer = new TurkishSentenceAnalyzer(morphology, disambiguator);
    String sentence = "Kırmızı kalemi al.";
    Log.info("Sentence  = " + sentence);
    SentenceAnalysis analysis = analyzer.analyze(sentence);
    Log.info("Before disambiguation.");
    writeParseResult(analysis);
    Log.info("\nAfter disambiguation.");
    analyzer.disambiguate(analysis);
    writeParseResult(analysis);
}
Also used : Z3MarkovModelDisambiguator(zemberek.morphology.ambiguity.Z3MarkovModelDisambiguator) TurkishSentenceAnalyzer(zemberek.morphology.analysis.tr.TurkishSentenceAnalyzer) SentenceAnalysis(zemberek.morphology.analysis.SentenceAnalysis) TurkishMorphology(zemberek.morphology.analysis.tr.TurkishMorphology) Ignore(org.junit.Ignore) Test(org.junit.Test)

Example 9 with TurkishMorphology

use of zemberek.morphology.analysis.tr.TurkishMorphology in project zemberek-nlp by ahmetaa.

the class ZemberekNlpScripts method disambiguationMemoryTest.

@Test
@Ignore("Not a Test.")
public void disambiguationMemoryTest() throws IOException {
    List<String> lines = Files.readAllLines(Paths.get("/media/depo/data/aaa/corpora/dunya.100k"));
    TurkishMorphology parser = TurkishMorphology.createWithDefaults();
    TurkishSentenceAnalyzer sentenceAnalyzer = new TurkishSentenceAnalyzer(parser, new Z3MarkovModelDisambiguator());
    int k = 0;
    for (int i = 0; i < 100; i++) {
        Stopwatch sw = Stopwatch.createStarted();
        for (String line : lines) {
            k += sentenceAnalyzer.bestParse(line).size();
        }
        Log.info(sw.elapsed(TimeUnit.MILLISECONDS));
    }
    Log.info(k);
}
Also used : Stopwatch(com.google.common.base.Stopwatch) Z3MarkovModelDisambiguator(zemberek.morphology.ambiguity.Z3MarkovModelDisambiguator) TurkishSentenceAnalyzer(zemberek.morphology.analysis.tr.TurkishSentenceAnalyzer) TurkishMorphology(zemberek.morphology.analysis.tr.TurkishMorphology) Ignore(org.junit.Ignore) Test(org.junit.Test)

Example 10 with TurkishMorphology

use of zemberek.morphology.analysis.tr.TurkishMorphology in project zemberek-nlp by ahmetaa.

the class ParseConsole method main.

public static void main(String[] args) throws IOException {
    // to test the development lexicon, use ParseConsoleTest
    // TurkishMorphology morphology = TurkishMorphology.createWithDefaults();
    TurkishMorphology morphology = TurkishMorphology.builder().addDefaultDictionaries().build();
    // morphology.getGraph().stats();
    new ParseConsole().run(morphology);
}
Also used : TurkishMorphology(zemberek.morphology.analysis.tr.TurkishMorphology)

Aggregations

TurkishMorphology (zemberek.morphology.analysis.tr.TurkishMorphology)26 Test (org.junit.Test)13 Ignore (org.junit.Ignore)5 Z3MarkovModelDisambiguator (zemberek.morphology.ambiguity.Z3MarkovModelDisambiguator)5 WordAnalysis (zemberek.morphology.analysis.WordAnalysis)5 TurkishSentenceAnalyzer (zemberek.morphology.analysis.tr.TurkishSentenceAnalyzer)5 Path (java.nio.file.Path)2 UnidentifiedTokenAnalyzer (zemberek.morphology.analysis.tr.UnidentifiedTokenAnalyzer)2 DictionaryItem (zemberek.morphology.lexicon.DictionaryItem)2 Stopwatch (com.google.common.base.Stopwatch)1 BufferedOutputStream (java.io.BufferedOutputStream)1 File (java.io.File)1 FileOutputStream (java.io.FileOutputStream)1 ArrayList (java.util.ArrayList)1 LinkedHashSet (java.util.LinkedHashSet)1 Before (org.junit.Before)1 SentenceAnalysis (zemberek.morphology.analysis.SentenceAnalysis)1 LexiconProto (zemberek.morphology.lexicon.proto.LexiconProto)1 Dictionary (zemberek.morphology.lexicon.proto.LexiconProto.Dictionary)1 TurkishSpellChecker (zemberek.normalization.TurkishSpellChecker)1