Examples with OptimaizeLangDetector org.apache.tika.langdetect.OptimaizeLangDetector used on opensource projects

Example 1 with OptimaizeLangDetector

use of org.apache.tika.langdetect.OptimaizeLangDetector in project tika by apache.

the class Language method languageDetection.

public static void languageDetection() throws IOException {
    LanguageDetector detector = new OptimaizeLangDetector().loadModels();
    LanguageResult result = detector.detect("Alla människor är födda fria och lika i värde och rättigheter.");
    System.out.println(result.getLanguage());
}

Example 2 with OptimaizeLangDetector

use of org.apache.tika.langdetect.OptimaizeLangDetector in project tika by apache.

the class LanguageDetectorExample method detectLanguage.

public String detectLanguage(String text) throws IOException {
    LanguageDetector detector = new OptimaizeLangDetector().loadModels();
    LanguageResult result = detector.detect(text);
    return result.getLanguage();
}

Example 3 with OptimaizeLangDetector

use of org.apache.tika.langdetect.OptimaizeLangDetector in project tika by apache.

the class Language method languageDetectionWithWriter.

public static void languageDetectionWithWriter() throws IOException {
    // TODO support version of LanguageWriter that doesn't need a detector.
    LanguageDetector detector = new OptimaizeLangDetector().loadModels();
    LanguageWriter writer = new LanguageWriter(detector);
    writer.append("Minden emberi lény");
    writer.append(" szabadon születik és");
    writer.append(" egyenlő méltósága és");
    writer.append(" joga van.");
    LanguageResult result = writer.getLanguage();
    System.out.println(result.getLanguage());
    writer.close();
}

Example 4 with OptimaizeLangDetector

use of org.apache.tika.langdetect.OptimaizeLangDetector in project tika by apache.

the class MyFirstTika method parseUsingComponents.

public static String parseUsingComponents(String filename, TikaConfig tikaConfig, Metadata metadata) throws Exception {
    MimeTypes mimeRegistry = tikaConfig.getMimeRepository();
    System.out.println("Examining: [" + filename + "]");
    metadata.set(Metadata.RESOURCE_NAME_KEY, filename);
    System.out.println("The MIME type (based on filename) is: [" + mimeRegistry.detect(null, metadata) + "]");
    InputStream stream = TikaInputStream.get(new File(filename));
    System.out.println("The MIME type (based on MAGIC) is: [" + mimeRegistry.detect(stream, metadata) + "]");
    stream = TikaInputStream.get(new File(filename));
    Detector detector = tikaConfig.getDetector();
    System.out.println("The MIME type (based on the Detector interface) is: [" + detector.detect(stream, metadata) + "]");
    LanguageDetector langDetector = new OptimaizeLangDetector().loadModels();
    LanguageResult lang = langDetector.detect(FileUtils.readFileToString(new File(filename), UTF_8));
    System.out.println("The language of this content is: [" + lang.getLanguage() + "]");
    // Get a non-detecting parser that handles all the types it can
    Parser parser = tikaConfig.getParser();
    // Tell it what we think the content is
    MediaType type = detector.detect(stream, metadata);
    metadata.set(Metadata.CONTENT_TYPE, type.toString());
    // Have the file parsed to get the content and metadata
    ContentHandler handler = new BodyContentHandler();
    parser.parse(stream, handler, metadata, new ParseContext());
    return handler.toString();
}

Example 5 with OptimaizeLangDetector

use of org.apache.tika.langdetect.OptimaizeLangDetector in project tika by apache.

the class TranslateResource method autoTranslate.

@PUT
@POST
@Path("/all/{translator}/{dest}")
@Consumes("*/*")
@Produces("text/plain")
public String autoTranslate(final InputStream is, @PathParam("translator") String translator, @PathParam("dest") String dLang) throws TikaException, IOException {
    final String content = IOUtils.toString(is, UTF_8);
    LanguageResult language = new OptimaizeLangDetector().loadModels().detect(content);
    if (language.isUnknown()) {
        throw new TikaException("Unable to detect language to use for translation of text");
    }
    String sLang = language.getLanguage();
    LOG.info("LanguageIdentifier: detected source lang: [{}]", sLang);
    return doTranslate(content, translator, sLang, dLang);
}