Search in sources :

Example 1 with Datastore

use of org.cogcomp.Datastore in project cogcomp-nlp by CogComp.

the class CommaLabeler method initialize.

@Override
public void initialize(ResourceManager resourceManager) {
    try {
        classifier = new LocalCommaClassifier();
        Datastore ds = new Datastore(new ResourceConfigurator().getDefaultConfig());
        File f = ds.getDirectory("org.cogcomp.comma-srl", "comma-srl-models", 2.2, false);
        String folder = f.toString() + File.separator + "comma-srl-models" + File.separator;
        classifier.readLexicon(folder + "LocalCommaClassifier.lex");
        classifier.readModel(folder + "LocalCommaClassifier.lc");
    } catch (Exception e) {
        e.printStackTrace();
    }
    assert classifier.getPrunedLexiconSize() > 1000;
    assert classifier.getLabelLexicon().size() > 5;
}
Also used : Datastore(org.cogcomp.Datastore) ResourceConfigurator(edu.illinois.cs.cogcomp.core.resources.ResourceConfigurator) File(java.io.File) DatastoreException(org.cogcomp.DatastoreException) AnnotatorException(edu.illinois.cs.cogcomp.annotation.AnnotatorException) LocalCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.LocalCommaClassifier)

Example 2 with Datastore

use of org.cogcomp.Datastore in project cogcomp-nlp by CogComp.

the class BrownClusterViewGenerator method loadFromDataStore.

public void loadFromDataStore() throws Exception {
    Datastore dsNoCredentials = new Datastore(new ResourceConfigurator().getDefaultConfig());
    File f = dsNoCredentials.getFile("org.cogcomp.brown-clusters", file, 1.3);
    InputStream is = new FileInputStream(f);
    loadFromInputStream(is);
}
Also used : Datastore(org.cogcomp.Datastore) GZIPInputStream(java.util.zip.GZIPInputStream) ResourceConfigurator(edu.illinois.cs.cogcomp.core.resources.ResourceConfigurator)

Example 3 with Datastore

use of org.cogcomp.Datastore in project cogcomp-nlp by CogComp.

the class SenseManager method getLegalSensesMap.

private Map<String, Set<String>> getLegalSensesMap() {
    Map<String, Set<String>> map = new HashMap<>();
    Datastore ds = null;
    File senseFile = null;
    try {
        ds = new Datastore(new ResourceConfigurator().getDefaultConfig());
        senseFile = ds.getFile("edu.illinois.cs.cogcomp.verbsense", "sense-list.txt", 1.0, false);
    } catch (InvalidPortException | InvalidEndpointException | DatastoreException e) {
        e.printStackTrace();
    }
    try {
        for (String line : LineIO.read(senseFile.getAbsolutePath())) {
            String predicate = line.split("\t")[0];
            String[] senseArray = line.split("\t")[1].split(",");
            Set<String> senseSet = new HashSet<>(Arrays.asList(senseArray));
            map.put(predicate, senseSet);
        }
    } catch (FileNotFoundException e) {
        log.error("Unable to load list of legal senses: ", e);
        System.exit(-1);
    }
    return map;
}
Also used : FileNotFoundException(java.io.FileNotFoundException) ResourceConfigurator(edu.illinois.cs.cogcomp.core.resources.ResourceConfigurator) DatastoreException(org.cogcomp.DatastoreException) InvalidPortException(io.minio.errors.InvalidPortException) InvalidEndpointException(io.minio.errors.InvalidEndpointException) Datastore(org.cogcomp.Datastore) File(java.io.File)

Example 4 with Datastore

use of org.cogcomp.Datastore in project cogcomp-nlp by CogComp.

the class RogetThesaurusFeatures method loadFromDatastore.

private synchronized void loadFromDatastore() throws Exception {
    if (loaded)
        return;
    Datastore dsNoCredentials = new Datastore(new ResourceConfigurator().getDefaultConfig());
    File f = dsNoCredentials.getFile("org.cogcomp.roget.thesaurus", "rogetThesaurus", 1.3);
    loadWithURL(f.toURI().toURL());
}
Also used : Datastore(org.cogcomp.Datastore) ResourceConfigurator(edu.illinois.cs.cogcomp.core.resources.ResourceConfigurator) File(java.io.File)

Example 5 with Datastore

use of org.cogcomp.Datastore in project cogcomp-nlp by CogComp.

the class VerbClassDictionary method getDictionaryFromDatastore.

public static VerbClassDictionary getDictionaryFromDatastore() {
    if (verbClassDictionary == null) {
        synchronized (LevinVerbClassFeature.class) {
            if (verbClassDictionary == null) {
                log.info("Reading verb class dictionary. Looking for " + verbClassFile + " in the datastore");
                try {
                    Datastore dsNoCredentials = new Datastore(new ResourceConfigurator().getDefaultConfig());
                    File f = dsNoCredentials.getFile("org.cogcomp.levin.verb.class", "levin-verbClass", 1.6);
                    InputStream resource = new FileInputStream(f);
                    verbClassDictionary = new VerbClassDictionary(resource);
                } catch (Exception e) {
                    e.printStackTrace();
                    log.error("Unable to read the verb class dictionary", e);
                    System.exit(-1);
                }
                List<String> strings = verbClassDictionary.getClass("give");
                log.info("Loaded verb class dictionary. Test: classes for 'give' are {}", strings);
            }
        }
    }
    return verbClassDictionary;
}
Also used : Datastore(org.cogcomp.Datastore) LevinVerbClassFeature(edu.illinois.cs.cogcomp.edison.features.factory.LevinVerbClassFeature) ResourceConfigurator(edu.illinois.cs.cogcomp.core.resources.ResourceConfigurator)

Aggregations

Datastore (org.cogcomp.Datastore)8 ResourceConfigurator (edu.illinois.cs.cogcomp.core.resources.ResourceConfigurator)7 File (java.io.File)5 DatastoreException (org.cogcomp.DatastoreException)5 InvalidEndpointException (io.minio.errors.InvalidEndpointException)3 InvalidPortException (io.minio.errors.InvalidPortException)3 FileNotFoundException (java.io.FileNotFoundException)2 AnnotatorException (edu.illinois.cs.cogcomp.annotation.AnnotatorException)1 LocalCommaClassifier (edu.illinois.cs.cogcomp.comma.lbj.LocalCommaClassifier)1 LevinVerbClassFeature (edu.illinois.cs.cogcomp.edison.features.factory.LevinVerbClassFeature)1 StringSplitterInterface (edu.illinois.cs.cogcomp.ner.ExpressiveFeatures.GazetteerTree.StringSplitterInterface)1 InFile (edu.illinois.cs.cogcomp.ner.IO.InFile)1 THashMap (gnu.trove.map.hash.THashMap)1 FileInputStream (java.io.FileInputStream)1 IOException (java.io.IOException)1 InputStream (java.io.InputStream)1 ArrayList (java.util.ArrayList)1 StringTokenizer (java.util.StringTokenizer)1 GZIPInputStream (java.util.zip.GZIPInputStream)1 CompletePipelineCMDLineOptions (se.lth.cs.srl.options.CompletePipelineCMDLineOptions)1