Search in sources :

Example 6 with FeatureVector

use of edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector in project cogcomp-nlp by CogComp.

the class Forms method classify.

public FeatureVector classify(Object __example) {
    if (!(__example instanceof Word)) {
        String type = __example == null ? "null" : __example.getClass().getName();
        System.err.println("Classifier 'Forms(Word)' defined on line 20 of CommonFeatures.lbj received '" + type + "' as input.");
        new Exception().printStackTrace();
        System.exit(1);
    }
    Word word = (Word) __example;
    FeatureVector __result;
    __result = new FeatureVector();
    String __id;
    String __value;
    int i;
    Word w = word, last = word;
    for (i = 0; i <= 2 && last != null; ++i) {
        last = (Word) last.next;
    }
    for (i = 0; i > -2 && w.previous != null; --i) {
        w = (Word) w.previous;
    }
    for (; w != last; w = (Word) w.next) {
        __id = "" + (i++);
        __value = "" + (w.form);
        __result.addFeature(new DiscretePrimitiveStringFeature(this.containingPackage, this.name, __id, __value, valueIndexOf(__value), (short) 0));
    }
    return __result;
}
Also used : FeatureVector(edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector) DiscretePrimitiveStringFeature(edu.illinois.cs.cogcomp.lbjava.classify.DiscretePrimitiveStringFeature)

Example 7 with FeatureVector

use of edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector in project cogcomp-nlp by CogComp.

the class FeatureVectorParser method main.

/**
     * Takes the names of the example and lexicon files as input on the command line and prints all
     * the examples to <code>STDOUT</code>.
     **/
public static void main(String[] args) {
    String exampleFile = null;
    String lexiconFile = null;
    try {
        exampleFile = args[0];
        lexiconFile = args[1];
        if (args.length > 2)
            throw new Exception();
    } catch (Exception e) {
        System.err.println("usage: java edu.illinois.cs.cogcomp.lbjava.parse.FeatureVectorParser <example file> <lexicon file>");
        System.exit(1);
    }
    Parser parser = new FeatureVectorParser(exampleFile, lexiconFile);
    for (FeatureVector v = (FeatureVector) parser.next(); v != null; v = (FeatureVector) parser.next()) {
        v.sort();
        logger.info(v.toString());
    }
}
Also used : FeatureVector(edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector) EOFException(java.io.EOFException) Parser(edu.illinois.cs.cogcomp.lbjava.parse.Parser)

Example 8 with FeatureVector

use of edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector in project cogcomp-nlp by CogComp.

the class LBJavaFeatureExtractor method classify.

@Override
public FeatureVector classify(Object o) {
    // Make sure the object is a Constituent
    if (!(o instanceof Constituent))
        throw new IllegalArgumentException("Instance must be of type Constituent");
    Constituent instance = (Constituent) o;
    FeatureVector featureVector = new FeatureVector();
    try {
        featureVector = FeatureUtilities.getLBJFeatures(getFeatures(instance));
    } catch (Exception e) {
        logger.debug("Couldn't generate feature {} for constituent {}", getName(), instance);
    }
    return featureVector;
}
Also used : FeatureVector(edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector) Constituent(edu.illinois.cs.cogcomp.core.datastructures.textannotation.Constituent) EdisonException(edu.illinois.cs.cogcomp.edison.utilities.EdisonException)

Example 9 with FeatureVector

use of edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector in project cogcomp-nlp by CogComp.

the class LBJavaFeatureExtractor method classify.

@Override
public FeatureVector classify(Object o) {
    // Make sure the object is a Constituent
    if (!(o instanceof Constituent))
        throw new IllegalArgumentException("Instance must be of type Constituent");
    Constituent instance = (Constituent) o;
    FeatureVector featureVector;
    try {
        featureVector = FeatureUtilities.getLBJFeatures(getFeatures(instance));
    } catch (EdisonException e) {
        throw new RuntimeException(e);
    }
    return featureVector;
}
Also used : FeatureVector(edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector) EdisonException(edu.illinois.cs.cogcomp.edison.utilities.EdisonException) Constituent(edu.illinois.cs.cogcomp.core.datastructures.textannotation.Constituent)

Example 10 with FeatureVector

use of edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector in project cogcomp-nlp by CogComp.

the class NETesterMultiDataset method dumpFeaturesLabeledData.

/**
     * NB: assuming column format
     */
public static void dumpFeaturesLabeledData(String testDatapath, String outDatapath) throws Exception {
    FeaturesLevel1SharedWithLevel2 features1 = new FeaturesLevel1SharedWithLevel2();
    FeaturesLevel2 features2 = new FeaturesLevel2();
    NETaggerLevel1 taggerLevel1 = new NETaggerLevel1(ParametersForLbjCode.currentParameters.pathToModelFile + ".level1", ParametersForLbjCode.currentParameters.pathToModelFile + ".level1.lex");
    NETaggerLevel2 taggerLevel2 = new NETaggerLevel2(ParametersForLbjCode.currentParameters.pathToModelFile + ".level2", ParametersForLbjCode.currentParameters.pathToModelFile + ".level2.lex");
    File f = new File(testDatapath);
    Vector<String> inFiles = new Vector<>();
    Vector<String> outFiles = new Vector<>();
    if (f.isDirectory()) {
        String[] files = f.list();
        for (String file : files) if (!file.startsWith(".")) {
            inFiles.addElement(testDatapath + "/" + file);
            outFiles.addElement(outDatapath + "/" + file);
        }
    } else {
        inFiles.addElement(testDatapath);
        outFiles.addElement(outDatapath);
    }
    for (int fileId = 0; fileId < inFiles.size(); fileId++) {
        Data testData = new Data(inFiles.elementAt(fileId), inFiles.elementAt(fileId), "-c", new String[] {}, new String[] {});
        ExpressiveFeaturesAnnotator.annotate(testData);
        Decoder.annotateDataBIO(testData, taggerLevel1, taggerLevel2);
        OutFile out = new OutFile(outFiles.elementAt(fileId));
        for (int docid = 0; docid < testData.documents.size(); docid++) {
            ArrayList<LinkedVector> sentences = testData.documents.get(docid).sentences;
            for (LinkedVector sentence : sentences) {
                for (int j = 0; j < sentence.size(); j++) {
                    NEWord w = (NEWord) sentence.get(j);
                    out.print(w.neLabel + "\t" + w.form + "\t");
                    FeatureVector fv1 = features1.classify(w);
                    FeatureVector fv2 = features2.classify(w);
                    for (int k = 0; k < fv1.size(); k++) {
                        String s = fv1.getFeature(k).toString();
                        out.print(" " + s.substring(s.indexOf(':') + 1, s.length()));
                    }
                    for (int k = 0; k < fv2.size(); k++) {
                        String s = fv2.getFeature(k).toString();
                        out.print(" " + s.substring(s.indexOf(':') + 1, s.length()));
                    }
                    out.println("");
                }
                out.println("");
            }
        }
        out.close();
    }
}
Also used : FeatureVector(edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector) LinkedVector(edu.illinois.cs.cogcomp.lbjava.parse.LinkedVector) File(java.io.File) OutFile(edu.illinois.cs.cogcomp.ner.IO.OutFile) LinkedVector(edu.illinois.cs.cogcomp.lbjava.parse.LinkedVector) Vector(java.util.Vector) FeatureVector(edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector) OutFile(edu.illinois.cs.cogcomp.ner.IO.OutFile)

Aggregations

FeatureVector (edu.illinois.cs.cogcomp.lbjava.classify.FeatureVector)17 DiscretePrimitiveStringFeature (edu.illinois.cs.cogcomp.lbjava.classify.DiscretePrimitiveStringFeature)9 Token (edu.illinois.cs.cogcomp.lbjava.nlp.seg.Token)5 Constituent (edu.illinois.cs.cogcomp.core.datastructures.textannotation.Constituent)3 EdisonException (edu.illinois.cs.cogcomp.edison.utilities.EdisonException)3 LinkedVector (edu.illinois.cs.cogcomp.lbjava.parse.LinkedVector)1 Parser (edu.illinois.cs.cogcomp.lbjava.parse.Parser)1 OutFile (edu.illinois.cs.cogcomp.ner.IO.OutFile)1 EOFException (java.io.EOFException)1 File (java.io.File)1 Vector (java.util.Vector)1 Test (org.junit.Test)1