Search in sources :

Example 1 with StructuredCommaClassifier

use of edu.illinois.cs.cogcomp.comma.sl.StructuredCommaClassifier in project cogcomp-nlp by CogComp.

the class ClassifierComparison method main.

public static void main(String[] args) throws Exception {
    PrettyCorpusReader pcr = new PrettyCorpusReader(CommaProperties.getInstance().getCommaLabeledDataFile());
    CommaParser parser = new CommaParser(pcr.getSentences(), Ordering.ORDERED, true);
    System.out.println("GOLD GOLD");
    localCVal(true, true, parser, 250, 0.003, 0, 2.0, false);
    System.out.println("GOLD AUTO");
    localCVal(true, false, parser, 200, 0.003, 0, 2.0, false);
    System.out.println("AUTO AUTO");
    localCVal(false, false, parser, 250, 0.003, 0, 3.5, false);
    List<Classifier> lbjExtractors = new ArrayList<>();
    lbjExtractors.add(new LocalCommaClassifier().getExtractor());
    Classifier lbjLabeler = new LocalCommaClassifier().getLabeler();
    System.out.println("STRUCTURED GOLD");
    StructuredCommaClassifier goldStructured = new StructuredCommaClassifier(lbjExtractors, lbjLabeler);
    structuredCVal(goldStructured, parser, true, false);
    System.out.println("STRUCTURED AUTO");
    StructuredCommaClassifier autoStructured = new StructuredCommaClassifier(lbjExtractors, lbjLabeler);
    structuredCVal(autoStructured, parser, false, false);
    System.out.println("BAYRAKTAR GOLD");
    EvaluateDiscrete bayraktarGold = getBayraktarBaselinePerformance(parser, true);
    bayraktarGold.printPerformance(System.out);
    System.out.println("BAYRAKTAR AUTO");
    EvaluateDiscrete bayraktarAuto = getBayraktarBaselinePerformance(parser, false);
    bayraktarAuto.printPerformance(System.out);
    reasonForBelievingThatStructuredIsPerformingWorseDueToOverfitting(parser, false);
    printConstrainedClassifierPerformance(parser);
}
Also used : EvaluateDiscrete(edu.illinois.cs.cogcomp.comma.utils.EvaluateDiscrete) CommaParser(edu.illinois.cs.cogcomp.comma.readers.CommaParser) ArrayList(java.util.ArrayList) OxfordCommaConstrainedCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.OxfordCommaConstrainedCommaClassifier) Classifier(edu.illinois.cs.cogcomp.lbjava.classify.Classifier) LocativePairConstrainedCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.LocativePairConstrainedCommaClassifier) StructuredCommaClassifier(edu.illinois.cs.cogcomp.comma.sl.StructuredCommaClassifier) SubstitutePairConstrainedCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.SubstitutePairConstrainedCommaClassifier) LocalCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.LocalCommaClassifier) ListCommasConstrainedCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.ListCommasConstrainedCommaClassifier) PrettyCorpusReader(edu.illinois.cs.cogcomp.comma.readers.PrettyCorpusReader) LocalCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.LocalCommaClassifier) StructuredCommaClassifier(edu.illinois.cs.cogcomp.comma.sl.StructuredCommaClassifier)

Example 2 with StructuredCommaClassifier

use of edu.illinois.cs.cogcomp.comma.sl.StructuredCommaClassifier in project cogcomp-nlp by CogComp.

the class ClassifierComparison method reasonForBelievingThatStructuredIsPerformingWorseDueToOverfitting.

/**
     * Structured's higher performance on the train set and lower performance on test set is
     * indicative of overfitting
     */
public static void reasonForBelievingThatStructuredIsPerformingWorseDueToOverfitting(Parser parser, boolean useGoldFeatures) throws Exception {
    List<Classifier> lbjExtractors = new ArrayList<>();
    lbjExtractors.add(new LocalCommaClassifier().getExtractor());
    Classifier lbjLabeler = new LocalCommaClassifier().getLabeler();
    StructuredCommaClassifier structured = new StructuredCommaClassifier(lbjExtractors, lbjLabeler);
    int learningRounds = 250;
    double learningRate = 0.003;
    double threshold = 0;
    double thickness = 3.5;
    EvaluateDiscrete structuredPerformanceOnTrainSet = structuredCVal(structured, parser, useGoldFeatures, true);
    EvaluateDiscrete structuredPerformanceOnTestSet = structuredCVal(structured, parser, useGoldFeatures, false);
    EvaluateDiscrete localPerformanceOnTrainSet = localCVal(useGoldFeatures, useGoldFeatures, parser, learningRounds, learningRate, threshold, thickness, true);
    EvaluateDiscrete localPerformanceOnTestSet = localCVal(useGoldFeatures, useGoldFeatures, parser, learningRounds, learningRate, threshold, thickness, false);
    System.out.println("Structured performance on train set " + structuredPerformanceOnTrainSet.getOverallStats()[2]);
    System.out.println("Structured performance on test set " + structuredPerformanceOnTestSet.getOverallStats()[2]);
    System.out.println("Local performance on train set " + localPerformanceOnTrainSet.getOverallStats()[2]);
    System.out.println("Localperformance on test set " + localPerformanceOnTestSet.getOverallStats()[2]);
}
Also used : EvaluateDiscrete(edu.illinois.cs.cogcomp.comma.utils.EvaluateDiscrete) ArrayList(java.util.ArrayList) OxfordCommaConstrainedCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.OxfordCommaConstrainedCommaClassifier) Classifier(edu.illinois.cs.cogcomp.lbjava.classify.Classifier) LocativePairConstrainedCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.LocativePairConstrainedCommaClassifier) StructuredCommaClassifier(edu.illinois.cs.cogcomp.comma.sl.StructuredCommaClassifier) SubstitutePairConstrainedCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.SubstitutePairConstrainedCommaClassifier) LocalCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.LocalCommaClassifier) ListCommasConstrainedCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.ListCommasConstrainedCommaClassifier) LocalCommaClassifier(edu.illinois.cs.cogcomp.comma.lbj.LocalCommaClassifier) StructuredCommaClassifier(edu.illinois.cs.cogcomp.comma.sl.StructuredCommaClassifier)

Aggregations

ListCommasConstrainedCommaClassifier (edu.illinois.cs.cogcomp.comma.lbj.ListCommasConstrainedCommaClassifier)2 LocalCommaClassifier (edu.illinois.cs.cogcomp.comma.lbj.LocalCommaClassifier)2 LocativePairConstrainedCommaClassifier (edu.illinois.cs.cogcomp.comma.lbj.LocativePairConstrainedCommaClassifier)2 OxfordCommaConstrainedCommaClassifier (edu.illinois.cs.cogcomp.comma.lbj.OxfordCommaConstrainedCommaClassifier)2 SubstitutePairConstrainedCommaClassifier (edu.illinois.cs.cogcomp.comma.lbj.SubstitutePairConstrainedCommaClassifier)2 StructuredCommaClassifier (edu.illinois.cs.cogcomp.comma.sl.StructuredCommaClassifier)2 EvaluateDiscrete (edu.illinois.cs.cogcomp.comma.utils.EvaluateDiscrete)2 Classifier (edu.illinois.cs.cogcomp.lbjava.classify.Classifier)2 ArrayList (java.util.ArrayList)2 CommaParser (edu.illinois.cs.cogcomp.comma.readers.CommaParser)1 PrettyCorpusReader (edu.illinois.cs.cogcomp.comma.readers.PrettyCorpusReader)1