Search in sources :

Example 1 with CrossValSplitter

use of edu.cmu.minorthird.classify.experiments.CrossValSplitter in project lucida by claritylab.

the class ScoreNormalizationFilter method evaluate.

/**
	 * Performs a cross-validation on the given data set for the given features
	 * and model.
	 * 
	 * @param serializedDir directory containing serialized results
	 * @param features selected features
	 * @param model selected model
	 * @return evaluation statistics
	 */
public static Evaluation evaluate(String serializedDir, String[] features, String model) {
    // create data set with selected features from serialized results
    Dataset dataSet = createDataset(features, serializedDir);
    // create learner for selected model
    ClassifierLearner learner = createLearner(model);
    // cross-validate model on data set
    RandomElement r = new RandomElement(System.currentTimeMillis());
    Splitter splitter = new CrossValSplitter(r, NUM_FOLDS);
    CrossValidatedDataset cvDataset = new CrossValidatedDataset(learner, dataSet, splitter, true);
    Evaluation eval = cvDataset.getEvaluation();
    return eval;
}
Also used : ClassifierLearner(edu.cmu.minorthird.classify.ClassifierLearner) CrossValidatedDataset(edu.cmu.minorthird.classify.experiments.CrossValidatedDataset) Evaluation(edu.cmu.minorthird.classify.experiments.Evaluation) Splitter(edu.cmu.minorthird.classify.Splitter) CrossValSplitter(edu.cmu.minorthird.classify.experiments.CrossValSplitter) BasicDataset(edu.cmu.minorthird.classify.BasicDataset) CrossValidatedDataset(edu.cmu.minorthird.classify.experiments.CrossValidatedDataset) Dataset(edu.cmu.minorthird.classify.Dataset) CrossValSplitter(edu.cmu.minorthird.classify.experiments.CrossValSplitter) RandomElement(edu.cmu.minorthird.classify.algorithms.random.RandomElement)

Example 2 with CrossValSplitter

use of edu.cmu.minorthird.classify.experiments.CrossValSplitter in project lucida by claritylab.

the class HierarchicalClassifierTrainer method runExperiment.

public Evaluation runExperiment() {
    runTime = System.currentTimeMillis();
    ClassifierLearner learner = createHierarchicalClassifierLearner(learnerNames);
    if (crossValidationFolds < 0) {
        evaluation = Tester.evaluate(learner, trainingSet, testingSet);
    } else {
        Splitter splitter = new CrossValSplitter(new RandomElement(System.currentTimeMillis()), crossValidationFolds);
        cvDataset = new CrossValidatedDataset(learner, trainingSet, splitter, true);
        evaluation = cvDataset.getEvaluation();
    //remove later
    //ViewerFrame frame=new ViewerFrame(trainingFile,cvDataset.toGUI());
    //frame.setVisible(true);
    //evaluation=Tester.evaluate(learner,trainingSet,splitter);
    }
    runTime = System.currentTimeMillis() - runTime;
    return evaluation;
}
Also used : ClassifierLearner(edu.cmu.minorthird.classify.ClassifierLearner) CrossValidatedDataset(edu.cmu.minorthird.classify.experiments.CrossValidatedDataset) Splitter(edu.cmu.minorthird.classify.Splitter) CrossValSplitter(edu.cmu.minorthird.classify.experiments.CrossValSplitter) CrossValSplitter(edu.cmu.minorthird.classify.experiments.CrossValSplitter) RandomElement(edu.cmu.minorthird.classify.algorithms.random.RandomElement)

Aggregations

ClassifierLearner (edu.cmu.minorthird.classify.ClassifierLearner)2 Splitter (edu.cmu.minorthird.classify.Splitter)2 RandomElement (edu.cmu.minorthird.classify.algorithms.random.RandomElement)2 CrossValSplitter (edu.cmu.minorthird.classify.experiments.CrossValSplitter)2 CrossValidatedDataset (edu.cmu.minorthird.classify.experiments.CrossValidatedDataset)2 BasicDataset (edu.cmu.minorthird.classify.BasicDataset)1 Dataset (edu.cmu.minorthird.classify.Dataset)1 Evaluation (edu.cmu.minorthird.classify.experiments.Evaluation)1