Search in sources :

Example 76 with StopWatch

use of org.apache.commons.lang3.time.StopWatch in project pyramid by cheng-li.

the class MLPlattScalingTest method test1.

private static void test1() throws Exception {
    MultiLabelClfDataSet dataSet = TRECFormat.loadMultiLabelClfDataSet(new File(DATASETS, "ohsumed/3/train.trec"), DataSetType.ML_CLF_SPARSE, true);
    IMLGradientBoosting boosting = new IMLGradientBoosting(dataSet.getNumClasses());
    List<MultiLabel> assignments = DataSetUtil.gatherMultiLabels(dataSet);
    boosting.setAssignments(assignments);
    IMLGBConfig trainConfig = new IMLGBConfig.Builder(dataSet).numLeaves(2).learningRate(0.1).numSplitIntervals(1000).minDataPerLeaf(2).dataSamplingRate(1).featureSamplingRate(1).build();
    IMLGBTrainer trainer = new IMLGBTrainer(trainConfig, boosting);
    StopWatch stopWatch = new StopWatch();
    stopWatch.start();
    for (int round = 0; round < 100; round++) {
        System.out.println("round=" + round);
        trainer.iterate();
        System.out.println(stopWatch);
    }
    MLPlattScaling plattScaling = new MLPlattScaling(dataSet, boosting);
    for (int i = 0; i < 10; i++) {
        System.out.println(Arrays.toString(boosting.predictClassScores(dataSet.getRow(i))));
        System.out.println(Arrays.toString(boosting.predictClassProbs(dataSet.getRow(i))));
        System.out.println(Arrays.toString(plattScaling.predictClassProbs(dataSet.getRow(i))));
        System.out.println("======================");
    }
}
Also used : IMLGradientBoosting(edu.neu.ccs.pyramid.multilabel_classification.imlgb.IMLGradientBoosting) IMLGBTrainer(edu.neu.ccs.pyramid.multilabel_classification.imlgb.IMLGBTrainer) IMLGBConfig(edu.neu.ccs.pyramid.multilabel_classification.imlgb.IMLGBConfig) File(java.io.File) StopWatch(org.apache.commons.lang3.time.StopWatch)

Example 77 with StopWatch

use of org.apache.commons.lang3.time.StopWatch in project pyramid by cheng-li.

the class AdaBoostMHTest method test1.

static void test1() throws Exception {
    MultiLabelClfDataSet dataSet = TRECFormat.loadMultiLabelClfDataSet(new File(DATASETS, "ohsumed/3/train.trec"), DataSetType.ML_CLF_SPARSE, true);
    MultiLabelClfDataSet testSet = TRECFormat.loadMultiLabelClfDataSet(new File(DATASETS, "ohsumed/3/test.trec"), DataSetType.ML_CLF_SPARSE, true);
    AdaBoostMH boosting = new AdaBoostMH(dataSet.getNumClasses());
    AdaBoostMHTrainer trainer = new AdaBoostMHTrainer(dataSet, boosting);
    StopWatch stopWatch = new StopWatch();
    stopWatch.start();
    for (int round = 0; round < 500; round++) {
        System.out.println("round=" + round);
        trainer.iterate();
        System.out.println(stopWatch);
    }
    System.out.println("training accuracy=" + Accuracy.accuracy(boosting, dataSet));
    System.out.println("training overlap = " + Overlap.overlap(boosting, dataSet));
    System.out.println("test accuracy=" + Accuracy.accuracy(boosting, testSet));
    System.out.println("test overlap = " + Overlap.overlap(boosting, testSet));
}
Also used : File(java.io.File) StopWatch(org.apache.commons.lang3.time.StopWatch)

Aggregations

StopWatch (org.apache.commons.lang3.time.StopWatch)76 File (java.io.File)47 ArrayList (java.util.ArrayList)17 ClfDataSet (edu.neu.ccs.pyramid.dataset.ClfDataSet)8 Vector (org.apache.mahout.math.Vector)8 VirtualMachine (com.microsoft.azure.management.compute.VirtualMachine)7 Creatable (com.microsoft.azure.management.resources.fluentcore.model.Creatable)7 Config (edu.neu.ccs.pyramid.configuration.Config)7 Network (com.microsoft.azure.management.network.Network)6 ResourceGroup (com.microsoft.azure.management.resources.ResourceGroup)5 PriorProbClassifier (edu.neu.ccs.pyramid.classification.PriorProbClassifier)5 IOException (java.io.IOException)5 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4 Region (com.microsoft.azure.management.resources.fluentcore.arm.Region)4 StorageAccount (com.microsoft.azure.management.storage.StorageAccount)4 PublicIPAddress (com.microsoft.azure.management.network.PublicIPAddress)3 LogisticRegression (edu.neu.ccs.pyramid.classification.logistic_regression.LogisticRegression)3 edu.neu.ccs.pyramid.dataset (edu.neu.ccs.pyramid.dataset)3 IntStream (java.util.stream.IntStream)3 SearchResponse (org.elasticsearch.action.search.SearchResponse)3