Search in sources :

Example 71 with MultiLabel

use of edu.neu.ccs.pyramid.dataset.MultiLabel in project pyramid by cheng-li.

the class CBMNoiseOptimizerFixed method updateBinaryTarget.

private void updateBinaryTarget(int dataPointIndex) {
    double[] comProb = targets[dataPointIndex];
    double[] marginals = new double[cbm.getNumClasses()];
    for (int c = 0; c < comProb.length; c++) {
        MultiLabel multiLabel = combinations.get(c);
        double prob = comProb[c];
        for (int l : multiLabel.getMatchedLabels()) {
            marginals[l] += prob;
        }
    }
    for (int l = 0; l < cbm.getNumClasses(); l++) {
        // we need to add some protection
        if (marginals[l] > 1) {
            marginals[l] = 1;
        }
        binaryTargetsDistributions[l][dataPointIndex][0] = 1 - marginals[l];
        binaryTargetsDistributions[l][dataPointIndex][1] = marginals[l];
    }
}
Also used : MultiLabel(edu.neu.ccs.pyramid.dataset.MultiLabel)

Example 72 with MultiLabel

use of edu.neu.ccs.pyramid.dataset.MultiLabel in project pyramid by cheng-li.

the class CBMS method predictByMarginals.

/**
     * sort marginals, and keep top few
     * @param vector
     * @param top
     * @return
     */
public MultiLabel predictByMarginals(Vector vector, int top) {
    double[] probs = predictClassProbs(vector);
    int[] sortedIndices = ArgSort.argSortDescending(probs);
    MultiLabel prediction = new MultiLabel();
    for (int i = 0; i < top; i++) {
        prediction.addLabel(sortedIndices[i]);
    }
    return prediction;
}
Also used : MultiLabel(edu.neu.ccs.pyramid.dataset.MultiLabel)

Example 73 with MultiLabel

use of edu.neu.ccs.pyramid.dataset.MultiLabel in project pyramid by cheng-li.

the class CBMS method predictLogAssignmentProbs.

/**
     * for batch jobs, use this to save computation
     * @param x
     * @param assignments
     * @return
     */
private double[] predictLogAssignmentProbs(Vector x, List<MultiLabel> assignments) {
    BMDistribution bmDistribution = computeBM(x);
    double[] probs = new double[assignments.size()];
    for (int c = 0; c < assignments.size(); c++) {
        MultiLabel multiLabel = assignments.get(c);
        probs[c] = bmDistribution.logProbability(multiLabel);
    }
    return probs;
}
Also used : MultiLabel(edu.neu.ccs.pyramid.dataset.MultiLabel)

Example 74 with MultiLabel

use of edu.neu.ccs.pyramid.dataset.MultiLabel in project pyramid by cheng-li.

the class CBMUtilityOptimizer method updateGamma.

private void updateGamma(int n) {
    Vector x = dataSet.getRow(n);
    BMDistribution bmDistribution = cbm.computeBM(x);
    // size = combination * components
    List<double[]> logPosteriors = new ArrayList<>();
    for (int c = 0; c < combinations.size(); c++) {
        MultiLabel combination = combinations.get(c);
        double[] pos = bmDistribution.logPosteriorMembership(combination);
        logPosteriors.add(pos);
    }
    double[] sums = new double[cbm.numComponents];
    for (int k = 0; k < cbm.numComponents; k++) {
        double sum = 0;
        for (int c = 0; c < combinations.size(); c++) {
            sum += targets[n][c] * logPosteriors.get(c)[k];
        }
        sums[k] = sum;
    }
    double[] posterior = MathUtil.softmax(sums);
    for (int k = 0; k < cbm.numComponents; k++) {
        gammas[n][k] = posterior[k];
        gammasT[k][n] = posterior[k];
    }
}
Also used : MultiLabel(edu.neu.ccs.pyramid.dataset.MultiLabel) ArrayList(java.util.ArrayList) Vector(org.apache.mahout.math.Vector)

Example 75 with MultiLabel

use of edu.neu.ccs.pyramid.dataset.MultiLabel in project pyramid by cheng-li.

the class CBMUtilityOptimizer method updateBinaryTarget.

private void updateBinaryTarget(int dataPointIndex) {
    double[] comProb = targets[dataPointIndex];
    double[] marginals = new double[cbm.getNumClasses()];
    for (int c = 0; c < comProb.length; c++) {
        MultiLabel multiLabel = combinations.get(c);
        double prob = comProb[c];
        for (int l : multiLabel.getMatchedLabels()) {
            marginals[l] += prob;
        }
    }
    for (int l = 0; l < cbm.getNumClasses(); l++) {
        // we need to add some protection
        if (marginals[l] > 1) {
            marginals[l] = 1;
        }
        binaryTargetsDistributions[l][dataPointIndex][0] = 1 - marginals[l];
        binaryTargetsDistributions[l][dataPointIndex][1] = marginals[l];
    }
}
Also used : MultiLabel(edu.neu.ccs.pyramid.dataset.MultiLabel)

Aggregations

MultiLabel (edu.neu.ccs.pyramid.dataset.MultiLabel)101 Vector (org.apache.mahout.math.Vector)22 MultiLabelClfDataSet (edu.neu.ccs.pyramid.dataset.MultiLabelClfDataSet)21 File (java.io.File)14 DenseVector (org.apache.mahout.math.DenseVector)13 CMLCRF (edu.neu.ccs.pyramid.multilabel_classification.crf.CMLCRF)12 Pair (edu.neu.ccs.pyramid.util.Pair)8 LBFGS (edu.neu.ccs.pyramid.optimization.LBFGS)7 ArrayList (java.util.ArrayList)7 MLMeasures (edu.neu.ccs.pyramid.eval.MLMeasures)6 CRFLoss (edu.neu.ccs.pyramid.multilabel_classification.crf.CRFLoss)6 MultiLabelClassifier (edu.neu.ccs.pyramid.multilabel_classification.MultiLabelClassifier)5 GeneralF1Predictor (edu.neu.ccs.pyramid.multilabel_classification.plugin_rule.GeneralF1Predictor)5 Collectors (java.util.stream.Collectors)5 EarlyStopper (edu.neu.ccs.pyramid.optimization.EarlyStopper)4 java.util (java.util)4 StopWatch (org.apache.commons.lang3.time.StopWatch)4 Config (edu.neu.ccs.pyramid.configuration.Config)3 DataSetUtil (edu.neu.ccs.pyramid.dataset.DataSetUtil)3 TRECFormat (edu.neu.ccs.pyramid.dataset.TRECFormat)3