Search in sources :

Example 51 with structures._Review

use of structures._Review in project IR_Base by Linda-sunshine.

the class CLRWithHDP method updateDocMembership.

public void updateDocMembership(_HDPAdaptStruct user, _Review r) {
    int index = -1;
    _HDPThetaStar curThetaStar = r.getHDPThetaStar();
    // remove the current review from the user side.
    user.incHDPThetaStarMemSize(r.getHDPThetaStar(), -1);
    // remove the current review from the theta side.
    // remove the lm stat first before decrease the document count
    curThetaStar.rmLMStat(r.getLMSparse());
    curThetaStar.updateMemCount(-1);
    // No data associated with the cluster
    if (curThetaStar.getMemSize() == 0) {
        // check if every dim gets 0 count in language model
        LMStatSanityCheck(curThetaStar);
        // recycle the gamma
        m_gamma_e += curThetaStar.getGamma();
        curThetaStar.resetGamma();
        // swap the disabled theta to the last for later use
        index = findHDPThetaStar(curThetaStar);
        // move it back to \theta*
        swapTheta(m_kBar - 1, index);
        // in case we forget to init some variable, we set it to null
        curThetaStar = null;
        // curThetaStar.disable();
        m_kBar--;
    }
}
Also used : structures._HDPThetaStar(structures._HDPThetaStar)

Example 52 with structures._Review

use of structures._Review in project IR_Base by Linda-sunshine.

the class CLinAdaptWithHDP method gradientByFunc.

@Override
protected void gradientByFunc(_AdaptStruct u, _Doc review, double weight, double[] g) {
    _Review r = (_Review) review;
    // feature index
    int n, k;
    int cIndex = r.getHDPThetaStar().getIndex();
    if (cIndex < 0 || cIndex >= m_kBar)
        System.err.println("Error,cannot find the theta star!");
    int offset = m_dim * 2 * cIndex;
    double delta = (review.getYLabel() - logit(review.getSparse(), r)) * weight;
    // Bias term for individual user.
    // a[0] = ws0*x0; x0=1
    g[offset] -= delta * m_gWeights[0];
    // b[0]
    g[offset + m_dim] -= delta;
    // Traverse all the feature dimension to calculate the gradient for both individual users and super user.
    for (_SparseFeature fv : review.getSparse()) {
        n = fv.getIndex() + 1;
        k = m_featureGroupMap[n];
        // w_si*x_di
        g[offset + k] -= delta * m_gWeights[n] * fv.getValue();
        // x_di
        g[offset + m_dim + k] -= delta * fv.getValue();
    }
}
Also used : structures._Review(structures._Review) structures._SparseFeature(structures._SparseFeature)

Example 53 with structures._Review

use of structures._Review in project IR_Base by Linda-sunshine.

the class MTCLRWithHDP method gradientByFunc.

@Override
protected void gradientByFunc(_AdaptStruct u, _Doc review, double weight, double[] g) {
    _Review r = (_Review) review;
    // feature index
    int n;
    int cIndex = r.getHDPThetaStar().getIndex();
    if (cIndex < 0 || cIndex >= m_kBar)
        System.err.println("Error,cannot find the theta star!");
    int offset = m_dim * cIndex;
    int offsetSup = m_dim * m_kBar;
    double delta = weight * (r.getYLabel() - logit(r.getSparse(), r));
    // Bias term.
    // x0=1, each cluster.
    g[offset] -= delta;
    // super model.
    g[offsetSup] -= m_q * delta;
    // Traverse all the feature dimension to calculate the gradient.
    for (_SparseFeature fv : review.getSparse()) {
        n = fv.getIndex() + 1;
        // cluster model.
        g[offset + n] -= delta * fv.getValue();
        // super model.
        g[offsetSup + n] -= delta * fv.getValue() * m_q;
    }
}
Also used : structures._Review(structures._Review) structures._SparseFeature(structures._SparseFeature)

Example 54 with structures._Review

use of structures._Review in project IR_Base by Linda-sunshine.

the class MTCLinAdaptWithHDP method logit.

// Logit function is different from the father class.
@Override
protected double logit(_SparseFeature[] fvs, _Review r) {
    int k, n;
    double[] Au = r.getHDPThetaStar().getModel();
    // Bias term: w_s0*a0+b0.
    double sum = Au[0] * getSupWeights(0) + Au[m_dim];
    for (_SparseFeature fv : fvs) {
        n = fv.getIndex() + 1;
        k = m_featureGroupMap[n];
        sum += (Au[k] * getSupWeights(n) + Au[m_dim + k]) * fv.getValue();
    }
    return Utils.logistic(sum);
}
Also used : structures._SparseFeature(structures._SparseFeature)

Example 55 with structures._Review

use of structures._Review in project IR_Base by Linda-sunshine.

the class MTCLinAdaptWithHDPMultipleE method calcLogLikelihoodY.

// In function logLikelihood, we update the loglikelihood and corresponding gradients.
// Thus, we only need to update the two functions correspondingly with.
protected double calcLogLikelihoodY(_Review r) {
    int index = -1;
    _HDPThetaStar oldTheta = r.getHDPThetaStar();
    HashMap<_HDPThetaStar, Integer> thetaCountMap = r.getThetaCountMap();
    double likelihood = 0;
    for (_HDPThetaStar theta : thetaCountMap.keySet()) {
        index = findHDPThetaStar(theta);
        // some of the cluster may disappear, ignore them.
        if (index >= m_kBar || index < 0)
            continue;
        r.setHDPThetaStar(theta);
        // log(likelihood^k) = k * log likelihood.
        likelihood += thetaCountMap.get(theta) * super.calcLogLikelihoodY(r);
    }
    r.setHDPThetaStar(oldTheta);
    return likelihood;
}
Also used : structures._HDPThetaStar(structures._HDPThetaStar)

Aggregations

structures._Review (structures._Review)44 structures._SparseFeature (structures._SparseFeature)24 structures._HDPThetaStar (structures._HDPThetaStar)9 ArrayList (java.util.ArrayList)8 Feature (Classifier.supervised.liblinear.Feature)6 Classifier.supervised.modelAdaptation._AdaptStruct (Classifier.supervised.modelAdaptation._AdaptStruct)6 structures._PerformanceStat (structures._PerformanceStat)6 IOException (java.io.IOException)5 File (java.io.File)4 structures._User (structures._User)4 FeatureNode (Classifier.supervised.liblinear.FeatureNode)3 Parameter (Classifier.supervised.liblinear.Parameter)3 Problem (Classifier.supervised.liblinear.Problem)3 structures._RankItem (structures._RankItem)3 BufferedReader (java.io.BufferedReader)2 FileInputStream (java.io.FileInputStream)2 InputStreamReader (java.io.InputStreamReader)2 PrintWriter (java.io.PrintWriter)2 MyPriorityQueue (structures.MyPriorityQueue)2 SolverType (Classifier.supervised.liblinear.SolverType)1