Examples with QuantifyDoubleFunc com.tencent.angel.ml.psf.compress.QuantifyDoubleFunc used on opensource projects

Example 1 with QuantifyDoubleFunc

use of com.tencent.angel.ml.psf.compress.QuantifyDoubleFunc in project angel by Tencent.

the class GradHistThread method run.

@Override
public void run() {
    LOG.debug(String.format("Run active node[%d]", this.nid));
    // 1. name of this node's grad histogram on PS
    String histParaName = this.controller.param.gradHistNamePrefix + nid;
    // 2. build the grad histogram of this node
    GradHistHelper histMaker = new GradHistHelper(this.controller, this.nid);
    IntDoubleVector histogram = histMaker.buildHistogram(insStart, insEnd);
    int bytesPerItem = this.controller.taskContext.getConf().getInt(MLConf.ANGEL_COMPRESS_BYTES(), MLConf.DEFAULT_ANGEL_COMPRESS_BYTES());
    if (bytesPerItem < 1 || bytesPerItem > 8) {
        LOG.info("Invalid compress configuration: " + bytesPerItem + ", it should be [1,8].");
        bytesPerItem = MLConf.DEFAULT_ANGEL_COMPRESS_BYTES();
    }
    // 3. push the histograms to PS
    try {
        if (bytesPerItem == 8) {
            this.model.increment(0, histogram);
        } else {
            QuantifyDoubleFunc func = new QuantifyDoubleFunc(this.model.getMatrixId(), 0, histogram, bytesPerItem * 8);
            this.model.update(func);
        }
    } catch (Exception e) {
        LOG.error(histParaName + " increment failed, ", e);
    }
    // 4. reset thread stats to finished
    this.controller.activeNodeStat[this.nid].decrementAndGet();
    LOG.debug(String.format("Active node[%d] finish", this.nid));
}

Example 2 with QuantifyDoubleFunc

use of com.tencent.angel.ml.psf.compress.QuantifyDoubleFunc in project angel by Tencent.

the class GBDTController method pushHistogram.

private void pushHistogram(int nid, int bytesPerItem) {
    String histParaName = this.param.gradHistNamePrefix + nid;
    PSModel histMat = this.model.getPSModel(histParaName);
    try {
        if (bytesPerItem == 8) {
            histMat.increment(0, this.histCache[nid]);
        } else {
            QuantifyDoubleFunc func = new QuantifyDoubleFunc(histMat.getMatrixId(), 0, this.histCache[nid], bytesPerItem * 8);
            histMat.update(func);
        }
    } catch (Exception e) {
        LOG.error(histParaName + " increment failed, ", e);
    }
}