Search in sources :

Example 16 with RebalanceDataSet

use of water.fvec.RebalanceDataSet in project h2o-2 by h2oai.

the class DRFTest method testReproducibility.

@Test
public void testReproducibility() {
    Frame tfr = null;
    final int N = 5;
    double[] mses = new double[N];
    Scope.enter();
    try {
        // Load data, hack frames
        tfr = parseFrame(Key.make("air.hex"), "./smalldata/covtype/covtype.20k.data");
        // rebalance to 256 chunks
        Key dest = Key.make("df.rebalanced.hex");
        RebalanceDataSet rb = new RebalanceDataSet(tfr, dest, 256);
        H2O.submitTask(rb);
        rb.join();
        tfr.delete();
        tfr = DKV.get(dest).get();
        for (int i = 0; i < N; ++i) {
            DRF parms = new DRF();
            parms.source = tfr;
            parms.response = tfr.lastVec();
            parms.nbins = 1000;
            parms.ntrees = 1;
            parms.max_depth = 8;
            parms.mtries = -1;
            parms.min_rows = 10;
            parms.classification = false;
            parms.seed = 1234;
            // Build a first model; all remaining models should be equal
            DRFModel drf = parms.fork().get();
            mses[i] = drf.mse();
            drf.delete();
        }
    } finally {
        if (tfr != null)
            tfr.delete();
    }
    Scope.exit();
    for (int i = 0; i < mses.length; ++i) {
        Log.info("trial: " + i + " -> mse: " + mses[i]);
    }
    for (int i = 0; i < mses.length; ++i) {
        assertEquals(mses[i], mses[0], 1e-15);
    }
}
Also used : Frame(water.fvec.Frame) DRFModel(hex.drf.DRF.DRFModel) RebalanceDataSet(water.fvec.RebalanceDataSet)

Aggregations

RebalanceDataSet (water.fvec.RebalanceDataSet)16 Frame (water.fvec.Frame)14 Test (org.junit.Test)11 SplitFrame (hex.SplitFrame)3 NFSFileVec (water.fvec.NFSFileVec)3 CreateFrame (hex.CreateFrame)2 DataInfo (hex.FrameTask.DataInfo)1 Aggregator (hex.aggregator.Aggregator)1 AggregatorModel (hex.aggregator.AggregatorModel)1 DeepLearningParameters (hex.deeplearning.DeepLearningModel.DeepLearningParameters)1 DRFModel (hex.drf.DRF.DRFModel)1 GLMModel (hex.glm.GLMModel)1 GramTask (hex.gram.Gram.GramTask)1 File (java.io.File)1 Key (water.Key)1 MRUtils.sampleFrame (water.util.MRUtils.sampleFrame)1 PrettyPrint (water.util.PrettyPrint)1