Search in sources :

Example 66 with Vec

use of water.fvec.Vec in project h2o-3 by h2oai.

the class AstModuloKFold method apply.

@Override
public ValFrame apply(Env env, Env.StackHelp stk, AstRoot[] asts) {
    Vec foldVec = stk.track(asts[1].exec(env)).getFrame().anyVec().makeZero();
    int nfolds = (int) asts[2].exec(env).getNum();
    return new ValFrame(new Frame(AstKFold.moduloKfoldColumn(foldVec, nfolds)));
}
Also used : ValFrame(water.rapids.vals.ValFrame) ValFrame(water.rapids.vals.ValFrame) Frame(water.fvec.Frame) Vec(water.fvec.Vec)

Example 67 with Vec

use of water.fvec.Vec in project h2o-3 by h2oai.

the class AstQtile method apply.

@Override
public ValFrame apply(Env env, Env.StackHelp stk, AstRoot[] asts) {
    QuantileModel.QuantileParameters parms = new QuantileModel.QuantileParameters();
    Frame fr = stk.track(asts[1].exec(env)).getFrame();
    // Force a bogus Key for Quantiles ModelBuilder
    Frame fr_wkey = new Frame(fr);
    DKV.put(fr_wkey);
    parms._train = fr_wkey._key;
    parms._probs = ((AstNumList) asts[2]).expand();
    for (double d : parms._probs) if (d < 0 || d > 1)
        throw new IllegalArgumentException("Probability must be between 0 and 1: " + d);
    String inter = asts[3].exec(env).getStr();
    parms._combine_method = QuantileModel.CombineMethod.valueOf(inter.toUpperCase());
    parms._weights_column = asts[4].str().equals("_") ? null : asts[4].str();
    // Compute Quantiles
    Job j = new Quantile(parms).trainModel();
    QuantileModel q = (QuantileModel) j.get();
    DKV.remove(j._key);
    // Remove bogus Key
    DKV.remove(fr_wkey._key);
    // Reshape all outputs as a Frame, with probs in col 0 and the
    // quantiles in cols 1 thru fr.numCols() - except the optional weights vec
    int ncols = fr.numCols();
    if (parms._weights_column != null)
        ncols--;
    Vec[] vecs = new Vec[1 + /*1 more for the probs themselves*/
    ncols];
    String[] names = new String[vecs.length];
    vecs[0] = Vec.makeCon(null, parms._probs);
    names[0] = "Probs";
    int w = 0;
    for (int i = 0; i < vecs.length - 1; ++i) {
        if (fr._names[i].equals(parms._weights_column))
            w = 1;
        vecs[i + 1] = Vec.makeCon(null, q._output._quantiles[i]);
        names[i + 1] = fr._names[w + i] + "Quantiles";
    }
    q.delete();
    return new ValFrame(new Frame(names, vecs));
}
Also used : ValFrame(water.rapids.vals.ValFrame) Frame(water.fvec.Frame) QuantileModel(hex.quantile.QuantileModel) ValFrame(water.rapids.vals.ValFrame) Vec(water.fvec.Vec) Job(water.Job) Quantile(hex.quantile.Quantile)

Example 68 with Vec

use of water.fvec.Vec in project h2o-3 by h2oai.

the class AstRunif method apply.

@Override
public ValFrame apply(Env env, Env.StackHelp stk, AstRoot[] asts) {
    Frame fr = stk.track(asts[1].exec(env)).getFrame();
    long seed = (long) asts[2].exec(env).getNum();
    if (seed == -1)
        seed = new Random().nextLong();
    return new ValFrame(new Frame(new String[] { "rnd" }, new Vec[] { fr.anyVec().makeRand(seed) }));
}
Also used : ValFrame(water.rapids.vals.ValFrame) ValFrame(water.rapids.vals.ValFrame) Frame(water.fvec.Frame) Random(java.util.Random) Vec(water.fvec.Vec)

Example 69 with Vec

use of water.fvec.Vec in project h2o-3 by h2oai.

the class AstSkewness method apply.

// (skewness ary na.rm)
@Override
public ValNums apply(Env env, Env.StackHelp stk, AstRoot[] asts) {
    Frame fr = stk.track(asts[1].exec(env)).getFrame();
    boolean narm = asts[2].exec(env).getNum() == 1;
    double[] ds = new double[fr.numCols()];
    Vec[] vecs = fr.vecs();
    for (int i = 0; i < fr.numCols(); i++) ds[i] = (!vecs[i].isNumeric() || vecs[i].length() == 0 || (!narm && vecs[i].naCnt() > 0)) ? Double.NaN : skewness(vecs[i]);
    return new ValNums(ds);
}
Also used : Frame(water.fvec.Frame) Vec(water.fvec.Vec) ValNums(water.rapids.vals.ValNums)

Example 70 with Vec

use of water.fvec.Vec in project h2o-3 by h2oai.

the class AstTable method apply.

@Override
public ValFrame apply(Env env, Env.StackHelp stk, AstRoot[] asts) {
    Frame fr1 = stk.track(asts[1].exec(env)).getFrame();
    final boolean dense = asts[asts.length - 1].exec(env).getNum() == 1;
    Frame fr2 = asts.length == 4 ? stk.track(asts[2].exec(env)).getFrame() : null;
    int ncols = fr1.numCols() + (fr2 == null ? 0 : fr2.numCols());
    Vec vec1 = fr1.vec(0);
    ValFrame res = fast_table(vec1, ncols, fr1._names[0]);
    if (res != null)
        return res;
    if (!(asts.length == 3 || asts.length == 4) || ncols > 2)
        throw new IllegalArgumentException("table expects one or two columns");
    Vec vec2 = fr1.numCols() == 2 ? fr1.vec(1) : fr2 != null ? fr2.vec(0) : null;
    int sz = fr1._names.length + (fr2 != null ? fr2._names.length : 0);
    String[] colnames = new String[sz];
    int i = 0;
    for (String name : fr1._names) colnames[i++] = name;
    if (fr2 != null)
        for (String name : fr2._names) colnames[i++] = name;
    return slow_table(vec1, vec2, colnames, dense);
}
Also used : ValFrame(water.rapids.vals.ValFrame) ValFrame(water.rapids.vals.ValFrame) Frame(water.fvec.Frame) Vec(water.fvec.Vec)

Aggregations

Vec (water.fvec.Vec)280 Frame (water.fvec.Frame)213 Test (org.junit.Test)82 NFSFileVec (water.fvec.NFSFileVec)48 ValFrame (water.rapids.vals.ValFrame)47 Chunk (water.fvec.Chunk)30 Random (java.util.Random)25 NewChunk (water.fvec.NewChunk)23 DeepLearningParameters (hex.deeplearning.DeepLearningModel.DeepLearningParameters)22 Key (water.Key)21 MRTask (water.MRTask)17 Val (water.rapids.Val)14 File (java.io.File)11 ArrayList (java.util.ArrayList)11 Futures (water.Futures)11 H2OIllegalArgumentException (water.exceptions.H2OIllegalArgumentException)11 ValNum (water.rapids.vals.ValNum)11 ShuffleSplitFrame (hex.splitframe.ShuffleSplitFrame)10 BufferedString (water.parser.BufferedString)10 AppendableVec (water.fvec.AppendableVec)9