Search in sources :

Example 26 with EvalFunc

use of org.apache.pig.EvalFunc in project sketches-pig by DataSketches.

the class DataToSketchTest method algebraicIntermediateFromInitial.

@Test
public void algebraicIntermediateFromInitial() throws Exception {
    @SuppressWarnings("unchecked") EvalFunc<Tuple> func = (EvalFunc<Tuple>) Class.forName(new DataToSketch().getIntermed()).newInstance();
    DataBag outerBag = bagFactory.newDefaultBag();
    DataBag innerBag = bagFactory.newDefaultBag();
    innerBag.add(tupleFactory.newTuple("a"));
    innerBag.add(tupleFactory.newTuple("b"));
    innerBag.add(tupleFactory.newTuple("c"));
    outerBag.add(tupleFactory.newTuple(innerBag));
    Tuple result = func.exec(tupleFactory.newTuple(outerBag));
    HllSketch sketch = getSketch((DataByteArray) result.get(0));
    Assert.assertFalse(sketch.isEmpty());
    Assert.assertEquals(sketch.getEstimate(), 3.0, 0.01);
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) DataBag(org.apache.pig.data.DataBag) EvalFunc(org.apache.pig.EvalFunc) Tuple(org.apache.pig.data.Tuple) Test(org.testng.annotations.Test)

Example 27 with EvalFunc

use of org.apache.pig.EvalFunc in project sketches-pig by DataSketches.

the class DataToSketchTest method algebraicFinalEmptyInputTuple.

@Test
public void algebraicFinalEmptyInputTuple() throws Exception {
    @SuppressWarnings("unchecked") EvalFunc<DataByteArray> func = (EvalFunc<DataByteArray>) Class.forName(new DataToSketch().getFinal()).getConstructor(String.class).newInstance("10");
    DataByteArray result = func.exec(tupleFactory.newTuple());
    HllSketch sketch = getSketch(result);
    Assert.assertTrue(sketch.isEmpty());
    Assert.assertEquals(sketch.getLgConfigK(), 10);
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) EvalFunc(org.apache.pig.EvalFunc) DataByteArray(org.apache.pig.data.DataByteArray) Test(org.testng.annotations.Test)

Example 28 with EvalFunc

use of org.apache.pig.EvalFunc in project sketches-pig by DataSketches.

the class DataToSketchTest method algebraicInitial.

@Test
public void algebraicInitial() throws Exception {
    @SuppressWarnings("unchecked") EvalFunc<Tuple> func = (EvalFunc<Tuple>) Class.forName(new DataToSketch().getInitial()).newInstance();
    Tuple input = tupleFactory.newTuple();
    Tuple result = func.exec(input);
    Assert.assertEquals(result, input);
}
Also used : EvalFunc(org.apache.pig.EvalFunc) Tuple(org.apache.pig.data.Tuple) Test(org.testng.annotations.Test)

Example 29 with EvalFunc

use of org.apache.pig.EvalFunc in project sketches-pig by DataSketches.

the class UnionSketchTest method algebraicFinalEmptyInputTuple.

@Test
public void algebraicFinalEmptyInputTuple() throws Exception {
    @SuppressWarnings("unchecked") EvalFunc<DataByteArray> func = (EvalFunc<DataByteArray>) Class.forName(new UnionSketch().getFinal()).getConstructor(String.class).newInstance("10");
    DataByteArray result = func.exec(tupleFactory.newTuple());
    HllSketch sketch = DataToSketchTest.getSketch(result);
    Assert.assertTrue(sketch.isEmpty());
    Assert.assertEquals(sketch.getLgConfigK(), 10);
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) EvalFunc(org.apache.pig.EvalFunc) DataByteArray(org.apache.pig.data.DataByteArray) Test(org.testng.annotations.Test)

Example 30 with EvalFunc

use of org.apache.pig.EvalFunc in project sketches-pig by DataSketches.

the class UnionSketchTest method algebraicInitial.

@Test
public void algebraicInitial() throws Exception {
    @SuppressWarnings("unchecked") EvalFunc<Tuple> func = (EvalFunc<Tuple>) Class.forName(new UnionSketch().getInitial()).newInstance();
    Tuple input = tupleFactory.newTuple();
    Tuple result = func.exec(input);
    Assert.assertEquals(result, input);
}
Also used : EvalFunc(org.apache.pig.EvalFunc) Tuple(org.apache.pig.data.Tuple) Test(org.testng.annotations.Test)

Aggregations

EvalFunc (org.apache.pig.EvalFunc)44 Test (org.testng.annotations.Test)44 Tuple (org.apache.pig.data.Tuple)35 HllSketch (com.yahoo.sketches.hll.HllSketch)18 DataBag (org.apache.pig.data.DataBag)16 DataByteArray (org.apache.pig.data.DataByteArray)16 FuncSpec (org.apache.pig.FuncSpec)12 ItemsSketch (com.yahoo.sketches.quantiles.ItemsSketch)2 BagFactory (org.apache.pig.data.BagFactory)2 TupleFactory (org.apache.pig.data.TupleFactory)2 DataToSketch (com.yahoo.sketches.pig.theta.DataToSketch)1 PigUtil.tupleToSketch (com.yahoo.sketches.pig.theta.PigUtil.tupleToSketch)1 Sketch (com.yahoo.sketches.theta.Sketch)1 Schema (org.apache.pig.impl.logicalLayer.schema.Schema)1