Search in sources :

Example 1 with UpdateDoublesSketch

use of com.yahoo.sketches.quantiles.UpdateDoublesSketch in project sketches-pig by DataSketches.

the class GetPmfFromDoublesSketchTest method normalCase.

@Test
public void normalCase() throws Exception {
    EvalFunc<Tuple> func = new GetPmfFromDoublesSketch();
    UpdateDoublesSketch sketch = DoublesSketch.builder().build();
    for (int i = 1; i <= 10; i++) sketch.update(i);
    Tuple resultTuple = func.exec(tupleFactory.newTuple(Arrays.asList(new DataByteArray(sketch.toByteArray()), 2.0, 7.0)));
    Assert.assertNotNull(resultTuple);
    Assert.assertEquals(resultTuple.size(), 3);
    Assert.assertEquals(((double) resultTuple.get(0)), 0.1);
    Assert.assertEquals(((double) resultTuple.get(1)), 0.5);
    Assert.assertEquals(((double) resultTuple.get(2)), 0.4);
}
Also used : UpdateDoublesSketch(com.yahoo.sketches.quantiles.UpdateDoublesSketch) DataByteArray(org.apache.pig.data.DataByteArray) Tuple(org.apache.pig.data.Tuple) Test(org.testng.annotations.Test)

Example 2 with UpdateDoublesSketch

use of com.yahoo.sketches.quantiles.UpdateDoublesSketch in project sketches-pig by DataSketches.

the class GetQuantileFromDoublesSketchTest method normalCase.

@Test
public void normalCase() throws Exception {
    EvalFunc<Double> func = new GetQuantileFromDoublesSketch();
    UpdateDoublesSketch sketch = DoublesSketch.builder().build();
    sketch.update(1.0);
    Double result = func.exec(tupleFactory.newTuple(Arrays.asList(new DataByteArray(sketch.toByteArray()), 0.5)));
    Assert.assertEquals(result, 1.0);
}
Also used : UpdateDoublesSketch(com.yahoo.sketches.quantiles.UpdateDoublesSketch) DataByteArray(org.apache.pig.data.DataByteArray) Test(org.testng.annotations.Test)

Example 3 with UpdateDoublesSketch

use of com.yahoo.sketches.quantiles.UpdateDoublesSketch in project sketches-pig by DataSketches.

the class GetQuantilesFromDoublesSketchTest method oneFraction.

@Test
public void oneFraction() throws Exception {
    EvalFunc<Tuple> func = new GetQuantilesFromDoublesSketch();
    UpdateDoublesSketch sketch = DoublesSketch.builder().build();
    for (int i = 1; i <= 10; i++) sketch.update(i);
    Tuple resultTuple = func.exec(tupleFactory.newTuple(Arrays.asList(new DataByteArray(sketch.toByteArray()), 0.5)));
    Assert.assertNotNull(resultTuple);
    Assert.assertEquals(resultTuple.size(), 1);
    Assert.assertEquals(((double) resultTuple.get(0)), 6.0);
}
Also used : UpdateDoublesSketch(com.yahoo.sketches.quantiles.UpdateDoublesSketch) DataByteArray(org.apache.pig.data.DataByteArray) Tuple(org.apache.pig.data.Tuple) Test(org.testng.annotations.Test)

Example 4 with UpdateDoublesSketch

use of com.yahoo.sketches.quantiles.UpdateDoublesSketch in project sketches-pig by DataSketches.

the class GetQuantilesFromDoublesSketchTest method severalFractions.

@Test
public void severalFractions() throws Exception {
    EvalFunc<Tuple> func = new GetQuantilesFromDoublesSketch();
    UpdateDoublesSketch sketch = DoublesSketch.builder().build();
    for (int i = 1; i <= 10; i++) sketch.update(i);
    Tuple resultTuple = func.exec(tupleFactory.newTuple(Arrays.asList(new DataByteArray(sketch.toByteArray()), 0.0, 0.5, 1.0)));
    Assert.assertNotNull(resultTuple);
    Assert.assertEquals(resultTuple.size(), 3);
    Assert.assertEquals(((double) resultTuple.get(0)), 1.0);
    Assert.assertEquals(((double) resultTuple.get(1)), 6.0);
    Assert.assertEquals(((double) resultTuple.get(2)), 10.0);
}
Also used : UpdateDoublesSketch(com.yahoo.sketches.quantiles.UpdateDoublesSketch) DataByteArray(org.apache.pig.data.DataByteArray) Tuple(org.apache.pig.data.Tuple) Test(org.testng.annotations.Test)

Example 5 with UpdateDoublesSketch

use of com.yahoo.sketches.quantiles.UpdateDoublesSketch in project sketches-pig by DataSketches.

the class UnionDoublesSketchTest method execNormalCase.

@Test
public void execNormalCase() throws Exception {
    EvalFunc<Tuple> func = new UnionDoublesSketch();
    DataBag bag = bagFactory.newDefaultBag();
    UpdateDoublesSketch inputSketch = DoublesSketch.builder().build();
    inputSketch.update(1.0);
    bag.add(tupleFactory.newTuple(new DataByteArray(inputSketch.toByteArray())));
    Tuple resultTuple = func.exec(tupleFactory.newTuple(bag));
    DoublesSketch sketch = getSketch(resultTuple);
    Assert.assertFalse(sketch.isEmpty());
    Assert.assertEquals(sketch.getN(), 1);
}
Also used : DoublesSketch(com.yahoo.sketches.quantiles.DoublesSketch) UpdateDoublesSketch(com.yahoo.sketches.quantiles.UpdateDoublesSketch) DataBag(org.apache.pig.data.DataBag) UpdateDoublesSketch(com.yahoo.sketches.quantiles.UpdateDoublesSketch) DataByteArray(org.apache.pig.data.DataByteArray) Tuple(org.apache.pig.data.Tuple) Test(org.testng.annotations.Test)

Aggregations

UpdateDoublesSketch (com.yahoo.sketches.quantiles.UpdateDoublesSketch)14 DataByteArray (org.apache.pig.data.DataByteArray)11 Test (org.testng.annotations.Test)9 Tuple (org.apache.pig.data.Tuple)8 DoublesSketch (com.yahoo.sketches.quantiles.DoublesSketch)6 DataBag (org.apache.pig.data.DataBag)4 Test (org.junit.jupiter.api.Test)2 DoublesSketchBuilder (com.yahoo.sketches.quantiles.DoublesSketchBuilder)1 ArrayOfDoublesSketch (com.yahoo.sketches.tuple.ArrayOfDoublesSketch)1 ArrayOfDoublesSketchIterator (com.yahoo.sketches.tuple.ArrayOfDoublesSketchIterator)1 DoubleSummary (com.yahoo.sketches.tuple.DoubleSummary)1 ArrayList (java.util.ArrayList)1 Edge (uk.gov.gchq.gaffer.data.element.Edge)1 Element (uk.gov.gchq.gaffer.data.element.Element)1 BinaryOperatorTest (uk.gov.gchq.koryphe.binaryoperator.BinaryOperatorTest)1