Search in sources :

Example 56 with HllSketch

use of com.yahoo.sketches.hll.HllSketch in project Gaffer by gchq.

the class HllSketchAggregatorTest method testAggregate.

@Test
public void testAggregate() {
    final HllSketchAggregator sketchAggregator = new HllSketchAggregator();
    HllSketch currentSketch = new HllSketch(10);
    currentSketch.update("A");
    currentSketch.update("B");
    assertEquals(2.0D, currentSketch.getEstimate(), DELTA);
    HllSketch newSketch = new HllSketch(10);
    newSketch.update("C");
    newSketch.update("D");
    currentSketch = sketchAggregator.apply(currentSketch, newSketch);
    assertEquals(4.0D, currentSketch.getEstimate(), DELTA);
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) Test(org.junit.jupiter.api.Test) BinaryOperatorTest(uk.gov.gchq.koryphe.binaryoperator.BinaryOperatorTest)

Example 57 with HllSketch

use of com.yahoo.sketches.hll.HllSketch in project Gaffer by gchq.

the class IterableToHllSketchTest method shouldCreateEmptyWhenNull.

@Test
public void shouldCreateEmptyWhenNull() {
    // Given
    IterableToHllSketch iterableToHllSketch = new IterableToHllSketch();
    // When
    HllSketch result = iterableToHllSketch.apply(null);
    // Then
    assertThat(result.getEstimate()).isEqualTo(0);
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) Test(org.junit.jupiter.api.Test) FunctionTest(uk.gov.gchq.koryphe.function.FunctionTest)

Example 58 with HllSketch

use of com.yahoo.sketches.hll.HllSketch in project Gaffer by gchq.

the class HllpSketchJsonSerialisationTest method shouldDeserialiseNewHllpWithValues.

@Test
public void shouldDeserialiseNewHllpWithValues() throws IOException {
    // Given
    final String sketchAsString = "{\"logK\": 20, \"values\": [\"value1\", \"value2\", \"value2\", \"value2\", \"value3\"]}";
    // When
    HllSketch hllp = JSONSerialiser.deserialise(sketchAsString, HllSketch.class);
    // Then
    HllSketch expected = new HllSketch(20);
    expected.update("value1");
    expected.update("value2");
    expected.update("value2");
    expected.update("value2");
    expected.update("value3");
    assertArrayEquals(expected.toCompactByteArray(), hllp.toCompactByteArray());
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) Test(org.junit.jupiter.api.Test)

Aggregations

HllSketch (com.yahoo.sketches.hll.HllSketch)58 Test (org.testng.annotations.Test)31 DataByteArray (org.apache.pig.data.DataByteArray)30 EvalFunc (org.apache.pig.EvalFunc)18 Test (org.junit.jupiter.api.Test)15 DataBag (org.apache.pig.data.DataBag)14 Tuple (org.apache.pig.data.Tuple)12 FunctionTest (uk.gov.gchq.koryphe.function.FunctionTest)5 Entity (uk.gov.gchq.gaffer.data.element.Entity)3 Union (com.yahoo.sketches.hll.Union)2 Edge (uk.gov.gchq.gaffer.data.element.Edge)2 Element (uk.gov.gchq.gaffer.data.element.Element)2 TreeNode (com.fasterxml.jackson.core.TreeNode)1 JsonNode (com.fasterxml.jackson.databind.JsonNode)1 ArrayNode (com.fasterxml.jackson.databind.node.ArrayNode)1 IntNode (com.fasterxml.jackson.databind.node.IntNode)1 TextNode (com.fasterxml.jackson.databind.node.TextNode)1 ArrayList (java.util.ArrayList)1 BeforeAll (org.junit.jupiter.api.BeforeAll)1 HllSketchElementGenerator (uk.gov.gchq.gaffer.doc.properties.generator.HllSketchElementGenerator)1