Search in sources :

Example 21 with HllSketch

use of com.yahoo.sketches.hll.HllSketch in project Gaffer by gchq.

the class HllpSketchJsonSerialisationTest method testValidHyperLogLogPlusSketchSerialisedCorrectly.

@Test
public void testValidHyperLogLogPlusSketchSerialisedCorrectly() throws IOException {
    // Given
    final String testString = "TestString";
    final HllSketch sketch = new HllSketch(10);
    sketch.update(testString);
    // When / Then
    runTestWithSketch(sketch);
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) Test(org.junit.jupiter.api.Test)

Example 22 with HllSketch

use of com.yahoo.sketches.hll.HllSketch in project Gaffer by gchq.

the class HllpSketchJsonSerialisationTest method testEmptyHyperLogLogPlusSketchIsSerialised.

@Test
public void testEmptyHyperLogLogPlusSketchIsSerialised() throws IOException {
    // Given
    final HllSketch sketch = new HllSketch(10);
    // When / Then
    runTestWithSketch(sketch);
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) Test(org.junit.jupiter.api.Test)

Example 23 with HllSketch

use of com.yahoo.sketches.hll.HllSketch in project Gaffer by gchq.

the class IterableToHllSketchTest method shouldCreateHllSketch.

@Test
public void shouldCreateHllSketch() {
    // Given
    IterableToHllSketch iterableToHllSketch = new IterableToHllSketch();
    List<Object> input = Arrays.asList("one", "two", "three", "four", "five");
    // When
    HllSketch result = iterableToHllSketch.apply(input);
    // Then
    assertThat(result.getEstimate()).isCloseTo(5, Percentage.withPercentage(0.001));
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) Test(org.junit.jupiter.api.Test) FunctionTest(uk.gov.gchq.koryphe.function.FunctionTest)

Example 24 with HllSketch

use of com.yahoo.sketches.hll.HllSketch in project Gaffer by gchq.

the class IterableToHllSketchTest method shouldCreateHllSketchCardinality.

@Test
public void shouldCreateHllSketchCardinality() {
    // Given
    IterableToHllSketch iterableToHllSketch = new IterableToHllSketch();
    List<Object> input = Arrays.asList("one", "one", "two", "two", "three");
    // When
    HllSketch result = iterableToHllSketch.apply(input);
    // Then
    assertThat(result.getEstimate()).isCloseTo(3, Percentage.withPercentage(0.001));
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) Test(org.junit.jupiter.api.Test) FunctionTest(uk.gov.gchq.koryphe.function.FunctionTest)

Example 25 with HllSketch

use of com.yahoo.sketches.hll.HllSketch in project Gaffer by gchq.

the class HllSketchIsLessThanTest method setup.

@BeforeAll
public static void setup() {
    hllSketchWithCardinality5 = new HllSketch(10);
    for (int i = 1; i <= 5; i++) {
        hllSketchWithCardinality5.update(i);
    }
    assertEquals(5d, hllSketchWithCardinality5.getEstimate(), DELTA);
    hllSketchWithCardinality18 = new HllSketch(10);
    for (int i = 1; i <= 18; i++) {
        hllSketchWithCardinality18.update(i);
    }
    assertEquals(18d, hllSketchWithCardinality18.getEstimate(), DELTA);
    hllSketchWithCardinality32 = new HllSketch(10);
    for (int i = 1; i <= 32; i++) {
        hllSketchWithCardinality32.update(i);
    }
    assertEquals(32d, hllSketchWithCardinality32.getEstimate(), DELTA);
}
Also used : HllSketch(com.yahoo.sketches.hll.HllSketch) BeforeAll(org.junit.jupiter.api.BeforeAll)

Aggregations

HllSketch (com.yahoo.sketches.hll.HllSketch)58 Test (org.testng.annotations.Test)31 DataByteArray (org.apache.pig.data.DataByteArray)30 EvalFunc (org.apache.pig.EvalFunc)18 Test (org.junit.jupiter.api.Test)15 DataBag (org.apache.pig.data.DataBag)14 Tuple (org.apache.pig.data.Tuple)12 FunctionTest (uk.gov.gchq.koryphe.function.FunctionTest)5 Entity (uk.gov.gchq.gaffer.data.element.Entity)3 Union (com.yahoo.sketches.hll.Union)2 Edge (uk.gov.gchq.gaffer.data.element.Edge)2 Element (uk.gov.gchq.gaffer.data.element.Element)2 TreeNode (com.fasterxml.jackson.core.TreeNode)1 JsonNode (com.fasterxml.jackson.databind.JsonNode)1 ArrayNode (com.fasterxml.jackson.databind.node.ArrayNode)1 IntNode (com.fasterxml.jackson.databind.node.IntNode)1 TextNode (com.fasterxml.jackson.databind.node.TextNode)1 ArrayList (java.util.ArrayList)1 BeforeAll (org.junit.jupiter.api.BeforeAll)1 HllSketchElementGenerator (uk.gov.gchq.gaffer.doc.properties.generator.HllSketchElementGenerator)1