Examples with LongsSketch com.yahoo.sketches.frequencies.LongsSketch used on opensource projects

Example 1 with LongsSketch

use of com.yahoo.sketches.frequencies.LongsSketch in project Gaffer by gchq.

the class LongsSketchAggregatorTest method testCloneOfBusySketch.

@Test
public void testCloneOfBusySketch() {
    final LongsSketchAggregator sketchAggregator = new LongsSketchAggregator();
    sketchAggregator.init();
    for (int i = 0; i < 100; i++) {
        final LongsSketch union = new LongsSketch(32);
        for (int j = 0; j < 100; j++) {
            union.update(j);
        }
        sketchAggregator._aggregate(union);
    }
    final LongsSketchAggregator clone = sketchAggregator.statelessClone();
    assertNotSame(sketchAggregator, clone);
    clone._aggregate(sketch1);
    assertEquals(sketch1.getEstimate(1L), ((LongsSketch) clone.state()[0]).getEstimate(1L));
}

Example 2 with LongsSketch

use of com.yahoo.sketches.frequencies.LongsSketch in project Gaffer by gchq.

the class LongsSketchSerialiserTest method testSerialiseAndDeserialise.

@Test
public void testSerialiseAndDeserialise() {
    final LongsSketch sketch = new LongsSketch(32);
    sketch.update(1L);
    sketch.update(2L);
    sketch.update(3L);
    testSerialiser(sketch);
    final LongsSketch emptySketch = new LongsSketch(32);
    testSerialiser(emptySketch);
}

Example 3 with LongsSketch

use of com.yahoo.sketches.frequencies.LongsSketch in project Gaffer by gchq.

the class LongsSketchSerialiserTest method testSerialiser.

private void testSerialiser(final LongsSketch sketch) {
    final long freqOf1 = sketch.getEstimate(1L);
    final byte[] sketchSerialised;
    try {
        sketchSerialised = SERIALISER.serialise(sketch);
    } catch (final SerialisationException exception) {
        fail("A SerialisationException occurred");
        return;
    }
    final LongsSketch sketchDeserialised;
    try {
        sketchDeserialised = SERIALISER.deserialise(sketchSerialised);
    } catch (final SerialisationException exception) {
        fail("A SerialisationException occurred");
        return;
    }
    assertEquals(freqOf1, sketchDeserialised.getEstimate(1L));
}

Example 4 with LongsSketch

use of com.yahoo.sketches.frequencies.LongsSketch in project Gaffer by gchq.

the class LoadAndQuery10 method run.

public Iterable<Entity> run() throws OperationException {
    // [user] Create a user
    // ---------------------------------------------------------
    final User user = new User("user01");
    // ---------------------------------------------------------
    // [graph] create a graph using our schema and store properties
    // ---------------------------------------------------------
    final Graph graph = new Graph.Builder().addSchemas(getSchemas()).storeProperties(getStoreProperties()).build();
    // ---------------------------------------------------------
    // [add] add the edges to the graph
    // ---------------------------------------------------------
    final Set<String> dummyData = Collections.singleton("");
    final OperationChain addOpChain = new OperationChain.Builder().first(new GenerateElements.Builder<String>().generator(new DataGenerator10()).objects(dummyData).build()).then(new AddElements()).build();
    graph.execute(addOpChain, user);
    // ---------------------------------------------------------
    log("Added an edge A-B 1000 times, each time with a LongsSketch containing a random long between 0 and 9.");
    // [get] Get all edges
    // ---------------------------------------------------------
    Iterable<Edge> allEdges = graph.execute(new GetAllEdges(), user);
    // ---------------------------------------------------------
    log("\nAll edges:");
    for (final Edge edge : allEdges) {
        log("GET_ALL_EDGES_RESULT", edge.toString());
    }
    // [get frequencies of 1L and 9L] Get the edge A-B and print estimates of frequencies of 1L and 9L
    // ---------------------------------------------------------
    final GetEdges<EdgeSeed> query = new GetEdges.Builder<EdgeSeed>().addSeed(new EdgeSeed("A", "B", false)).build();
    final Iterable<Edge> edges = graph.execute(query, user);
    final Edge edge = edges.iterator().next();
    final LongsSketch longsSketch = (LongsSketch) edge.getProperty("longsSketch");
    final String estimates = "Edge A-B: 1L seen approximately " + longsSketch.getEstimate(1L) + " times, 9L seen approximately " + longsSketch.getEstimate(9L) + " times.";
    // ---------------------------------------------------------
    log("\nEdge A-B with estimates of the frequencies of 1 and 9");
    log("GET_FREQUENCIES_OF_1_AND_9_FOR_EDGE_A_B", estimates);
    return null;
}

Example 5 with LongsSketch

use of com.yahoo.sketches.frequencies.LongsSketch in project Gaffer by gchq.

the class DataGenerator10 method getElements.

@Override
public Iterable<Element> getElements(final String line) {
    final Set<Element> elements = new HashSet<>();
    for (int i = 0; i < 1000; i++) {
        final LongsSketch longsSketch = new LongsSketch(32);
        longsSketch.update((long) (RANDOM.nextDouble() * 10));
        final Edge edge = new Edge.Builder().group("red").source("A").dest("B").property("longsSketch", longsSketch).build();
        elements.add(edge);
    }
    return elements;
}