Examples with Union com.yahoo.sketches.theta.Union used on opensource projects

Example 11 with Union

use of com.yahoo.sketches.theta.Union in project Gaffer by gchq.

the class UnionAggregatorTest method testAggregate.

@Test
public void testAggregate() {
    final UnionAggregator unionAggregator = new UnionAggregator();
    Union currentUnion = Sketches.setOperationBuilder().buildUnion();
    currentUnion.update("A");
    currentUnion.update("B");
    assertEquals(2.0D, currentUnion.getResult().getEstimate(), DELTA);
    Union newUnion = Sketches.setOperationBuilder().buildUnion();
    newUnion.update("C");
    newUnion.update("D");
    currentUnion = unionAggregator.apply(currentUnion, newUnion);
    assertEquals(4.0D, currentUnion.getResult().getEstimate(), DELTA);
}

Example 12 with Union

use of com.yahoo.sketches.theta.Union in project Gaffer by gchq.

the class UnionConverterTest method testConverter.

@Test
public void testConverter() throws ConversionException {
    final Union union = SetOperation.builder().buildUnion();
    union.update(1.0D);
    union.update(2.0D);
    union.update(3.0D);
    assertEquals(union.getResult().getEstimate(), UNION_CONVERTER.convert(union), DELTA);
    final Union emptyUnion = SetOperation.builder().buildUnion();
    assertEquals(emptyUnion.getResult().getEstimate(), UNION_CONVERTER.convert(emptyUnion), DELTA);
}

Example 13 with Union

use of com.yahoo.sketches.theta.Union in project Gaffer by gchq.

the class UnionSerialiser method deserialise.

@Override
public Union deserialise(final byte[] bytes) throws SerialisationException {
    final Union union = Sketches.setOperationBuilder().buildUnion();
    union.update(WritableMemory.wrap(bytes));
    return union;
}

Example 14 with Union

use of com.yahoo.sketches.theta.Union in project Gaffer by gchq.

the class LoadAndQuery13 method run.

public Iterable<Entity> run() throws OperationException {
    // [user] Create a user
    // ---------------------------------------------------------
    final User user = new User("user01");
    // ---------------------------------------------------------
    // [graph] create a graph using our schema and store properties
    // ---------------------------------------------------------
    final Graph graph = new Graph.Builder().addSchemas(getSchemas()).storeProperties(getStoreProperties()).build();
    // ---------------------------------------------------------
    // [add] add the edges to the graph
    // ---------------------------------------------------------
    final Set<String> dummyData = Collections.singleton("");
    final OperationChain addOpChain = new OperationChain.Builder().first(new GenerateElements.Builder<String>().generator(new DataGenerator13()).objects(dummyData).build()).then(new AddElements()).build();
    graph.execute(addOpChain, user);
    // ---------------------------------------------------------
    log("Added 1000 edges A-B0, A-B1, ..., A-B999 on 10/1/17. For each edge we create an Entity with a union sketch" + " containing a string of the source and destination from the edge. Added 500 edges A-B750, A-B751, " + "..., A-B1249 for day 11/1/17. Again for each edge we create an Entity with a union sketch.");
    // [get entities] Get the entities for separate days
    // ---------------------------------------------------------
    final GetAllEntities get = new GetAllEntities();
    final Iterable<Entity> entities = graph.execute(get, user);
    for (final Entity entity : entities) {
        log("GET_ENTITIES", entity.toString());
    }
    // ---------------------------------------------------------
    // [get estimate separate days] Get the estimates out of the sketches for the separate days
    // ---------------------------------------------------------
    final GetAllEntities getAllEntities2 = new GetAllEntities();
    final Iterable<Entity> allEntities2 = graph.execute(getAllEntities2, user);
    final Iterator<Entity> it = allEntities2.iterator();
    final Entity entityDay1 = it.next();
    final CompactSketch sketchDay1 = ((Union) entityDay1.getProperty("size")).getResult();
    final Entity entityDay2 = it.next();
    final CompactSketch sketchDay2 = ((Union) entityDay2.getProperty("size")).getResult();
    final double estimateDay1 = sketchDay1.getEstimate();
    final double estimateDay2 = sketchDay2.getEstimate();
    // ---------------------------------------------------------
    log("\nThe estimates for the different days");
    log("GET_ESTIMATE_OVER_SEPARATE_DAYS", "" + estimateDay1);
    log("GET_ESTIMATE_OVER_SEPARATE_DAYS", "" + estimateDay2);
    // [get intersection] Get the number of edges in common across the two days
    // ---------------------------------------------------------
    final Intersection intersection = Sketches.setOperationBuilder().buildIntersection();
    intersection.update(sketchDay1);
    intersection.update(sketchDay2);
    final double intersectionSizeEstimate = intersection.getResult().getEstimate();
    // ---------------------------------------------------------
    log("\nThe estimate of the number of edges in common across the different days");
    log("PRINT_ESTIMATE", "" + intersectionSizeEstimate);
    // [get union across all days] Get the total number edges across the two days
    // ---------------------------------------------------------
    final GetAllEntities getAllEntities = new GetAllEntities.Builder().view(new View.Builder().entity("size", new ViewElementDefinition.Builder().groupBy().build()).build()).build();
    final Iterable<Entity> allEntities = graph.execute(getAllEntities, user);
    final Entity entity = allEntities.iterator().next();
    final double unionSizeEstimate = ((Union) entity.getProperty("size")).getResult().getEstimate();
    // ---------------------------------------------------------
    log("\nThe estimate of the number of edges across the different days");
    log("UNION_ESTIMATE", "" + unionSizeEstimate);
    return null;
}

Example 15 with Union

use of com.yahoo.sketches.theta.Union in project druid by druid-io.

the class SketchBufferAggregator method aggregate.

@Override
public void aggregate(ByteBuffer buf, int position) {
    Object update = selector.get();
    if (update == null) {
        return;
    }
    Union union = getUnion(buf, position);
    SketchAggregator.updateUnion(union, update);
}