Search in sources :

Example 6 with BulletRecord

use of com.yahoo.bullet.record.BulletRecord in project bullet-core by yahoo.

the class GroupDataTest method testNoRecordAvg.

@Test
public void testNoRecordAvg() {
    GroupData data = make(new GroupOperation(GroupOperation.GroupOperationType.AVG, "someField", "avg"));
    // AVG will return null if no records are observed
    BulletRecord expected = RecordBox.get().addNull("avg").getRecord();
    Assert.assertEquals(data.getMetricsAsBulletRecord(), expected);
}
Also used : BulletRecord(com.yahoo.bullet.record.BulletRecord) Test(org.testng.annotations.Test)

Example 7 with BulletRecord

use of com.yahoo.bullet.record.BulletRecord in project bullet-core by yahoo.

the class GroupDataTest method testMergingMetric.

@Test
public void testMergingMetric() {
    GroupData data = make(new GroupOperation(GroupOperation.GroupOperationType.COUNT, "shouldBeIgnored", "myCount"), new GroupOperation(GroupOperation.GroupOperationType.MIN, "groupField", "myMin"), new GroupOperation(GroupOperation.GroupOperationType.MAX, "groupField", "myMax"), new GroupOperation(GroupOperation.GroupOperationType.SUM, "groupField", "mySum"), new GroupOperation(GroupOperation.GroupOperationType.AVG, "groupField", "myAvg"));
    asList(0.0, -8.8, 51.0).stream().map(x -> RecordBox.get().add("groupField", x).getRecord()).forEach(data::consume);
    GroupData another = make(new GroupOperation(GroupOperation.GroupOperationType.COUNT, "alsoIgnored", "myCount"), new GroupOperation(GroupOperation.GroupOperationType.MIN, "groupField", "myMin"), new GroupOperation(GroupOperation.GroupOperationType.MAX, "groupField", "myMax"), new GroupOperation(GroupOperation.GroupOperationType.SUM, "groupField", "mySum"), new GroupOperation(GroupOperation.GroupOperationType.AVG, "groupField", "myAvg"));
    asList(1.1, 4.4, -44.0, 12345.67, 3.3).stream().map(x -> RecordBox.get().add("groupField", x).getRecord()).forEach(data::consume);
    byte[] serialized = SerializerDeserializer.toBytes(another);
    data.combine(serialized);
    BulletRecord expected = RecordBox.get().add("myCount", 8L).add("myMin", -44.0).add("myMax", 12345.67).add("mySum", 12352.67).add("myAvg", 1544.08375).getRecord();
    Assert.assertTrue(expected.equals(data.getMetricsAsBulletRecord()));
}
Also used : IntStream(java.util.stream.IntStream) BulletRecord(com.yahoo.bullet.record.BulletRecord) SerializerDeserializer(com.yahoo.bullet.common.SerializerDeserializer) Test(org.testng.annotations.Test) HashMap(java.util.HashMap) RecordBox(com.yahoo.bullet.result.RecordBox) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) List(java.util.List) Pair(org.apache.commons.lang3.tuple.Pair) Assert(org.testng.Assert) Arrays.asList(java.util.Arrays.asList) BulletConfig(com.yahoo.bullet.common.BulletConfig) Map(java.util.Map) Collections(java.util.Collections) BulletRecord(com.yahoo.bullet.record.BulletRecord) Test(org.testng.annotations.Test)

Example 8 with BulletRecord

use of com.yahoo.bullet.record.BulletRecord in project bullet-core by yahoo.

the class GroupDataTest method testNoRecordMin.

@Test
public void testNoRecordMin() {
    GroupData data = make(new GroupOperation(GroupOperation.GroupOperationType.MIN, null, "min"));
    // MIN will return null if no records are observed
    BulletRecord expected = RecordBox.get().addNull("min").getRecord();
    Assert.assertEquals(data.getMetricsAsBulletRecord(), expected);
}
Also used : BulletRecord(com.yahoo.bullet.record.BulletRecord) Test(org.testng.annotations.Test)

Example 9 with BulletRecord

use of com.yahoo.bullet.record.BulletRecord in project bullet-core by yahoo.

the class GroupDataTest method testGroupFieldsInData.

@Test
public void testGroupFieldsInData() {
    Map<String, String> fields = new HashMap<>();
    fields.put("fieldA", "foo");
    fields.put("fieldB", "bar");
    GroupData data = make(fields, new GroupOperation(GroupOperation.GroupOperationType.SUM, "someField", "sum"));
    BulletRecord expectedUnmapped = RecordBox.get().addNull("sum").getRecord();
    Map<String, String> fieldMapping = new HashMap<>();
    fieldMapping.put("fieldA", "newFieldNameA");
    fieldMapping.put("fieldB", "fieldB");
    BulletRecord expected = RecordBox.get().add("newFieldNameA", "foo").add("fieldB", "bar").addNull("sum").getRecord();
    Assert.assertTrue(data.getMetricsAsBulletRecord().equals(expectedUnmapped));
    Assert.assertTrue(data.getAsBulletRecord(fieldMapping).equals(expected));
    data.consume(RecordBox.get().add("someField", 21.0).getRecord());
    data.consume(RecordBox.get().add("someField", 21.0).getRecord());
    data.consume(RecordBox.get().addNull("someField").getRecord());
    expected = RecordBox.get().add("foo", "foo").add("bar", "bar").add("sum", 42.0).getRecord();
    Assert.assertTrue(data.getAsBulletRecord(fields).equals(expected));
}
Also used : BulletRecord(com.yahoo.bullet.record.BulletRecord) HashMap(java.util.HashMap) Test(org.testng.annotations.Test)

Example 10 with BulletRecord

use of com.yahoo.bullet.record.BulletRecord in project bullet-core by yahoo.

the class GroupDataTest method testMergingRawMetricFail.

@Test
public void testMergingRawMetricFail() {
    GroupData data = make(new GroupOperation(GroupOperation.GroupOperationType.COUNT, null, null));
    BulletRecord someRecord = RecordBox.get().add("foo", 1).getRecord();
    IntStream.range(0, 10).forEach(i -> data.consume(someRecord));
    // Not a serialized GroupData
    data.combine(String.valueOf(242).getBytes());
    // Unchanged count
    BulletRecord expected = RecordBox.get().add(GroupOperation.GroupOperationType.COUNT.getName(), 10L).getRecord();
    Assert.assertEquals(data.getMetricsAsBulletRecord(), expected);
}
Also used : BulletRecord(com.yahoo.bullet.record.BulletRecord) Test(org.testng.annotations.Test)

Aggregations

BulletRecord (com.yahoo.bullet.record.BulletRecord)210 Test (org.testng.annotations.Test)196 Tuple (org.apache.storm.tuple.Tuple)55 CountDistinctTest (com.yahoo.bullet.aggregations.CountDistinctTest)53 DistributionTest (com.yahoo.bullet.aggregations.DistributionTest)53 TopKTest (com.yahoo.bullet.aggregations.TopKTest)53 Clip (com.yahoo.bullet.result.Clip)53 HashMap (java.util.HashMap)53 Map (java.util.Map)53 BulletConfig (com.yahoo.bullet.common.BulletConfig)46 List (java.util.List)45 IntStream (java.util.stream.IntStream)45 Assert (org.testng.Assert)45 RecordBox (com.yahoo.bullet.result.RecordBox)43 Arrays.asList (java.util.Arrays.asList)40 Pair (org.apache.commons.lang3.tuple.Pair)40 AggregationUtils.makeAttributes (com.yahoo.bullet.parsing.AggregationUtils.makeAttributes)34 BulletError (com.yahoo.bullet.common.BulletError)33 Aggregation (com.yahoo.bullet.parsing.Aggregation)33 Concept (com.yahoo.bullet.result.Meta.Concept)33