Search in sources :

Example 1 with CustomCollector

use of com.yahoo.bullet.storm.testing.CustomCollector in project bullet-storm by yahoo.

the class FilterBoltTest method testNoConsumptionAfterDone.

@Test
public void testNoConsumptionAfterDone() {
    Tuple query = makeIDTuple(TupleClassifier.Type.QUERY_TUPLE, "42", makeSimpleAggregationFilterQuery("field", singletonList("b235gf23b"), EQUALS, RAW, 5, Window.Unit.RECORD, 1, Window.Unit.RECORD, 1), METADATA);
    bolt.execute(query);
    BulletRecord record = RecordBox.get().add("field", "b235gf23b").getRecord();
    Tuple matching = makeRecordTuple(record);
    bolt.execute(matching);
    bolt.execute(matching);
    bolt.execute(matching);
    Tuple expected = makeSlidingTuple(TupleClassifier.Type.DATA_TUPLE, "42", record);
    Assert.assertTrue(wasRawRecordEmittedTo(TopologyConstants.DATA_STREAM, 3, expected));
    collector = new CustomCollector();
    // Will be done after 2 consumes (no ticks)
    bolt = ComponentUtils.prepare(new DonableFilterBolt(2, 1, new BulletStormConfig()), collector);
    bolt.execute(query);
    bolt.execute(matching);
    bolt.execute(matching);
    // Now the query should be done, so it should not consume
    bolt.execute(matching);
    Assert.assertTrue(wasRawRecordEmittedTo(TopologyConstants.DATA_STREAM, 2, expected));
}
Also used : BulletRecord(com.yahoo.bullet.record.BulletRecord) CustomCollector(com.yahoo.bullet.storm.testing.CustomCollector) TupleUtils.makeTuple(com.yahoo.bullet.storm.testing.TupleUtils.makeTuple) TupleUtils.makeRawTuple(com.yahoo.bullet.storm.testing.TupleUtils.makeRawTuple) Tuple(org.apache.storm.tuple.Tuple) TupleUtils.makeIDTuple(com.yahoo.bullet.storm.testing.TupleUtils.makeIDTuple) Test(org.testng.annotations.Test) DistributionTest(com.yahoo.bullet.aggregations.DistributionTest) TopKTest(com.yahoo.bullet.aggregations.TopKTest) CountDistinctTest(com.yahoo.bullet.aggregations.CountDistinctTest)

Example 2 with CustomCollector

use of com.yahoo.bullet.storm.testing.CustomCollector in project bullet-storm by yahoo.

the class JoinBoltTest method setup.

public void setup(JoinBolt bolt) {
    collector = new CustomCollector();
    context = new CustomTopologyContext();
    this.bolt = ComponentUtils.prepare(new HashMap<>(), bolt, context, collector);
}
Also used : CustomCollector(com.yahoo.bullet.storm.testing.CustomCollector) HashMap(java.util.HashMap) CustomTopologyContext(com.yahoo.bullet.storm.testing.CustomTopologyContext)

Example 3 with CustomCollector

use of com.yahoo.bullet.storm.testing.CustomCollector in project bullet-storm by yahoo.

the class LoopBoltTest method setup.

@BeforeMethod
public void setup() {
    config = new BulletStormConfig("src/test/resources/test_config.yaml");
    bolt = new LoopBolt(config);
    collector = new CustomCollector();
    ComponentUtils.prepare(bolt, collector);
    publisher = (CustomPublisher) bolt.getPublisher();
}
Also used : CustomCollector(com.yahoo.bullet.storm.testing.CustomCollector) BeforeMethod(org.testng.annotations.BeforeMethod)

Example 4 with CustomCollector

use of com.yahoo.bullet.storm.testing.CustomCollector in project bullet-storm by yahoo.

the class QueryBoltTest method testNullMetaTupleIgnored.

@Test
public void testNullMetaTupleIgnored() {
    CustomCollector collector = new CustomCollector();
    TestQueryBolt bolt = new TestQueryBolt(new BulletStormConfig());
    ComponentUtils.prepare(bolt, collector);
    Map<String, Querier> queries = bolt.getQueries();
    queries.put("foo", null);
    Tuple meta = makeIDTuple(Type.METADATA_TUPLE, "foo", null);
    bolt.execute(meta);
    Assert.assertTrue(queries.containsKey("foo"));
}
Also used : Querier(com.yahoo.bullet.querying.Querier) CustomCollector(com.yahoo.bullet.storm.testing.CustomCollector) Tuple(org.apache.storm.tuple.Tuple) TupleUtils.makeIDTuple(com.yahoo.bullet.storm.testing.TupleUtils.makeIDTuple) Test(org.testng.annotations.Test)

Example 5 with CustomCollector

use of com.yahoo.bullet.storm.testing.CustomCollector in project bullet-storm by yahoo.

the class QueryBoltTest method testMetricsUpdateOnMetricsDisabled.

@Test
public void testMetricsUpdateOnMetricsDisabled() {
    CustomTopologyContext context = new CustomTopologyContext();
    CustomCollector collector = new CustomCollector();
    TestQueryBolt bolt = new TestQueryBolt(new BulletStormConfig());
    ComponentUtils.prepare(new HashMap<>(), bolt, context, collector);
    Assert.assertFalse(bolt.isMetricsEnabled());
    bolt.execute(null);
    IMetric averager = context.getRegisteredMetricByName("foo");
    IMetric counter = context.getRegisteredMetricByName("bar");
    Assert.assertNull(averager.getValueAndReset());
    Assert.assertEquals(counter.getValueAndReset(), 0L);
}
Also used : CustomCollector(com.yahoo.bullet.storm.testing.CustomCollector) IMetric(org.apache.storm.metric.api.IMetric) CustomTopologyContext(com.yahoo.bullet.storm.testing.CustomTopologyContext) Test(org.testng.annotations.Test)

Aggregations

CustomCollector (com.yahoo.bullet.storm.testing.CustomCollector)12 Test (org.testng.annotations.Test)8 TupleUtils.makeIDTuple (com.yahoo.bullet.storm.testing.TupleUtils.makeIDTuple)6 Tuple (org.apache.storm.tuple.Tuple)6 Querier (com.yahoo.bullet.querying.Querier)4 CustomTopologyContext (com.yahoo.bullet.storm.testing.CustomTopologyContext)4 BeforeMethod (org.testng.annotations.BeforeMethod)4 Metadata (com.yahoo.bullet.pubsub.Metadata)3 CountDistinctTest (com.yahoo.bullet.aggregations.CountDistinctTest)2 DistributionTest (com.yahoo.bullet.aggregations.DistributionTest)2 TopKTest (com.yahoo.bullet.aggregations.TopKTest)2 CountDistinct (com.yahoo.bullet.aggregations.CountDistinct)1 Distribution (com.yahoo.bullet.aggregations.Distribution)1 TopK (com.yahoo.bullet.aggregations.TopK)1 GroupData (com.yahoo.bullet.aggregations.grouping.GroupData)1 GroupOperation (com.yahoo.bullet.aggregations.grouping.GroupOperation)1 COUNT (com.yahoo.bullet.aggregations.grouping.GroupOperation.GroupOperationType.COUNT)1 COUNT_FIELD (com.yahoo.bullet.aggregations.sketches.QuantileSketch.COUNT_FIELD)1 END_EXCLUSIVE (com.yahoo.bullet.aggregations.sketches.QuantileSketch.END_EXCLUSIVE)1 NEGATIVE_INFINITY_START (com.yahoo.bullet.aggregations.sketches.QuantileSketch.NEGATIVE_INFINITY_START)1