Search in sources :

Example 21 with Meta

use of com.yahoo.bullet.result.Meta in project bullet-storm by yahoo.

the class JoinBoltTest method testQueryIdentifierMetadata.

@Test
public void testQueryIdentifierMetadata() {
    config = configWithRawMaxAndEmptyMeta();
    enableMetadataInConfig(config, Concept.QUERY_METADATA.getName(), "meta");
    enableMetadataInConfig(config, Concept.QUERY_ID.getName(), "id");
    setup(new JoinBolt(config));
    Tuple query = TupleUtils.makeIDTuple(TupleClassifier.Type.QUERY_TUPLE, "42", "{}", EMPTY);
    bolt.execute(query);
    List<BulletRecord> sent = sendRawRecordTuplesTo(bolt, "42");
    Meta meta = new Meta();
    meta.add("meta", singletonMap("id", "42"));
    Tuple expected = TupleUtils.makeTuple(TupleClassifier.Type.RESULT_TUPLE, "42", Clip.of(sent).add(meta).asJSON(), COMPLETED);
    Assert.assertTrue(wasResultEmittedTo(TopologyConstants.RESULT_STREAM, expected));
    Tuple metadata = TupleUtils.makeTuple(TupleClassifier.Type.FEEDBACK_TUPLE, "42", new Metadata(Metadata.Signal.COMPLETE, null));
    Assert.assertTrue(wasMetadataEmittedTo(TopologyConstants.FEEDBACK_STREAM, metadata));
    Assert.assertEquals(collector.getAllEmittedTo(TopologyConstants.RESULT_STREAM).count(), 1);
    Assert.assertEquals(collector.getAllEmittedTo(TopologyConstants.FEEDBACK_STREAM).count(), 1);
}
Also used : Meta(com.yahoo.bullet.result.Meta) BulletRecord(com.yahoo.bullet.record.BulletRecord) Metadata(com.yahoo.bullet.pubsub.Metadata) Tuple(org.apache.storm.tuple.Tuple) Test(org.testng.annotations.Test) GroupByTest(com.yahoo.bullet.aggregations.GroupByTest) DistributionTest(com.yahoo.bullet.aggregations.DistributionTest) TopKTest(com.yahoo.bullet.aggregations.TopKTest) CountDistinctTest(com.yahoo.bullet.aggregations.CountDistinctTest)

Example 22 with Meta

use of com.yahoo.bullet.result.Meta in project bullet-storm by yahoo.

the class JoinBoltTest method testUnknownAggregation.

@Test
public void testUnknownAggregation() {
    // Lowercase "top" is not valid and will not be parsed since there is no enum for it
    // In this case aggregation type should be set to null and an error should be emitted
    Tuple query = TupleUtils.makeIDTuple(TupleClassifier.Type.QUERY_TUPLE, "42", "{\"aggregation\": {\"type\": \"garbage\"}}", EMPTY);
    bolt.execute(query);
    Assert.assertEquals(collector.getEmittedCount(), 1);
    BulletError expectedError = Aggregation.TYPE_NOT_SUPPORTED_ERROR;
    Meta expectedMetadata = Meta.of(expectedError);
    List<Object> expected = TupleUtils.makeTuple("42", Clip.of(expectedMetadata).asJSON(), FAILED).getValues();
    List<Object> actual = collector.getNthTupleEmittedTo(TopologyConstants.RESULT_STREAM, 1).get();
    Assert.assertTrue(isSameResult(actual, expected));
}
Also used : Meta(com.yahoo.bullet.result.Meta) JsonObject(com.google.gson.JsonObject) BulletError(com.yahoo.bullet.common.BulletError) Tuple(org.apache.storm.tuple.Tuple) Test(org.testng.annotations.Test) GroupByTest(com.yahoo.bullet.aggregations.GroupByTest) DistributionTest(com.yahoo.bullet.aggregations.DistributionTest) TopKTest(com.yahoo.bullet.aggregations.TopKTest) CountDistinctTest(com.yahoo.bullet.aggregations.CountDistinctTest)

Example 23 with Meta

use of com.yahoo.bullet.result.Meta in project bullet-storm by yahoo.

the class JoinBoltTest method testErrorEmittedProperly.

@Test
public void testErrorEmittedProperly() {
    Tuple query = TupleUtils.makeIDTuple(TupleClassifier.Type.QUERY_TUPLE, "42", "garbage", EMPTY);
    bolt.execute(query);
    Assert.assertEquals(collector.getEmittedCount(), 1);
    String error = ParsingError.GENERIC_JSON_ERROR + ":\ngarbage\n" + "IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 1 path $";
    BulletError expectedError = ParsingError.makeError(error, ParsingError.GENERIC_JSON_RESOLUTION);
    Meta expectedMetadata = Meta.of(expectedError);
    List<Object> expected = TupleUtils.makeTuple("42", Clip.of(expectedMetadata).asJSON(), FAILED).getValues();
    List<Object> actual = collector.getNthTupleEmittedTo(TopologyConstants.RESULT_STREAM, 1).get();
    Assert.assertTrue(isSameResult(actual, expected));
}
Also used : Meta(com.yahoo.bullet.result.Meta) JsonObject(com.google.gson.JsonObject) BulletError(com.yahoo.bullet.common.BulletError) Tuple(org.apache.storm.tuple.Tuple) Test(org.testng.annotations.Test) GroupByTest(com.yahoo.bullet.aggregations.GroupByTest) DistributionTest(com.yahoo.bullet.aggregations.DistributionTest) TopKTest(com.yahoo.bullet.aggregations.TopKTest) CountDistinctTest(com.yahoo.bullet.aggregations.CountDistinctTest)

Example 24 with Meta

use of com.yahoo.bullet.result.Meta in project bullet-storm by yahoo.

the class JoinBoltTest method testUnhandledExceptionErrorEmitted.

@Test
public void testUnhandledExceptionErrorEmitted() {
    // An empty query should throw an null-pointer exception which should be caught in JoinBolt
    // and an error should be emitted
    Tuple query = TupleUtils.makeIDTuple(TupleClassifier.Type.QUERY_TUPLE, "42", "", EMPTY);
    bolt.execute(query);
    sendRawRecordTuplesTo(bolt, "42");
    Assert.assertEquals(collector.getEmittedCount(), 1);
    String error = ParsingError.GENERIC_JSON_ERROR + ":\n\nNullPointerException: ";
    BulletError expectedError = ParsingError.makeError(error, ParsingError.GENERIC_JSON_RESOLUTION);
    Meta expectedMetadata = Meta.of(expectedError);
    List<Object> expected = TupleUtils.makeTuple("42", Clip.of(expectedMetadata).asJSON(), FAILED).getValues();
    List<Object> actual = collector.getNthTupleEmittedTo(TopologyConstants.RESULT_STREAM, 1).get();
    Assert.assertTrue(isSameResult(actual, expected));
}
Also used : Meta(com.yahoo.bullet.result.Meta) JsonObject(com.google.gson.JsonObject) BulletError(com.yahoo.bullet.common.BulletError) Tuple(org.apache.storm.tuple.Tuple) Test(org.testng.annotations.Test) GroupByTest(com.yahoo.bullet.aggregations.GroupByTest) DistributionTest(com.yahoo.bullet.aggregations.DistributionTest) TopKTest(com.yahoo.bullet.aggregations.TopKTest) CountDistinctTest(com.yahoo.bullet.aggregations.CountDistinctTest)

Aggregations

Meta (com.yahoo.bullet.result.Meta)24 Test (org.testng.annotations.Test)20 Map (java.util.Map)11 Clip (com.yahoo.bullet.result.Clip)8 BulletRecord (com.yahoo.bullet.record.BulletRecord)7 CountDistinctTest (com.yahoo.bullet.aggregations.CountDistinctTest)6 DistributionTest (com.yahoo.bullet.aggregations.DistributionTest)6 GroupByTest (com.yahoo.bullet.aggregations.GroupByTest)6 TopKTest (com.yahoo.bullet.aggregations.TopKTest)6 HashMap (java.util.HashMap)6 Tuple (org.apache.storm.tuple.Tuple)6 BulletError (com.yahoo.bullet.common.BulletError)5 Family (com.yahoo.sketches.Family)5 SketchesArgumentException (com.yahoo.sketches.SketchesArgumentException)5 ErrorType (com.yahoo.sketches.frequencies.ErrorType)5 List (java.util.List)5 IntStream (java.util.stream.IntStream)5 Assert (org.testng.Assert)5 JsonObject (com.google.gson.JsonObject)4 Metadata (com.yahoo.bullet.pubsub.Metadata)3