Search in sources :

Example 31 with Fields

use of org.apache.storm.tuple.Fields in project storm by apache.

the class EsLookupBoltTest method fieldsAreDeclaredThroughProvidedOutput.

@Test
public void fieldsAreDeclaredThroughProvidedOutput() throws Exception {
    Fields fields = new Fields(UUID.randomUUID().toString());
    when(output.fields()).thenReturn(fields);
    OutputFieldsDeclarer declarer = mock(OutputFieldsDeclarer.class);
    bolt.declareOutputFields(declarer);
    ArgumentCaptor<Fields> declaredFields = ArgumentCaptor.forClass(Fields.class);
    verify(declarer).declare(declaredFields.capture());
    assertThat(declaredFields.getValue(), is(fields));
}
Also used : Fields(org.apache.storm.tuple.Fields) OutputFieldsDeclarer(org.apache.storm.topology.OutputFieldsDeclarer) Test(org.junit.Test)

Example 32 with Fields

use of org.apache.storm.tuple.Fields in project storm by apache.

the class AvroGenericRecordBoltTest method generateTestTuple.

private static Tuple generateTestTuple(GenericRecord record) {
    TopologyBuilder builder = new TopologyBuilder();
    GeneralTopologyContext topologyContext = new GeneralTopologyContext(builder.createTopology(), new Config(), new HashMap(), new HashMap(), new HashMap(), "") {

        @Override
        public Fields getComponentOutputFields(String componentId, String streamId) {
            return new Fields("record");
        }
    };
    return new TupleImpl(topologyContext, new Values(record), 1, "");
}
Also used : Fields(org.apache.storm.tuple.Fields) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) HashMap(java.util.HashMap) Config(org.apache.storm.Config) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 33 with Fields

use of org.apache.storm.tuple.Fields in project storm by apache.

the class TestHdfsBolt method generateTestTuple.

private Tuple generateTestTuple(Object id, Object msg, Object city, Object state) {
    TopologyBuilder builder = new TopologyBuilder();
    GeneralTopologyContext topologyContext = new GeneralTopologyContext(builder.createTopology(), new Config(), new HashMap(), new HashMap(), new HashMap(), "") {

        @Override
        public Fields getComponentOutputFields(String componentId, String streamId) {
            return new Fields("id", "msg", "city", "state");
        }
    };
    return new TupleImpl(topologyContext, new Values(id, msg, city, state), 1, "");
}
Also used : Fields(org.apache.storm.tuple.Fields) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) HashMap(java.util.HashMap) Config(org.apache.storm.Config) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 34 with Fields

use of org.apache.storm.tuple.Fields in project storm by apache.

the class TestSequenceFileBolt method generateTestTuple.

private Tuple generateTestTuple(Long key, String value) {
    TopologyBuilder builder = new TopologyBuilder();
    GeneralTopologyContext topologyContext = new GeneralTopologyContext(builder.createTopology(), new Config(), new HashMap(), new HashMap(), new HashMap(), "") {

        @Override
        public Fields getComponentOutputFields(String componentId, String streamId) {
            return new Fields("key", "value");
        }
    };
    return new TupleImpl(topologyContext, new Values(key, value), 1, "");
}
Also used : Fields(org.apache.storm.tuple.Fields) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) HashMap(java.util.HashMap) Config(org.apache.storm.Config) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 35 with Fields

use of org.apache.storm.tuple.Fields in project storm by apache.

the class TestHiveBolt method testWithoutPartitions.

@Test
public void testWithoutPartitions() throws Exception {
    HiveSetupUtil.dropDB(conf, dbName1);
    HiveSetupUtil.createDbAndTable(conf, dbName1, tblName1, null, colNames, colTypes, null, dbLocation);
    DelimitedRecordHiveMapper mapper = new DelimitedRecordHiveMapper().withColumnFields(new Fields(colNames));
    HiveOptions hiveOptions = new HiveOptions(metaStoreURI, dbName1, tblName1, mapper).withTxnsPerBatch(2).withBatchSize(2).withAutoCreatePartitions(false);
    bolt = new HiveBolt(hiveOptions);
    bolt.prepare(config, null, collector);
    Integer id = 100;
    String msg = "test-123";
    String city = "sunnyvale";
    String state = "ca";
    checkRecordCountInTable(tblName1, dbName1, 0);
    Set<Tuple> tupleSet = new HashSet<Tuple>();
    for (int i = 0; i < 4; i++) {
        Tuple tuple = generateTestTuple(id, msg, city, state);
        bolt.execute(tuple);
        tupleSet.add(tuple);
    }
    for (Tuple t : tupleSet) verify(collector).ack(t);
    bolt.cleanup();
    checkRecordCountInTable(tblName1, dbName1, 4);
}
Also used : Fields(org.apache.storm.tuple.Fields) DelimitedRecordHiveMapper(org.apache.storm.hive.bolt.mapper.DelimitedRecordHiveMapper) HiveOptions(org.apache.storm.hive.common.HiveOptions) Tuple(org.apache.storm.tuple.Tuple) HashSet(java.util.HashSet) Test(org.junit.Test)

Aggregations

Fields (org.apache.storm.tuple.Fields)170 Test (org.junit.Test)44 Values (org.apache.storm.tuple.Values)38 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)36 TridentTopology (org.apache.storm.trident.TridentTopology)32 HashMap (java.util.HashMap)31 Config (org.apache.storm.Config)31 Stream (org.apache.storm.trident.Stream)25 LocalCluster (org.apache.storm.LocalCluster)19 LocalTopology (org.apache.storm.LocalCluster.LocalTopology)17 TridentState (org.apache.storm.trident.TridentState)17 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)16 ArrayList (java.util.ArrayList)14 Map (java.util.Map)14 HiveOptions (org.apache.storm.hive.common.HiveOptions)14 AbstractTest (org.apache.flink.storm.util.AbstractTest)13 DelimitedRecordHiveMapper (org.apache.storm.hive.bolt.mapper.DelimitedRecordHiveMapper)12 IRichBolt (org.apache.storm.topology.IRichBolt)12 StateFactory (org.apache.storm.trident.state.StateFactory)12 Tuple (org.apache.storm.tuple.Tuple)12