Search in sources :

Example 81 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class StormTupleTest method testSelect.

@Test
public void testSelect() throws Exception {
    Tuple tuple = Tuple.getTupleClass(arity).newInstance();
    Values values = new Values();
    ArrayList<String> attributeNames = new ArrayList<String>(arity);
    ArrayList<String> filterNames = new ArrayList<String>(arity);
    for (int i = 0; i < arity; ++i) {
        tuple.setField(i, i);
        attributeNames.add("a" + i);
        if (r.nextBoolean()) {
            filterNames.add("a" + i);
            values.add(i);
        }
    }
    Fields schema = new Fields(attributeNames);
    Fields selector = new Fields(filterNames);
    Assert.assertEquals(values, new StormTuple<>(tuple, schema, -1, null, null, null).select(selector));
}
Also used : Fields(org.apache.storm.tuple.Fields) Values(org.apache.storm.tuple.Values) ArrayList(java.util.ArrayList) Tuple(org.apache.flink.api.java.tuple.Tuple) Test(org.junit.Test) AbstractTest(org.apache.flink.storm.util.AbstractTest)

Example 82 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class FlinkOutputFieldsDeclarerTest method testGetGroupingFieldIndexes.

@Test
public void testGetGroupingFieldIndexes() {
    final int numberOfAttributes = 5 + this.r.nextInt(20);
    final String[] attributes = new String[numberOfAttributes];
    for (int i = 0; i < numberOfAttributes; ++i) {
        attributes[i] = "a" + i;
    }
    final FlinkOutputFieldsDeclarer declarer = new FlinkOutputFieldsDeclarer();
    declarer.declare(new Fields(attributes));
    final int numberOfKeys = 1 + this.r.nextInt(24);
    final LinkedList<String> groupingFields = new LinkedList<String>();
    final boolean[] indexes = new boolean[numberOfAttributes];
    for (int i = 0; i < numberOfAttributes; ++i) {
        if (this.r.nextInt(25) < numberOfKeys) {
            groupingFields.add(attributes[i]);
            indexes[i] = true;
        } else {
            indexes[i] = false;
        }
    }
    final int[] expectedResult = new int[groupingFields.size()];
    int j = 0;
    for (int i = 0; i < numberOfAttributes; ++i) {
        if (indexes[i]) {
            expectedResult[j++] = i;
        }
    }
    final int[] result = declarer.getGroupingFieldIndexes(Utils.DEFAULT_STREAM_ID, groupingFields);
    Assert.assertEquals(expectedResult.length, result.length);
    for (int i = 0; i < expectedResult.length; ++i) {
        Assert.assertEquals(expectedResult[i], result[i]);
    }
}
Also used : Fields(org.apache.storm.tuple.Fields) LinkedList(java.util.LinkedList) AbstractTest(org.apache.flink.storm.util.AbstractTest) Test(org.junit.Test)

Example 83 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class FlinkTopologyTest method testFieldsGroupingOnMultipleSpoutOutputStreams.

@Test
public void testFieldsGroupingOnMultipleSpoutOutputStreams() {
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("spout", new TestDummySpout());
    builder.setBolt("sink", new TestSink()).fieldsGrouping("spout", TestDummySpout.spoutStreamId, new Fields("id"));
    FlinkTopology.createTopology(builder);
}
Also used : Fields(org.apache.storm.tuple.Fields) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) TestSink(org.apache.flink.storm.util.TestSink) TestDummySpout(org.apache.flink.storm.util.TestDummySpout) Test(org.junit.Test)

Example 84 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class FlinkTopologyTest method testFieldsGroupingOnMultipleBoltOutputStreams.

@Test
public void testFieldsGroupingOnMultipleBoltOutputStreams() {
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("spout", new TestDummySpout());
    builder.setBolt("bolt", new TestDummyBolt()).shuffleGrouping("spout");
    builder.setBolt("sink", new TestSink()).fieldsGrouping("bolt", TestDummyBolt.groupingStreamId, new Fields("id"));
    FlinkTopology.createTopology(builder);
}
Also used : Fields(org.apache.storm.tuple.Fields) TestDummyBolt(org.apache.flink.storm.util.TestDummyBolt) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) TestSink(org.apache.flink.storm.util.TestSink) TestDummySpout(org.apache.flink.storm.util.TestDummySpout) Test(org.junit.Test)

Example 85 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class TestDummyBolt method declareOutputFields.

@Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
    declarer.declareStream(shuffleStreamId, new Fields("data"));
    declarer.declareStream(groupingStreamId, new Fields("id", "data"));
}
Also used : Fields(org.apache.storm.tuple.Fields)

Aggregations

Fields (org.apache.storm.tuple.Fields)170 Test (org.junit.Test)44 Values (org.apache.storm.tuple.Values)38 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)36 TridentTopology (org.apache.storm.trident.TridentTopology)32 HashMap (java.util.HashMap)31 Config (org.apache.storm.Config)31 Stream (org.apache.storm.trident.Stream)25 LocalCluster (org.apache.storm.LocalCluster)19 LocalTopology (org.apache.storm.LocalCluster.LocalTopology)17 TridentState (org.apache.storm.trident.TridentState)17 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)16 ArrayList (java.util.ArrayList)14 Map (java.util.Map)14 HiveOptions (org.apache.storm.hive.common.HiveOptions)14 AbstractTest (org.apache.flink.storm.util.AbstractTest)13 DelimitedRecordHiveMapper (org.apache.storm.hive.bolt.mapper.DelimitedRecordHiveMapper)12 IRichBolt (org.apache.storm.topology.IRichBolt)12 StateFactory (org.apache.storm.trident.state.StateFactory)12 Tuple (org.apache.storm.tuple.Tuple)12