Search in sources :

Example 6 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class BoltWrapperTest method testWrapperToManyAttributes1.

@Test(expected = IllegalArgumentException.class)
public void testWrapperToManyAttributes1() throws Exception {
    final SetupOutputFieldsDeclarer declarer = new SetupOutputFieldsDeclarer();
    final String[] schema = new String[26];
    for (int i = 0; i < schema.length; ++i) {
        schema[i] = "a" + i;
    }
    declarer.declare(new Fields(schema));
    PowerMockito.whenNew(SetupOutputFieldsDeclarer.class).withNoArguments().thenReturn(declarer);
    new BoltWrapper<Object, Object>(mock(IRichBolt.class));
}
Also used : IRichBolt(org.apache.storm.topology.IRichBolt) Fields(org.apache.storm.tuple.Fields) AbstractTest(org.apache.flink.storm.util.AbstractTest) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 7 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class SpoutWrapperTest method testRunExecuteFixedNumber.

@SuppressWarnings("unchecked")
@Test
public void testRunExecuteFixedNumber() throws Exception {
    final SetupOutputFieldsDeclarer declarer = new SetupOutputFieldsDeclarer();
    declarer.declare(new Fields("dummy"));
    PowerMockito.whenNew(SetupOutputFieldsDeclarer.class).withNoArguments().thenReturn(declarer);
    final StreamingRuntimeContext taskContext = mock(StreamingRuntimeContext.class);
    when(taskContext.getExecutionConfig()).thenReturn(mock(ExecutionConfig.class));
    when(taskContext.getTaskName()).thenReturn("name");
    final IRichSpout spout = mock(IRichSpout.class);
    final int numberOfCalls = this.r.nextInt(50);
    final SpoutWrapper<?> spoutWrapper = new SpoutWrapper<Object>(spout, numberOfCalls);
    spoutWrapper.setRuntimeContext(taskContext);
    spoutWrapper.run(mock(SourceContext.class));
    verify(spout, times(numberOfCalls)).nextTuple();
}
Also used : Fields(org.apache.storm.tuple.Fields) IRichSpout(org.apache.storm.topology.IRichSpout) StreamingRuntimeContext(org.apache.flink.streaming.api.operators.StreamingRuntimeContext) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) SourceContext(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext) AbstractTest(org.apache.flink.storm.util.AbstractTest) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 8 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class WrapperSetupHelperTest method testRawType.

@Test(expected = IllegalArgumentException.class)
public void testRawType() throws Exception {
    IComponent boltOrSpout;
    if (this.r.nextBoolean()) {
        boltOrSpout = mock(IRichSpout.class);
    } else {
        boltOrSpout = mock(IRichBolt.class);
    }
    final SetupOutputFieldsDeclarer declarer = new SetupOutputFieldsDeclarer();
    declarer.declare(new Fields("dummy1", "dummy2"));
    PowerMockito.whenNew(SetupOutputFieldsDeclarer.class).withNoArguments().thenReturn(declarer);
    WrapperSetupHelper.getNumberOfAttributes(boltOrSpout, new HashSet<String>(singleton(Utils.DEFAULT_STREAM_ID)));
}
Also used : IRichBolt(org.apache.storm.topology.IRichBolt) IRichSpout(org.apache.storm.topology.IRichSpout) Fields(org.apache.storm.tuple.Fields) IComponent(org.apache.storm.topology.IComponent) Test(org.junit.Test) AbstractTest(org.apache.flink.storm.util.AbstractTest) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest)

Example 9 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class StormFieldsGroupingITCase method testProgram.

@Override
protected void testProgram() throws Exception {
    final String[] tokens = this.resultPath.split(":");
    final String outputFile = tokens[tokens.length - 1];
    final TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout(spoutId, new FiniteRandomSpout(0, 10, 2));
    builder.setBolt(boltId, new TaskIdBolt(), 2).fieldsGrouping(spoutId, FiniteRandomSpout.STREAM_PREFIX + 0, new Fields("number"));
    builder.setBolt(sinkId, new BoltFileSink(outputFile)).shuffleGrouping(boltId);
    final FlinkLocalCluster cluster = FlinkLocalCluster.getLocalCluster();
    Config conf = new Config();
    // only required to stabilize integration test
    conf.put(FlinkLocalCluster.SUBMIT_BLOCKING, true);
    cluster.submitTopology(topologyId, conf, FlinkTopology.createTopology(builder));
    cluster.shutdown();
}
Also used : BoltFileSink(org.apache.flink.storm.util.BoltFileSink) Fields(org.apache.storm.tuple.Fields) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) FlinkLocalCluster(org.apache.flink.storm.api.FlinkLocalCluster) Config(org.apache.storm.Config) FiniteRandomSpout(org.apache.flink.storm.tests.operators.FiniteRandomSpout) TaskIdBolt(org.apache.flink.storm.tests.operators.TaskIdBolt)

Example 10 with Fields

use of org.apache.storm.tuple.Fields in project flink by apache.

the class StormTupleTest method testGetByField.

private <T> StormTuple<?> testGetByField(int arity, int index, T value) throws Exception {
    assert (index < arity);
    Tuple tuple = Tuple.getTupleClass(arity).newInstance();
    tuple.setField(value, index);
    ArrayList<String> attributeNames = new ArrayList<String>(arity);
    for (int i = 0; i < arity; ++i) {
        if (i == index) {
            attributeNames.add(fieldName);
        } else {
            attributeNames.add("" + i);
        }
    }
    Fields schema = new Fields(attributeNames);
    return new StormTuple<>(tuple, schema, -1, null, null, null);
}
Also used : Fields(org.apache.storm.tuple.Fields) ArrayList(java.util.ArrayList) Tuple(org.apache.flink.api.java.tuple.Tuple)

Aggregations

Fields (org.apache.storm.tuple.Fields)176 Test (org.junit.Test)44 Values (org.apache.storm.tuple.Values)38 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)36 HashMap (java.util.HashMap)32 TridentTopology (org.apache.storm.trident.TridentTopology)32 Config (org.apache.storm.Config)31 Stream (org.apache.storm.trident.Stream)25 LocalCluster (org.apache.storm.LocalCluster)19 LocalTopology (org.apache.storm.LocalCluster.LocalTopology)17 TridentState (org.apache.storm.trident.TridentState)17 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)16 Map (java.util.Map)15 ArrayList (java.util.ArrayList)14 HiveOptions (org.apache.storm.hive.common.HiveOptions)14 AbstractTest (org.apache.flink.storm.util.AbstractTest)13 DelimitedRecordHiveMapper (org.apache.storm.hive.bolt.mapper.DelimitedRecordHiveMapper)12 IRichBolt (org.apache.storm.topology.IRichBolt)12 StateFactory (org.apache.storm.trident.state.StateFactory)12 Tuple (org.apache.storm.tuple.Tuple)12