Search in sources :

Example 1 with GeneralTopologyContext

use of org.apache.storm.task.GeneralTopologyContext in project storm by apache.

the class TestSequenceFileBolt method generateTestTuple.

private Tuple generateTestTuple(Long key, String value) {
    TopologyBuilder builder = new TopologyBuilder();
    GeneralTopologyContext topologyContext = new GeneralTopologyContext(builder.createTopology(), new Config(), new HashMap<>(), new HashMap<>(), new HashMap<>(), "") {

        @Override
        public Fields getComponentOutputFields(String componentId, String streamId) {
            return new Fields("key", "value");
        }
    };
    return new TupleImpl(topologyContext, new Values(key, value), topologyContext.getComponentId(1), 1, "");
}
Also used : Fields(org.apache.storm.tuple.Fields) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) Config(org.apache.storm.Config) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 2 with GeneralTopologyContext

use of org.apache.storm.task.GeneralTopologyContext in project storm by apache.

the class AvroGenericRecordBoltTest method generateTestTuple.

private static Tuple generateTestTuple(GenericRecord record) {
    TopologyBuilder builder = new TopologyBuilder();
    GeneralTopologyContext topologyContext = new GeneralTopologyContext(builder.createTopology(), new Config(), new HashMap(), new HashMap(), new HashMap(), "") {

        @Override
        public Fields getComponentOutputFields(String componentId, String streamId) {
            return new Fields("record");
        }
    };
    return new TupleImpl(topologyContext, new Values(record), topologyContext.getComponentId(1), 1, "");
}
Also used : Fields(org.apache.storm.tuple.Fields) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) HashMap(java.util.HashMap) Config(org.apache.storm.Config) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 3 with GeneralTopologyContext

use of org.apache.storm.task.GeneralTopologyContext in project storm by apache.

the class TestHdfsBolt method generateTestTuple.

private Tuple generateTestTuple(Object id, Object msg, Object city, Object state) {
    TopologyBuilder builder = new TopologyBuilder();
    GeneralTopologyContext topologyContext = new GeneralTopologyContext(builder.createTopology(), new Config(), new HashMap<>(), new HashMap<>(), new HashMap<>(), "") {

        @Override
        public Fields getComponentOutputFields(String componentId, String streamId) {
            return new Fields("id", "msg", "city", "state");
        }
    };
    return new TupleImpl(topologyContext, new Values(id, msg, city, state), topologyContext.getComponentId(1), 1, "");
}
Also used : Fields(org.apache.storm.tuple.Fields) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) Config(org.apache.storm.Config) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 4 with GeneralTopologyContext

use of org.apache.storm.task.GeneralTopologyContext in project storm by apache.

the class EsTestUtil method generateTestTuple.

public static Tuple generateTestTuple(String source, String index, String type, String id) {
    TopologyBuilder builder = new TopologyBuilder();
    GeneralTopologyContext topologyContext = new GeneralTopologyContext(builder.createTopology(), new Config(), new HashMap<>(), new HashMap<>(), new HashMap<>(), "") {

        @Override
        public Fields getComponentOutputFields(String componentId, String streamId) {
            return new Fields("source", "index", "type", "id");
        }
    };
    return new TupleImpl(topologyContext, new Values(source, index, type, id), source, 1, "");
}
Also used : Fields(org.apache.storm.tuple.Fields) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) Config(org.apache.storm.Config) Values(org.apache.storm.tuple.Values) TupleImpl(org.apache.storm.tuple.TupleImpl)

Example 5 with GeneralTopologyContext

use of org.apache.storm.task.GeneralTopologyContext in project storm by apache.

the class ExecutorTransferMultiThreadingTest method setup.

@Before
public void setup() throws NoSuchFieldException {
    topoConf = Utils.readStormConfig();
    String topologyId = "multi-threaded-topo-test";
    StormTopology stormTopology = createStormTopology();
    WorkerTopologyContext workerTopologyContext = mock(WorkerTopologyContext.class);
    when(workerTopologyContext.getRawTopology()).thenReturn(stormTopology);
    when(workerTopologyContext.getComponentId(selfTaskId)).thenReturn(sourceComp);
    when(workerTopologyContext.getComponentId(remoteTaskId)).thenReturn(destComp);
    workerState = mock(WorkerState.class);
    when(workerState.getWorkerTopologyContext()).thenReturn(workerTopologyContext);
    Map<Integer, JCQueue> receiveQMap = new HashMap<>();
    // local recvQ is not important in this test; simple mock it
    receiveQMap.put(selfTaskId, mock(JCQueue.class));
    when(workerState.getLocalReceiveQueues()).thenReturn(receiveQMap);
    when(workerState.getTopologyId()).thenReturn(topologyId);
    when(workerState.getPort()).thenReturn(6701);
    when(workerState.getMetricRegistry()).thenReturn(new StormMetricRegistry());
    when(workerState.tryTransferRemote(any(), any(), any())).thenCallRealMethod();
    // the actual worker transfer queue to be used in this test
    // taskId for worker transfer queue should be -1.
    // But there is already one worker transfer queue initialized by WorkerTransfer class (taskId=-1).
    // However the taskId is only used for metrics and it is not important here. Making it -100 to avoid collision.
    transferQueue = new JCQueue("worker-transfer-queue", "worker-transfer-queue", 1024, 0, 1, new WaitStrategyPark(100), workerState.getTopologyId(), Constants.SYSTEM_COMPONENT_ID, Collections.singletonList(-100), workerState.getPort(), workerState.getMetricRegistry());
    // Replace the transferQueue inside WorkerTransfer (inside WorkerState) with the customized transferQueue to be used in this test
    WorkerTransfer workerTransfer = new WorkerTransfer(workerState, topoConf, 2);
    FieldSetter.setField(workerTransfer, workerTransfer.getClass().getDeclaredField("transferQueue"), transferQueue);
    FieldSetter.setField(workerState, workerState.getClass().getDeclaredField("workerTransfer"), workerTransfer);
    generalTopologyContext = mock(GeneralTopologyContext.class);
}
Also used : WorkerTopologyContext(org.apache.storm.task.WorkerTopologyContext) GeneralTopologyContext(org.apache.storm.task.GeneralTopologyContext) JCQueue(org.apache.storm.utils.JCQueue) HashMap(java.util.HashMap) StormTopology(org.apache.storm.generated.StormTopology) WaitStrategyPark(org.apache.storm.policy.WaitStrategyPark) StormMetricRegistry(org.apache.storm.metrics2.StormMetricRegistry) WorkerState(org.apache.storm.daemon.worker.WorkerState) WorkerTransfer(org.apache.storm.daemon.worker.WorkerTransfer) Before(org.junit.Before)

Aggregations

GeneralTopologyContext (org.apache.storm.task.GeneralTopologyContext)10 Fields (org.apache.storm.tuple.Fields)9 Config (org.apache.storm.Config)7 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)7 TupleImpl (org.apache.storm.tuple.TupleImpl)7 Values (org.apache.storm.tuple.Values)7 HashMap (java.util.HashMap)4 StormTopology (org.apache.storm.generated.StormTopology)3 Before (org.junit.Before)3 ImmutableMap (com.google.common.collect.ImmutableMap)2 Map (java.util.Map)2 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)2 HashSet (java.util.HashSet)1 WorkerState (org.apache.storm.daemon.worker.WorkerState)1 WorkerTransfer (org.apache.storm.daemon.worker.WorkerTransfer)1 StormMetricRegistry (org.apache.storm.metrics2.StormMetricRegistry)1 WaitStrategyPark (org.apache.storm.policy.WaitStrategyPark)1 WorkerTopologyContext (org.apache.storm.task.WorkerTopologyContext)1 JCQueue (org.apache.storm.utils.JCQueue)1