Search in sources :

Example 1 with WindowsStateFactory

use of org.apache.storm.trident.windowing.WindowsStateFactory in project storm by apache.

the class Stream method window.

private Stream window(WindowConfig windowConfig, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields, boolean storeTuplesInStore) {
    projectionValidation(inputFields);
    windowConfig.validate();
    Fields fields = addTriggerField(functionFields);
    // when storeTuplesInStore is false then the given windowStoreFactory is only used to store triggers and
    // that store is passed to WindowStateUpdater to remove them after committing the batch.
    Stream stream = _topology.addSourcedNode(this, new ProcessorNode(_topology.getUniqueStreamId(), _name, fields, fields, new WindowTridentProcessor(windowConfig, _topology.getUniqueWindowId(), windowStoreFactory, inputFields, aggregator, storeTuplesInStore)));
    Stream effectiveStream = stream.project(functionFields);
    // create StateUpdater with the given windowStoreFactory to remove triggered aggregation results form store
    // when they are successfully processed.
    StateFactory stateFactory = new WindowsStateFactory();
    StateUpdater stateUpdater = new WindowsStateUpdater(windowStoreFactory);
    stream.partitionPersist(stateFactory, new Fields(WindowTridentProcessor.TRIGGER_FIELD_NAME), stateUpdater, new Fields());
    return effectiveStream;
}
Also used : Fields(org.apache.storm.tuple.Fields) ProcessorNode(org.apache.storm.trident.planner.ProcessorNode) StateFactory(org.apache.storm.trident.state.StateFactory) WindowsStateFactory(org.apache.storm.trident.windowing.WindowsStateFactory) WindowsStateUpdater(org.apache.storm.trident.windowing.WindowsStateUpdater) GroupedStream(org.apache.storm.trident.fluent.GroupedStream) IAggregatableStream(org.apache.storm.trident.fluent.IAggregatableStream) WindowTridentProcessor(org.apache.storm.trident.windowing.WindowTridentProcessor) WindowsStateFactory(org.apache.storm.trident.windowing.WindowsStateFactory) CombinerAggStateUpdater(org.apache.storm.trident.operation.impl.CombinerAggStateUpdater) ReducerAggStateUpdater(org.apache.storm.trident.operation.impl.ReducerAggStateUpdater) WindowsStateUpdater(org.apache.storm.trident.windowing.WindowsStateUpdater) StateUpdater(org.apache.storm.trident.state.StateUpdater)

Aggregations

GroupedStream (org.apache.storm.trident.fluent.GroupedStream)1 IAggregatableStream (org.apache.storm.trident.fluent.IAggregatableStream)1 CombinerAggStateUpdater (org.apache.storm.trident.operation.impl.CombinerAggStateUpdater)1 ReducerAggStateUpdater (org.apache.storm.trident.operation.impl.ReducerAggStateUpdater)1 ProcessorNode (org.apache.storm.trident.planner.ProcessorNode)1 StateFactory (org.apache.storm.trident.state.StateFactory)1 StateUpdater (org.apache.storm.trident.state.StateUpdater)1 WindowTridentProcessor (org.apache.storm.trident.windowing.WindowTridentProcessor)1 WindowsStateFactory (org.apache.storm.trident.windowing.WindowsStateFactory)1 WindowsStateUpdater (org.apache.storm.trident.windowing.WindowsStateUpdater)1 Fields (org.apache.storm.tuple.Fields)1