Search in sources :

Example 11 with TriggerOption

use of org.apache.apex.malhar.lib.window.TriggerOption in project apex-malhar by apache.

the class TwitterAutoComplete method populateDAG.

/**
 * Populate the dag with High-Level API.
 * @param dag
 * @param conf
 */
@Override
public void populateDAG(DAG dag, Configuration conf) {
    TwitterSampleInput input = new TwitterSampleInput();
    WindowOption windowOption = new WindowOption.GlobalWindow();
    ApexStream<String> tags = StreamFactory.fromInput(input, input.text, name("tweetSampler")).filter(new ASCIIFilter(), name("ACSII Filter")).flatMap(new ExtractHashtags(), name("Extract Hashtags"));
    ApexStream<Tuple.WindowedTuple<KeyValPair<String, List<CompletionCandidate>>>> s = tags.window(windowOption, new TriggerOption().accumulatingFiredPanes().withEarlyFiringsAtEvery(Duration.standardSeconds(10))).addCompositeStreams(ComputeTopCompletions.top(10, true)).print();
    s.populateDag(dag);
}
Also used : TriggerOption(org.apache.apex.malhar.lib.window.TriggerOption) WindowOption(org.apache.apex.malhar.lib.window.WindowOption) TwitterSampleInput(org.apache.apex.malhar.contrib.twitter.TwitterSampleInput) List(java.util.List) LinkedList(java.util.LinkedList)

Aggregations

TriggerOption (org.apache.apex.malhar.lib.window.TriggerOption)11 WindowOption (org.apache.apex.malhar.lib.window.WindowOption)11 List (java.util.List)4 Function (org.apache.apex.malhar.lib.function.Function)4 KeyValPair (org.apache.apex.malhar.lib.util.KeyValPair)3 ConsoleOutputOperator (org.apache.apex.malhar.lib.io.ConsoleOutputOperator)2 ArrayList (java.util.ArrayList)1 LinkedList (java.util.LinkedList)1 TwitterSampleInput (org.apache.apex.malhar.contrib.twitter.TwitterSampleInput)1 JdbcPOJOInputOperator (org.apache.apex.malhar.lib.db.jdbc.JdbcPOJOInputOperator)1 JdbcPOJOInsertOutputOperator (org.apache.apex.malhar.lib.db.jdbc.JdbcPOJOInsertOutputOperator)1 JdbcStore (org.apache.apex.malhar.lib.db.jdbc.JdbcStore)1 JdbcTransactionalStore (org.apache.apex.malhar.lib.db.jdbc.JdbcTransactionalStore)1 Tuple (org.apache.apex.malhar.lib.window.Tuple)1 WindowState (org.apache.apex.malhar.lib.window.WindowState)1 WindowedStorage (org.apache.apex.malhar.lib.window.WindowedStorage)1 PojoInnerJoin (org.apache.apex.malhar.lib.window.accumulation.PojoInnerJoin)1 RemoveDuplicates (org.apache.apex.malhar.lib.window.accumulation.RemoveDuplicates)1