Search in sources :

Example 1 with GroupAlsoByWindowFnRunner

use of org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner in project beam by apache.

the class BatchGroupAlsoByWindowReshuffleDoFnTest method makeRunner.

private static <K, InputT, OutputT, W extends BoundedWindow> DoFnRunner<KV<K, Iterable<WindowedValue<InputT>>>, KV<K, OutputT>> makeRunner(GroupAlsoByWindowDoFnFactory<K, InputT, OutputT> fnFactory, WindowingStrategy<?, W> windowingStrategy, TupleTag<KV<K, OutputT>> outputTag, DoFnRunners.OutputManager outputManager) {
    final StepContext stepContext = new TestStepContext(STEP_NAME);
    StateInternalsFactory<K> stateInternalsFactory = key -> stepContext.stateInternals();
    BatchGroupAlsoByWindowFn<K, InputT, OutputT> fn = fnFactory.forStrategy(windowingStrategy, stateInternalsFactory);
    return new GroupAlsoByWindowFnRunner<>(PipelineOptionsFactory.create(), fn, NullSideInputReader.empty(), outputManager, outputTag, stepContext);
}
Also used : Arrays(java.util.Arrays) KV(org.apache.beam.sdk.values.KV) StateInternalsFactory(org.apache.beam.runners.core.StateInternalsFactory) WindowedValue(org.apache.beam.sdk.util.WindowedValue) InMemoryStateInternals(org.apache.beam.runners.core.InMemoryStateInternals) DoFnRunner(org.apache.beam.runners.core.DoFnRunner) Duration(org.joda.time.Duration) RunWith(org.junit.runner.RunWith) StepContext(org.apache.beam.runners.core.StepContext) TimerInternals(org.apache.beam.runners.core.TimerInternals) PipelineOptionsFactory(org.apache.beam.sdk.options.PipelineOptionsFactory) DoFnRunners(org.apache.beam.runners.core.DoFnRunners) GroupAlsoByWindowFnRunner(org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner) TupleTag(org.apache.beam.sdk.values.TupleTag) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) StateInternals(org.apache.beam.runners.core.StateInternals) ExpectedException(org.junit.rules.ExpectedException) PaneInfo(org.apache.beam.sdk.transforms.windowing.PaneInfo) NullSideInputReader(org.apache.beam.runners.core.NullSideInputReader) GroupAlsoByWindowDoFnFactory(org.apache.beam.runners.dataflow.worker.util.GroupAlsoByWindowProperties.GroupAlsoByWindowDoFnFactory) FixedWindows(org.apache.beam.sdk.transforms.windowing.FixedWindows) Test(org.junit.Test) JUnit4(org.junit.runners.JUnit4) List(java.util.List) Rule(org.junit.Rule) Matchers.contains(org.hamcrest.Matchers.contains) Matchers.equalTo(org.hamcrest.Matchers.equalTo) BoundedWindow(org.apache.beam.sdk.transforms.windowing.BoundedWindow) Instant(org.joda.time.Instant) IntervalWindow(org.apache.beam.sdk.transforms.windowing.IntervalWindow) WindowingStrategy(org.apache.beam.sdk.values.WindowingStrategy) StepContext(org.apache.beam.runners.core.StepContext) GroupAlsoByWindowFnRunner(org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner)

Aggregations

Arrays (java.util.Arrays)1 List (java.util.List)1 DoFnRunner (org.apache.beam.runners.core.DoFnRunner)1 DoFnRunners (org.apache.beam.runners.core.DoFnRunners)1 InMemoryStateInternals (org.apache.beam.runners.core.InMemoryStateInternals)1 NullSideInputReader (org.apache.beam.runners.core.NullSideInputReader)1 StateInternals (org.apache.beam.runners.core.StateInternals)1 StateInternalsFactory (org.apache.beam.runners.core.StateInternalsFactory)1 StepContext (org.apache.beam.runners.core.StepContext)1 TimerInternals (org.apache.beam.runners.core.TimerInternals)1 GroupAlsoByWindowFnRunner (org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner)1 GroupAlsoByWindowDoFnFactory (org.apache.beam.runners.dataflow.worker.util.GroupAlsoByWindowProperties.GroupAlsoByWindowDoFnFactory)1 PipelineOptionsFactory (org.apache.beam.sdk.options.PipelineOptionsFactory)1 BoundedWindow (org.apache.beam.sdk.transforms.windowing.BoundedWindow)1 FixedWindows (org.apache.beam.sdk.transforms.windowing.FixedWindows)1 IntervalWindow (org.apache.beam.sdk.transforms.windowing.IntervalWindow)1 PaneInfo (org.apache.beam.sdk.transforms.windowing.PaneInfo)1 WindowedValue (org.apache.beam.sdk.util.WindowedValue)1 KV (org.apache.beam.sdk.values.KV)1 TupleTag (org.apache.beam.sdk.values.TupleTag)1