Search in sources :

Example 1 with FoldLeftFunction

use of org.apache.samza.operators.functions.FoldLeftFunction in project samza by apache.

the class TestOperatorSpecs method testCreateWindowOperatorWithRelaxedTypes.

@Test
public void testCreateWindowOperatorWithRelaxedTypes() throws Exception {
    Function<TestMessageEnvelope, String> keyExtractor = m -> m.getKey();
    FoldLeftFunction<TestMessageEnvelope, Integer> aggregator = (m, c) -> c + 1;
    Supplier<Integer> initialValue = () -> 0;
    //instantiate a window using reflection
    WindowInternal<TestInputMessageEnvelope, String, Integer> window = new WindowInternal(null, initialValue, aggregator, keyExtractor, null, WindowType.TUMBLING);
    MessageStreamImpl<WindowPane<String, Integer>> mockWndOut = mock(MessageStreamImpl.class);
    WindowOperatorSpec spec = OperatorSpecs.createWindowOperatorSpec(window, mockWndOut, 1);
    assertEquals(spec.getWindow(), window);
    assertEquals(spec.getWindow().getKeyExtractor(), keyExtractor);
    assertEquals(spec.getWindow().getFoldLeftFunction(), aggregator);
    // make sure that the functions with relaxed types work as expected
    TestInputMessageEnvelope inputMsg = new TestInputMessageEnvelope("test-input-key1", "test-value-1", 23456L, "input-id-1");
    assertEquals("test-input-key1", spec.getWindow().getKeyExtractor().apply(inputMsg));
    assertEquals(1, spec.getWindow().getFoldLeftFunction().apply(inputMsg, 0));
}
Also used : PartialJoinFunction(org.apache.samza.operators.functions.PartialJoinFunction) TestInputMessageEnvelope(org.apache.samza.operators.data.TestInputMessageEnvelope) Function(java.util.function.Function) Supplier(java.util.function.Supplier) WindowPane(org.apache.samza.operators.windows.WindowPane) ArrayList(java.util.ArrayList) OutputStreamInternalImpl(org.apache.samza.operators.stream.OutputStreamInternalImpl) MessageCollector(org.apache.samza.task.MessageCollector) MessageStreamImpl(org.apache.samza.operators.MessageStreamImpl) SystemStream(org.apache.samza.system.SystemStream) Mockito.doAnswer(org.mockito.Mockito.doAnswer) TestOutputMessageEnvelope(org.apache.samza.operators.data.TestOutputMessageEnvelope) TestMessageEnvelope(org.apache.samza.operators.data.TestMessageEnvelope) WindowType(org.apache.samza.operators.windows.internal.WindowType) MessageType(org.apache.samza.operators.data.MessageType) Assert.assertNotNull(org.junit.Assert.assertNotNull) Collection(java.util.Collection) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) FlatMapFunction(org.apache.samza.operators.functions.FlatMapFunction) TaskCoordinator(org.apache.samza.task.TaskCoordinator) StreamGraphImpl(org.apache.samza.operators.StreamGraphImpl) Matchers.any(org.mockito.Matchers.any) List(java.util.List) SinkFunction(org.apache.samza.operators.functions.SinkFunction) OutgoingMessageEnvelope(org.apache.samza.system.OutgoingMessageEnvelope) TestMessageStreamImplUtil(org.apache.samza.operators.TestMessageStreamImplUtil) WindowInternal(org.apache.samza.operators.windows.internal.WindowInternal) FoldLeftFunction(org.apache.samza.operators.functions.FoldLeftFunction) Assert.assertEquals(org.junit.Assert.assertEquals) Mockito.mock(org.mockito.Mockito.mock) WindowInternal(org.apache.samza.operators.windows.internal.WindowInternal) TestInputMessageEnvelope(org.apache.samza.operators.data.TestInputMessageEnvelope) WindowPane(org.apache.samza.operators.windows.WindowPane) TestMessageEnvelope(org.apache.samza.operators.data.TestMessageEnvelope) Test(org.junit.Test)

Example 2 with FoldLeftFunction

use of org.apache.samza.operators.functions.FoldLeftFunction in project samza by apache.

the class TestOperatorSpecs method testCreateWindowOperator.

@Test
public void testCreateWindowOperator() throws Exception {
    Function<TestMessageEnvelope, String> keyExtractor = m -> "globalkey";
    FoldLeftFunction<TestMessageEnvelope, Integer> aggregator = (m, c) -> c + 1;
    Supplier<Integer> initialValue = () -> 0;
    //instantiate a window using reflection
    WindowInternal window = new WindowInternal(null, initialValue, aggregator, keyExtractor, null, WindowType.TUMBLING);
    MessageStreamImpl<WindowPane<String, Integer>> mockWndOut = mock(MessageStreamImpl.class);
    WindowOperatorSpec spec = OperatorSpecs.<TestMessageEnvelope, String, Integer>createWindowOperatorSpec(window, mockWndOut, 1);
    assertEquals(spec.getWindow(), window);
    assertEquals(spec.getWindow().getKeyExtractor(), keyExtractor);
    assertEquals(spec.getWindow().getFoldLeftFunction(), aggregator);
}
Also used : PartialJoinFunction(org.apache.samza.operators.functions.PartialJoinFunction) TestInputMessageEnvelope(org.apache.samza.operators.data.TestInputMessageEnvelope) Function(java.util.function.Function) Supplier(java.util.function.Supplier) WindowPane(org.apache.samza.operators.windows.WindowPane) ArrayList(java.util.ArrayList) OutputStreamInternalImpl(org.apache.samza.operators.stream.OutputStreamInternalImpl) MessageCollector(org.apache.samza.task.MessageCollector) MessageStreamImpl(org.apache.samza.operators.MessageStreamImpl) SystemStream(org.apache.samza.system.SystemStream) Mockito.doAnswer(org.mockito.Mockito.doAnswer) TestOutputMessageEnvelope(org.apache.samza.operators.data.TestOutputMessageEnvelope) TestMessageEnvelope(org.apache.samza.operators.data.TestMessageEnvelope) WindowType(org.apache.samza.operators.windows.internal.WindowType) MessageType(org.apache.samza.operators.data.MessageType) Assert.assertNotNull(org.junit.Assert.assertNotNull) Collection(java.util.Collection) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) FlatMapFunction(org.apache.samza.operators.functions.FlatMapFunction) TaskCoordinator(org.apache.samza.task.TaskCoordinator) StreamGraphImpl(org.apache.samza.operators.StreamGraphImpl) Matchers.any(org.mockito.Matchers.any) List(java.util.List) SinkFunction(org.apache.samza.operators.functions.SinkFunction) OutgoingMessageEnvelope(org.apache.samza.system.OutgoingMessageEnvelope) TestMessageStreamImplUtil(org.apache.samza.operators.TestMessageStreamImplUtil) WindowInternal(org.apache.samza.operators.windows.internal.WindowInternal) FoldLeftFunction(org.apache.samza.operators.functions.FoldLeftFunction) Assert.assertEquals(org.junit.Assert.assertEquals) Mockito.mock(org.mockito.Mockito.mock) TestMessageEnvelope(org.apache.samza.operators.data.TestMessageEnvelope) WindowInternal(org.apache.samza.operators.windows.internal.WindowInternal) WindowPane(org.apache.samza.operators.windows.WindowPane) Test(org.junit.Test)

Aggregations

ArrayList (java.util.ArrayList)2 Collection (java.util.Collection)2 List (java.util.List)2 Function (java.util.function.Function)2 Supplier (java.util.function.Supplier)2 MessageStreamImpl (org.apache.samza.operators.MessageStreamImpl)2 StreamGraphImpl (org.apache.samza.operators.StreamGraphImpl)2 TestMessageStreamImplUtil (org.apache.samza.operators.TestMessageStreamImplUtil)2 MessageType (org.apache.samza.operators.data.MessageType)2 TestInputMessageEnvelope (org.apache.samza.operators.data.TestInputMessageEnvelope)2 TestMessageEnvelope (org.apache.samza.operators.data.TestMessageEnvelope)2 TestOutputMessageEnvelope (org.apache.samza.operators.data.TestOutputMessageEnvelope)2 FlatMapFunction (org.apache.samza.operators.functions.FlatMapFunction)2 FoldLeftFunction (org.apache.samza.operators.functions.FoldLeftFunction)2 PartialJoinFunction (org.apache.samza.operators.functions.PartialJoinFunction)2 SinkFunction (org.apache.samza.operators.functions.SinkFunction)2 OutputStreamInternalImpl (org.apache.samza.operators.stream.OutputStreamInternalImpl)2 WindowPane (org.apache.samza.operators.windows.WindowPane)2 WindowInternal (org.apache.samza.operators.windows.internal.WindowInternal)2 WindowType (org.apache.samza.operators.windows.internal.WindowType)2