Search in sources :

Example 16 with MetricsRegistryMap

use of org.apache.samza.metrics.MetricsRegistryMap in project samza by apache.

the class TestOperatorImpls method testEmptyChain.

@Test
public void testEmptyChain() throws InvocationTargetException, IllegalAccessException {
    // test creation of empty chain
    MessageStreamImpl<TestMessageEnvelope> testStream = mock(MessageStreamImpl.class);
    TaskContext mockContext = mock(TaskContext.class);
    when(mockContext.getMetricsRegistry()).thenReturn(new MetricsRegistryMap());
    Config mockConfig = mock(Config.class);
    OperatorImplGraph opGraph = new OperatorImplGraph();
    RootOperatorImpl operatorChain = (RootOperatorImpl) createOpsMethod.invoke(opGraph, testStream, mockConfig, mockContext);
    assertTrue(operatorChain != null);
}
Also used : TestMessageEnvelope(org.apache.samza.operators.data.TestMessageEnvelope) TaskContext(org.apache.samza.task.TaskContext) Config(org.apache.samza.config.Config) MetricsRegistryMap(org.apache.samza.metrics.MetricsRegistryMap) Test(org.junit.Test)

Example 17 with MetricsRegistryMap

use of org.apache.samza.metrics.MetricsRegistryMap in project samza by apache.

the class TestOperatorImpls method testLinearChain.

@Test
public void testLinearChain() throws IllegalAccessException, InvocationTargetException {
    // test creation of linear chain
    StreamGraphImpl mockGraph = mock(StreamGraphImpl.class);
    MessageStreamImpl<TestMessageEnvelope> testInput = TestMessageStreamImplUtil.getMessageStreamImpl(mockGraph);
    TaskContext mockContext = mock(TaskContext.class);
    when(mockContext.getMetricsRegistry()).thenReturn(new MetricsRegistryMap());
    Config mockConfig = mock(Config.class);
    testInput.map(m -> m).window(Windows.keyedSessionWindow(TestMessageEnvelope::getKey, Duration.ofMinutes(10)));
    OperatorImplGraph opGraph = new OperatorImplGraph();
    RootOperatorImpl operatorChain = (RootOperatorImpl) createOpsMethod.invoke(opGraph, testInput, mockConfig, mockContext);
    Set<OperatorImpl> subsSet = (Set<OperatorImpl>) nextOperatorsField.get(operatorChain);
    assertEquals(subsSet.size(), 1);
    OperatorImpl<TestMessageEnvelope, TestMessageEnvelope> firstOpImpl = subsSet.iterator().next();
    Set<OperatorImpl> subsOps = (Set<OperatorImpl>) nextOperatorsField.get(firstOpImpl);
    assertEquals(subsOps.size(), 1);
    OperatorImpl wndOpImpl = subsOps.iterator().next();
    subsOps = (Set<OperatorImpl>) nextOperatorsField.get(wndOpImpl);
    assertEquals(subsOps.size(), 0);
}
Also used : StreamOperatorSpec(org.apache.samza.operators.spec.StreamOperatorSpec) Assert.assertNotSame(org.junit.Assert.assertNotSame) ArrayList(java.util.ArrayList) OperatorSpec(org.apache.samza.operators.spec.OperatorSpec) PartialJoinOperatorSpec(org.apache.samza.operators.spec.PartialJoinOperatorSpec) SinkOperatorSpec(org.apache.samza.operators.spec.SinkOperatorSpec) MessageStreamImpl(org.apache.samza.operators.MessageStreamImpl) Duration(java.time.Duration) TestOutputMessageEnvelope(org.apache.samza.operators.data.TestOutputMessageEnvelope) Method(java.lang.reflect.Method) Before(org.junit.Before) TestMessageEnvelope(org.apache.samza.operators.data.TestMessageEnvelope) WindowType(org.apache.samza.operators.windows.internal.WindowType) TaskContext(org.apache.samza.task.TaskContext) Windows(org.apache.samza.operators.windows.Windows) Iterator(java.util.Iterator) WindowOperatorSpec(org.apache.samza.operators.spec.WindowOperatorSpec) Set(java.util.Set) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) JoinFunction(org.apache.samza.operators.functions.JoinFunction) Field(java.lang.reflect.Field) FlatMapFunction(org.apache.samza.operators.functions.FlatMapFunction) StreamGraphImpl(org.apache.samza.operators.StreamGraphImpl) InvocationTargetException(java.lang.reflect.InvocationTargetException) SinkFunction(org.apache.samza.operators.functions.SinkFunction) Config(org.apache.samza.config.Config) TestMessageStreamImplUtil(org.apache.samza.operators.TestMessageStreamImplUtil) WindowInternal(org.apache.samza.operators.windows.internal.WindowInternal) Assert.assertEquals(org.junit.Assert.assertEquals) MetricsRegistryMap(org.apache.samza.metrics.MetricsRegistryMap) Mockito.mock(org.mockito.Mockito.mock) TaskContext(org.apache.samza.task.TaskContext) Set(java.util.Set) Config(org.apache.samza.config.Config) TestMessageEnvelope(org.apache.samza.operators.data.TestMessageEnvelope) StreamGraphImpl(org.apache.samza.operators.StreamGraphImpl) MetricsRegistryMap(org.apache.samza.metrics.MetricsRegistryMap) Test(org.junit.Test)

Example 18 with MetricsRegistryMap

use of org.apache.samza.metrics.MetricsRegistryMap in project samza by apache.

the class TestOperatorImpl method testOnTimerPropagatesResultsAndTimer.

@Test
public void testOnTimerPropagatesResultsAndTimer() {
    TaskContext mockTaskContext = mock(TaskContext.class);
    when(mockTaskContext.getMetricsRegistry()).thenReturn(new MetricsRegistryMap());
    Object mockTestOpImplOutput = mock(Object.class);
    OperatorImpl<Object, Object> opImpl = new TestOpImpl(mockTestOpImplOutput);
    opImpl.init(mock(Config.class), mockTaskContext);
    // register a couple of operators
    OperatorImpl mockNextOpImpl1 = mock(OperatorImpl.class);
    when(mockNextOpImpl1.getOperatorSpec()).thenReturn(new TestOpSpec());
    when(mockNextOpImpl1.handleMessage(anyObject(), anyObject(), anyObject())).thenReturn(Collections.emptyList());
    mockNextOpImpl1.init(mock(Config.class), mockTaskContext);
    opImpl.registerNextOperator(mockNextOpImpl1);
    OperatorImpl mockNextOpImpl2 = mock(OperatorImpl.class);
    when(mockNextOpImpl2.getOperatorSpec()).thenReturn(new TestOpSpec());
    when(mockNextOpImpl2.handleMessage(anyObject(), anyObject(), anyObject())).thenReturn(Collections.emptyList());
    mockNextOpImpl2.init(mock(Config.class), mockTaskContext);
    opImpl.registerNextOperator(mockNextOpImpl2);
    // send a timer tick to this operator
    MessageCollector mockCollector = mock(MessageCollector.class);
    TaskCoordinator mockCoordinator = mock(TaskCoordinator.class);
    opImpl.onTimer(mockCollector, mockCoordinator);
    // verify that it propagates its handleTimer results to next operators
    verify(mockNextOpImpl1, times(1)).handleMessage(mockTestOpImplOutput, mockCollector, mockCoordinator);
    verify(mockNextOpImpl2, times(1)).handleMessage(mockTestOpImplOutput, mockCollector, mockCoordinator);
    // verify that it propagates the timer tick to next operators
    verify(mockNextOpImpl1, times(1)).handleTimer(mockCollector, mockCoordinator);
    verify(mockNextOpImpl2, times(1)).handleTimer(mockCollector, mockCoordinator);
}
Also used : TaskContext(org.apache.samza.task.TaskContext) Config(org.apache.samza.config.Config) MessageCollector(org.apache.samza.task.MessageCollector) Matchers.anyObject(org.mockito.Matchers.anyObject) TaskCoordinator(org.apache.samza.task.TaskCoordinator) MetricsRegistryMap(org.apache.samza.metrics.MetricsRegistryMap) Test(org.junit.Test)

Example 19 with MetricsRegistryMap

use of org.apache.samza.metrics.MetricsRegistryMap in project samza by apache.

the class TestOperatorImpl method testOnMessagePropagatesResults.

@Test
public void testOnMessagePropagatesResults() {
    TaskContext mockTaskContext = mock(TaskContext.class);
    when(mockTaskContext.getMetricsRegistry()).thenReturn(new MetricsRegistryMap());
    Object mockTestOpImplOutput = mock(Object.class);
    OperatorImpl<Object, Object> opImpl = new TestOpImpl(mockTestOpImplOutput);
    opImpl.init(mock(Config.class), mockTaskContext);
    // register a couple of operators
    OperatorImpl mockNextOpImpl1 = mock(OperatorImpl.class);
    when(mockNextOpImpl1.getOperatorSpec()).thenReturn(new TestOpSpec());
    when(mockNextOpImpl1.handleMessage(anyObject(), anyObject(), anyObject())).thenReturn(Collections.emptyList());
    mockNextOpImpl1.init(mock(Config.class), mockTaskContext);
    opImpl.registerNextOperator(mockNextOpImpl1);
    OperatorImpl mockNextOpImpl2 = mock(OperatorImpl.class);
    when(mockNextOpImpl2.getOperatorSpec()).thenReturn(new TestOpSpec());
    when(mockNextOpImpl2.handleMessage(anyObject(), anyObject(), anyObject())).thenReturn(Collections.emptyList());
    mockNextOpImpl2.init(mock(Config.class), mockTaskContext);
    opImpl.registerNextOperator(mockNextOpImpl2);
    // send a message to this operator
    MessageCollector mockCollector = mock(MessageCollector.class);
    TaskCoordinator mockCoordinator = mock(TaskCoordinator.class);
    opImpl.onMessage(mock(Object.class), mockCollector, mockCoordinator);
    // verify that it propagates its handleMessage results to next operators
    verify(mockNextOpImpl1, times(1)).handleMessage(mockTestOpImplOutput, mockCollector, mockCoordinator);
    verify(mockNextOpImpl2, times(1)).handleMessage(mockTestOpImplOutput, mockCollector, mockCoordinator);
}
Also used : TaskContext(org.apache.samza.task.TaskContext) Config(org.apache.samza.config.Config) MessageCollector(org.apache.samza.task.MessageCollector) Matchers.anyObject(org.mockito.Matchers.anyObject) TaskCoordinator(org.apache.samza.task.TaskCoordinator) MetricsRegistryMap(org.apache.samza.metrics.MetricsRegistryMap) Test(org.junit.Test)

Example 20 with MetricsRegistryMap

use of org.apache.samza.metrics.MetricsRegistryMap in project samza by apache.

the class TestOperatorImpl method testMultipleInitShouldThrow.

@Test(expected = IllegalStateException.class)
public void testMultipleInitShouldThrow() {
    OperatorImpl<Object, Object> opImpl = new TestOpImpl(mock(Object.class));
    TaskContext mockTaskContext = mock(TaskContext.class);
    when(mockTaskContext.getMetricsRegistry()).thenReturn(new MetricsRegistryMap());
    opImpl.init(mock(Config.class), mockTaskContext);
    opImpl.init(mock(Config.class), mockTaskContext);
}
Also used : TaskContext(org.apache.samza.task.TaskContext) Config(org.apache.samza.config.Config) Matchers.anyObject(org.mockito.Matchers.anyObject) MetricsRegistryMap(org.apache.samza.metrics.MetricsRegistryMap) Test(org.junit.Test)

Aggregations

MetricsRegistryMap (org.apache.samza.metrics.MetricsRegistryMap)23 Config (org.apache.samza.config.Config)16 Test (org.junit.Test)15 TaskContext (org.apache.samza.task.TaskContext)9 MapConfig (org.apache.samza.config.MapConfig)6 JobConfig (org.apache.samza.config.JobConfig)5 Before (org.junit.Before)5 HashMap (java.util.HashMap)4 TestMessageEnvelope (org.apache.samza.operators.data.TestMessageEnvelope)4 SystemStreamPartition (org.apache.samza.system.SystemStreamPartition)4 Set (java.util.Set)3 StreamGraphImpl (org.apache.samza.operators.StreamGraphImpl)3 TestOutputMessageEnvelope (org.apache.samza.operators.data.TestOutputMessageEnvelope)3 JoinFunction (org.apache.samza.operators.functions.JoinFunction)3 Matchers.anyObject (org.mockito.Matchers.anyObject)3 Field (java.lang.reflect.Field)2 InvocationTargetException (java.lang.reflect.InvocationTargetException)2 Method (java.lang.reflect.Method)2 Duration (java.time.Duration)2 ArrayList (java.util.ArrayList)2